AI server for courts and the judiciary: anonymised judgments, hearing transcripts, triage

What an AI server does for courts and the judiciary

An AI server for courts runs the language and speech models of a court, a court administration or a justice ministry's IT centre on hardware the judiciary controls. It anonymises judgments before they are published, turns hearing recordings into text, sorts and extracts incoming claims when thousands of similar cases arrive, and lets judges search the court's own decisions in plain language. A server with two NVIDIA RTX PRO 6000 Blackwell Server Edition cards carries all four for one court; four cards per server, and two such servers, suit an IT centre that serves many courts.

The first sections describe the work and its sizing for court presidents and justice administrators; the AI Act and data-protection points follow, then the configurations and the notes for the IT team. Figures are published measurements with their source, or our estimates, labelled as such. This page covers the judiciary; authorities and municipalities are the subject of our guide to GPU servers for public administration, and the advocate's side is in our guide to a private LLM for law firms.

Four court workloads and what sizes each

Our summary. Model facts from the model cards and the Open ASR Leaderboard paper as reported in our Whisper guide; sizing in the sections below.

Three of the four are batch work. A judgment waits for publication for days, a recording for its transcript for hours, and a pile of new claims can be sorted overnight. Only the search for judges is interactive and needs answers within seconds. That lets one server do the batch jobs by night on cards that serve the judges by day, as our guide to batch LLM inference on-premise explains for document archives.

Anonymising judgments for publication

Courts publish only part of their decisions. LTO reported on 20 May 2026 that Bavaria is starting "Projekt 50k" to publish 50,000 judgments anonymised by AI over the coming years, and cited estimates that only about 3 per cent of German judgments are published. The tool, ALeKS, grew out of research with the University of Erlangen-Nuremberg, Lower Saxony helped develop the prototype further, and the Higher Regional Court of Munich has tested it since March 2026. It removes names, dates of birth and addresses. A judge decides whether a judgment is published, and court staff check every anonymised decision by hand, because a combination of occupations, places or awards can still identify a person. The report does not say which model the tool uses or where it runs.

A workable pipeline has three steps. Rules and patterns catch what has a fixed form: case numbers of other proceedings, dates, postcodes, account numbers and vehicle plates. A language model reads the judgment and returns every remaining mention of a person, a company, a place or an occupation as a list with its category and position, constrained to a JSON schema with vLLM's structured outputs. Code then replaces each mention consistently, so that the same party is "A" throughout, and a reviewer sees the marked text and accepts or corrects it. The model never writes the published text itself, which keeps the judge's wording intact and makes the review fast.

That choice also decides the GPU time, because a language model spends most of its time generating output tokens. An entity list is a few hundred tokens per judgment, a rewritten judgment as many tokens as the judgment has. Our guide to PII redaction before an LLM compares rule-based, small-model and LLM-based detection; the court case adds indirect identification, which is why the reviewer stays in the loop.

Sizing anonymisation by judgments per year

MLPerf® Inference v6.1 publishes offline throughput for eight RTX PRO 6000 Server Edition cards: 15,256 tokens per second on gpt-oss-120b and 50,965.3 on Llama 3.1 8B in entry 6.1-0050. Divided by eight, that is about 1,907 and 6,371 tokens per second per card, our division and not an MLPerf metric. The benchmark inputs are not judgments, so this example plans with half the published rate, the planning margin of our batch inference guide, and counts output tokens only.

Our arithmetic for 50,000 judgments at half the per-card rate. MLPerf® Inference: Datacenter v6.1, closed division, available category, offline scenario, gpt-oss-120b and Llama 3.1 8B, entry 6.1-0050, HPE ProLiant Compute DL380a Gen12 with eight RTX PRO 6000 Blackwell Server Edition, retrieved from mlcommons.org on 24 October 2026. Result verified by MLCommons Association. Per card is our division of the system result, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

With the entity-list approach, the 50,000 judgments of the Bavarian project fit into one night on one card with gpt-oss-120b, even at half the published rate. Reading a long judgment also takes compute before the first output token, which these figures do not include, so time a sample of a few hundred real judgments before you fix the server. The memory side is relaxed: by our users-per-card rule, gpt-oss-120b leaves 22.2 GiB of cache on one card, and a judgment of up to 16,384 tokens takes 0.5625 GiB in a 16-bit cache, so 39 judgments are in flight at once. The model card of gpt-oss lists no languages, so test the model, or any candidate, on judgments in your own language, with your reviewers grading the result.

We build servers with two, four or eight RTX PRO 6000 Server Edition cards for this work; send us your judgments per year and the hours of recordings per day, and our team replies with a configuration.

Hearing recordings and transcripts

Where hearings are recorded, a transcript makes the recording searchable and gives the judge a draft to check. Whether and how a court records a hearing, and what status a transcript has, is a matter of national procedural law. Technically the load is small. OpenAI gives about 10 GB of GPU memory for Whisper large, and NVIDIA's Parakeet TDT 0.6B v3 has 600 million parameters and covers 25 European languages, as our guide to a Whisper speech-to-text server sets out.

Take 20 courtrooms with 4 recorded hours a day as an example, 80 hours of audio. On the A100 of the Open ASR Leaderboard paper, Whisper large-v3 reached an RTFx of 145.5, seconds of audio per second of computing, and Parakeet TDT 0.6B v3 3,333. By our arithmetic, the 80 hours are about 33 minutes of computing with Whisper and under 2 minutes with Parakeet. Speaker labels cost far more: in NVIDIA's ASR NIM figures for the L40S, adding speaker diarisation cut the offline RTFx from 3,638 to 101.5, which still turns the 80 hours into about 47 minutes. None of these figures was measured on an RTX PRO 6000 or on a MIG instance of one, so a test with your own recordings, with the microphones and room acoustics of your courtrooms, decides the setting. A 24 GB MIG instance holds either model by the published memory figures.

Triage in mass proceedings and search for judges

Some courts receive thousands of nearly identical claims in a few months. A language model can read each statement of claim and return the claim type, the parties, the dates and the amounts as fields, so that the registry can assign, bundle and schedule the cases. This runs as a batch job on the same card as the anonymisation, with output of tens of tokens per claim. A second use is search over the court's own decisions and the case file: the judge asks in plain language and receives the passages with their source. By our estimate from the figures in our law-firm guide, gpt-oss-120b holds 19 conversations at a 32K context on one card and about 93 on a tensor-parallel pair that shares one copy of the model.

These two uses sit closer to the judicial function than anonymisation does, which matters for the AI Act below. A triage tool that only fills in fields is one thing; a tool that suggests how a claim should be decided is another, and a court should draw that line before the pilot, with its legal department.

AI Act and GDPR points for courts

Annex III, point 8(a), of the AI Act lists as high-risk "AI systems intended to be used by a judicial authority or on their behalf to assist a judicial authority in researching and interpreting facts and the law and in applying the law to a concrete set of facts", or used in a similar way in alternative dispute resolution. Recital 61 adds that final decision-making "must remain a human-driven activity", and that the high-risk classification should not extend to systems for "purely ancillary administrative activities that do not affect the actual administration of justice in individual cases, such as anonymisation or pseudonymisation of judicial decisions, documents or data, communication between personnel, administrative tasks."

Article 6(3) lets an Annex III system fall outside the high-risk class where it does not pose a significant risk of harm, for example where it is "intended to perform a narrow procedural task" or "a preparatory task to an assessment". An Annex III system that performs profiling of natural persons is always considered high-risk, and under Article 6(4) a provider who considers an Annex III system not high-risk documents that assessment before the system is placed on the market or put into service. As our article on EU AI Act deployer obligations sets out, Article 113 as amended by Regulation (EU) 2026/1744, the Digital Omnibus on AI, applies the high-risk rules to Annex III uses from 2 December 2027, and the AI literacy duty of Article 4 has applied since 2 February 2025.

Data protection law applies to every recording and every judgment on the platform. The GDPR's Article 55(3) states that supervisory authorities "shall not be competent to supervise processing operations of courts acting in their judicial capacity", and Recital 20 gives the same reason; Article 23(1)(f) allows restrictions to protect "judicial independence and judicial proceedings". How this applies to the administrative side of a court, such as publication, is set by national law. This is general information on EU law as of October 2026, from the AI Act text on the Commission's AI Act Service Desk (consolidated version of 27 July 2026) and on EUR-Lex, and from the GDPR on EUR-Lex; national rules may differ, and your data-protection officer and legal department confirm how they apply. A data protection impact assessment is the usual place to record the decision.

Configuration options by card count

The options follow the reference configurations on our AI servers page and NVIDIA's guide for certified systems: two processors, at least six cores per GPU and system memory of twice the GPU memory.

Reference configurations on our AI servers page; sessions by our users-per-card rule with gpt-oss-120b in a 16-bit cache at 32K; batch figures from the table above; power by our estimate (cards at 600 W, processors, plus 15 per cent for the rest of the server).

Two cards. The entry server has two RTX PRO 6000 Server Edition, two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives. One card runs the language model for search during office hours and the anonymisation and triage queues at night. The other is split by MIG into four isolated 24 GB instances for speech recognition, speaker labels, the embedding model and reranker, and a test instance. It suits one court or a pilot.

Four cards. The inference node with 768 GB and four NVMe drives gives the judges a pair of their own and keeps a whole card for batch jobs, so publication and triage also run during the day. This is the size we would propose to a court administration that serves several courts.

Two servers or eight cards. A ministry IT centre that serves many courts plans two four-card servers that each carry the judges' peak, so that one can fail or be patched while the other answers, as our guide to LLM high availability with two GPU nodes describes. Eight cards in one server give the same capacity in one chassis, at about 6.3 kW and without failover.

The RTX PRO 6000 Max-Q is the workstation card, a 300 W blower card; in a rack it belongs only in a chassis whose maker lists it, and for continuous operation we recommend the Server Edition. We set up the models, the anonymisation pipeline and search through our Private AI/ML service; describe the workload you would pilot first, and we propose the server and the pilot.

Notes for the court IT centre

1. Tenants. When one platform serves several courts, give each court its own API keys, index and log, and keep the batch queues apart. An LLM gateway with keys and a query log in front of the engines records who asked what.
2. Review interface. The anonymisation result goes to a reviewer as marked text with each replacement visible. Log every accepted and corrected mention, so that the rules and the prompt can be improved from real corrections.
3. Integration. vLLM serves an OpenAI-compatible API with structured outputs, and its batch runner reads JSONL files, so the case-management system can hand over judgments as files at night. Keep the engines on an internal network: vLLM's --api-key protects only paths under /v1, /v2, /inference and /cohere, while routes such as /invocations and /metrics answer without it (vLLM 0.32.0 security guide).
4. Versions. Run vLLM 0.31.0 or later, as our overview of vLLM security advisories explains, keep the NVIDIA Container Toolkit at 1.17.8 or later and the driver at a release NVIDIA lists as fixed in its bulletin of 30 September 2026.
5. Software on delivery. The server arrives on request with the operating system, the NVIDIA driver, CUDA and a container runtime installed. Your own acceptance test, as in our guide to GPU server acceptance testing, can be part of the tender.
6. Tender. Most courts buy through a public tender. Our guide to the GPU server tender specification shows how to write memory, partitioning and power as measurable requirements.

Power and room

By our estimate, the two-card server draws about 1.8 kW at the wall with two EPYC 9135 processors and fits one 230 V 16 A feed, which carries 2.94 kW at 80 per cent. The four-card server draws about 3.4 kW with the cards at 600 W and needs 32 A or three-phase, or about 2.7 kW with the cards capped at 450 W. Eight cards draw about 6.3 kW by our estimate; Dell's datasheet gives about 8 kW per eight-card node as an indicative figure, and feeds should be planned on the maker's figure. These machines belong in the server room of the IT centre or in a data centre, as our guide to GPU server power on 16 A, 32 A and three-phase feeds works through; the estimate is confirmed by a measurement on the delivered system.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build the servers to order with two, four or eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, delivered anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service, with our engineering partner Vixen.UNO, deploys the models, the anonymisation pipeline, search and the query log on your servers. Our guide to buying a GPU server for LLMs lists every configuration.

FAQ