PII redaction before an LLM, on-premise: detection models, pseudonymisation and GPU sizing

Where redaction sits in a private AI platform

PII redaction on-premise means finding names, addresses, account numbers and other personal data in documents and prompts on your own server, and replacing them before the text is indexed or sent to a language model. Three methods do the finding: patterns for data with a fixed format, small PII detection models for names and places in free text, and an LLM for the cases the first two miss. The detection models are small enough that one NVIDIA L4 carries them; the GPU decision only grows when an LLM rewrites whole documents.

Redaction has two natural places. At ingestion for retrieval, it runs before documents are split, embedded and stored, so the vector index never holds the names; our guide to RAG on company data covers the rest of that pipeline. At the LLM gateway, it runs on every prompt before the model call and, if needed, on every answer after it. LiteLLM, for example, offers pre_call and post_call modes for masking, as our guide to an LLM gateway with keys, budgets and a query log describes. Redaction matters most on routes to external APIs; on a private route it can remove a name that a contract review needs, so the policy is set per route.

The replacement decides what you can do later. Masking writes a type label such as <PERSON> and keeps nothing. Pseudonymisation writes a consistent placeholder, <PERSON_1> for every mention of the same person, and keeps a mapping table, so an answer can be turned back into names for the user who is allowed to see them. Only the second keeps a summary readable when several people appear in one document.

Three ways to find personal data

Our summary. Failure modes of encoder models from the OpenAI Privacy Filter card ("missing uncommon names", over-redaction, fragmented spans, novel secret formats), read on 24 October 2026.

Patterns are exact and cheap, and they fail quietly on anything without a format. Encoder models classify every token of a text in one pass and return spans with a type and a score; they are fast, they need no prompt, and their result can be checked span by span. An LLM reads context: it can see that "the head of the only bakery in the village" identifies a person, which no pattern and few classifiers will. It is also the slowest option and the hardest to audit, because a name the model left in place leaves no trace in its output.

A practical order is patterns first, then an encoder model, then an LLM pass only on documents where the encoder's scores are uncertain or where indirect identification matters, such as court decisions. Our guide to an AI server for courts shows that case, where staff still check every anonymised decision.

Open PII detection models as of October 2026

Hugging Face cards and file lists of openai/privacy-filter, OpenMed/privacy-filter-multilingual-v2, fastino/gliner2-privacy-filter-PII-multi, nvidia/gliner-PII and iiiorg/piiranha-v1-detect-personal-information, read on 24 October 2026. Sizes as stated on each card; file sizes from the "Files" pages.

OpenAI's Privacy Filter is a bidirectional token classifier with eight transformer blocks and a sparse mixture of experts, 128 experts with four active per token, and a context of 128,000 tokens. Its eight categories are account numbers, private addresses, emails, persons, phone numbers, URLs, dates and secrets. The card calls it "a redaction and data minimization aid" and states that it is not an anonymisation or compliance tool; it names its failure modes and asks for in-domain evaluation and human review in medical, legal, financial, HR, education and government settings. It also warns that performance may drop on non-English text. OpenMed's fine-tune extends it to 54 categories in 16 languages, among them German, French, Italian, Spanish, Portuguese and Dutch, and states that German is among its strongest languages without publishing a German score.

Fastino's model, adapted from GLiNER2 with a 205M-parameter base, covers English, French, Spanish, German, Italian, Portuguese and Dutch, with 42 entity types from personal identity to tax identifiers, bank data and credentials. NVIDIA's gliner-PII, published on 28 October 2025, is English only. Piiranha covers six languages including German and reads at most 256 tokens at a time, but its CC BY-NC-ND 4.0 licence excludes commercial use, so it is not an option for a company pipeline.

The cards of all five give no throughput, latency or GPU memory figure. NVIDIA's card names an A100 as its test hardware and lists Ampere, Blackwell, Hopper and older GPUs as well as x86 CPUs; OpenAI's card states that its model runs "in a web browser or on a laptop".

What the published scores say

Model cards report their best numbers on the data they were trained near. NVIDIA's gliner-PII scores a strict F1 of 0.87 on NVIDIA's own Nemotron-PII set, 0.70 on Argilla PII and 0.64 on AI4Privacy. Fastino's paper of May 2026 compared five models on its SPY benchmark, exact-match spans in legal and medical documents.

Fastino Labs, "GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction", arXiv 2605.09973, Table 2, read on 24 October 2026. Benchmark and winning model come from the same authors; the paper reports no per-language scores and no throughput.

Two points matter for planning. Exact span matching is strict, so the absolute figures look low, and the ranking is the authors' own. More useful is the shape: on the legal set, OpenAI's Privacy Filter reached a recall of 0.640 at a precision of 0.250, so it removes much of the personal data and a lot of other text with it; Fastino's model reached a recall of 0.722 at 0.354. For redaction, recall is the figure that protects people, and precision is the figure that keeps the text useful. Which balance you need depends on the route: before an external API, recall; inside a private platform, both.

We supply L4 cards for servers you already run and build GPU servers for private AI platforms; send us your document types, languages and daily volume, and our team replies with a model choice to test and the card it needs.

Pseudonymisation or anonymisation under the GDPR

The GDPR uses the two words for different results. Article 4(5) defines pseudonymisation as processing personal data so that they "can no longer be attributed to a specific data subject without the use of additional information", provided that this information "is kept separately" and protected by technical and organisational measures. Recital 26 states that pseudonymised data which could be attributed to a person by such additional information "should be considered to be information on an identifiable natural person", and that the principles of data protection do not apply to anonymous information, where the person "is not or no longer identifiable". Whether someone is identifiable is judged by "all the means reasonably likely to be used", taking into account costs, time and available technology.

For a redaction pipeline this means: placeholders with a mapping table produce pseudonymised data, and the index or prompt log built from them is still personal data. Recital 28 adds that pseudonymisation "can reduce the risks to the data subjects", and Articles 25(1) and 32(1)(a) name it as a measure for data protection by design and for the security of processing. Removing every identifier and deleting the mapping moves towards anonymisation, but indirect identifiers, a job title, a place, a rare event, can still point to one person, which is why the courts example keeps a reviewer. The European Data Protection Board lists its Guidelines 01/2025 on Pseudonymisation, dated 17 January 2025, on its guidelines page.

Store the mapping table on a different system or under a different key from the redacted text, with access limited to the service that restores names for users entitled to see them. What counts as sufficient is decided by your data-protection officer; our guide to a DPIA for an internal LLM shows where redaction appears in the assessment. This is general information as of October 2026, read from the GDPR text on EUR-Lex, not legal advice.

German names, addresses and identifiers

German text is harder than the English examples in most model cards. Many surnames are ordinary words, such as Bauer, Koch, Wolf or Klein, and a classifier has to tell "Herr Koch" from "der Koch". Addresses combine street, house number, postcode and town in one line, and dates of birth appear as "geb. 03.04.1961". Tax and social-insurance numbers, IBANs and licence plates have fixed formats, so patterns catch them before the model sees the text.

Of the models above, Fastino's GLiNER2 PII and OpenMed's multilingual fine-tune name German among their training languages, and the OpenAI base model and NVIDIA's gliner-PII do not. No card publishes a German score. A German test set of a few hundred of your own documents, labelled by hand, is therefore the first step for an EU platform, and the same applies to Polish, Czech or Hungarian text.

GPU sizing: L4, RTX PRO 4500 Server Edition or a MIG instance

Encoder PII models are small next to the cards we supply. By our users-per-card rule, 90 per cent of the driver-visible memory less 3 GiB, an L4 holds 17.2 GiB with ECC on, an RTX PRO 4500 Server Edition 25.7 GiB, as our guides to the 8× L4 server and the 8× RTX PRO 4500 Server Edition server derive.

Our estimate by the users-per-card rule. File sizes from the Hugging Face "Files" pages, read on 24 October 2026; gpt-oss-20b 13.76 GB (12.82 GiB) with 24 KiB of 16-bit cache per token, 0.1875 GiB per 8,192-token document, as in our model guides.

For gpt-oss-20b on an L4, 17.2 − 12.82 = 4.38 GiB remain for cache, which holds 23 documents of 8,192 tokens at once; on the RTX PRO 4500 Server Edition 12.88 GiB remain, 68 documents. The L4 is a passive, low-profile, single-slot card at 72 W on PCIe 4.0, without MIG, so it fits almost any server with a free slot. The RTX PRO 4500 Server Edition at 165 W splits into two 16 GB MIG halves, one for the encoder and one for embedding or a guard model, as our guide to guardrail models and GPU sizing lays out. In a larger platform, one 24 GB MIG instance of an RTX PRO 6000 Server Edition does the same job.

None of the encoder cards publishes throughput, so measure documents per second on your own text before sizing for volume. For the LLM route there are published rates. In MLPerf® Inference v6.1, closed division, data centre, entry 6.1-0050, an HPE ProLiant Compute DL380a Gen12 with eight RTX PRO 6000 Blackwell Server Edition cards produced 15,256 tokens per second on gpt-oss-120b and 50,965.3 on Llama 3.1 8B in the offline scenario, retrieved from mlcommons.org on 24 October 2026, results verified by MLCommons Association. Divided by eight, that is about 1,907 and 6,371 tokens per second per card, our division of the system result, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Take 10,000 documents a day as an example. If the LLM only lists the personal data it finds, about 300 output tokens per document, the day needs 3 million output tokens: about 16 minutes on one RTX PRO 6000 Server Edition with an 8B-class model and about 52 minutes with gpt-oss-120b, by our estimate at half the published per-card rate, counting output tokens only. If it rewrites each document of 2,000 tokens, gpt-oss-120b needs about 5.8 hours. The benchmark inputs are not your documents, and MLPerf v6.1 holds no L4 entry, so the L4 rate for an LLM redactor is a figure to measure.

Testing on a labelled sample

Choose 200 to 500 documents that represent the real input, including scans after OCR, emails with signatures and tables. Two people label every personal datum with its type; where they disagree, a third decides. Then run each candidate pipeline and count, per entity type, how many labelled items it found (recall) and how many of its spans were personal data (precision). Report the missed items in full, because one missed health detail matters more than ten over-redacted dates.

Tune thresholds against that set. NVIDIA's card evaluates gliner-PII at a threshold of 0.3 and uses 0.5 in its example, and OpenAI's Privacy Filter offers preset operating points between precision and recall. Repeat the test after every model or threshold change, and keep the labelled set out of any fine-tuning. Our engineering partner Vixen.UNO builds redaction steps into RAG ingestion and LLM gateways as part of our Private AI/ML service; tell us where the data enter your platform and which routes leave it, and we set up the test and size the hardware from its results.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the NVIDIA L4, the RTX PRO 4500 Server Edition and the RTX PRO 6000 Server Edition as GPUs for servers you already run, or in AI servers built to order, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. Operating system, drivers, CUDA and a container runtime are installed on request; the redaction pipeline, the gateway and the models are part of our Private AI/ML service.

FAQ