On-premise AI scribe for medical practice groups: notes and letters on your own server

What an on-premise AI scribe does

An on-premise AI scribe records the conversation between doctor and patient, with the patient's agreement, and turns it into a draft of the consultation note and, where needed, a referral or report letter. All of it runs on a server that the practices own, in their own server room or a data centre they choose, so the audio, the transcript and the drafts never reach an outside AI service. The doctor reads the draft, corrects it and signs it into the practice software; nothing enters the record without that step.

This guide is for the managing director, the practice manager or the medical director of a group of practices or an outpatient centre with roughly 20 to 100 doctors. It explains what the scribe does, what the GDPR and the medical device rules mean for it, and how large the server must be, in one section with a table. The notes at the end are for the IT team or the IT service provider. Care homes, where carers dictate care notes rather than consultations, are covered in our guide to voice documentation in care homes. The legal parts are general information on EU law as of October 2026; your data-protection officer and legal counsel confirm what applies to your practices.

Ambient documentation: from the recorded consultation to the signed note

The scribe is a chain of three models, each small by today's standards. Each step leaves a result that a person can check.

Steps and models as described in this article; models from their Hugging Face cards, read 25 October 2026.

Speech recognition. NVIDIA's Parakeet TDT 0.6B v3 has 600 million parameters, transcribes 25 European languages with punctuation and word timestamps, detects the language by itself and is published under CC BY 4.0, which allows commercial use. Its card states up to 24 minutes of audio in one pass with full attention. For languages outside its list, Whisper large-v3 covers 99, and its card warns that "the predictions may include texts that are not actually spoken in the audio input (i.e. hallucination)". The doctor therefore reads the draft against their own memory of the visit.

Speaker labels. Diarisation, the step that marks which voice spoke when, lets the note separate what the patient reported from what the doctor found. The pyannote community-1 pipeline is published under CC BY 4.0; its files download only after you accept its conditions on Hugging Face, and the card says it runs on the CPU unless moved to the GPU. The same card also offers a hosted model on the provider's own servers, which an on-premise scribe does not use. Our guide to speaker diarisation on-premise compares the options.

Note and letter drafts. A language model receives the labelled transcript and the practice's note template and writes the draft. Qwen3.8-27B is an open model under Apache 2.0, and NVIDIA's NVFP4 version of it, 21.9 GB, fits one RTX PRO 4500 Server Edition. The model drafts only what was said in the consultation; it leaves fields empty that the conversation did not cover, and it does not add a diagnosis that nobody mentioned.

We supply the RTX PRO 4500 Server Edition and build AI servers to order for this work; send us the number of doctors, the languages spoken in your practices and your practice software, and our team replies with a configuration.

AI medical dictation, health data and the GDPR

A consultation recording is data concerning health, which the GDPR defines in Article 4(15) as personal data "related to the physical or mental health of a natural person". Article 9(1) prohibits processing special categories of data unless an exception in Article 9(2) applies. Point (h) covers processing "necessary for the purposes of preventive or occupational medicine", medical diagnosis and the provision of care, under the conditions of Article 9(3), which ties it to data handled by or under the responsibility of a professional subject to professional secrecy. Point (a) is explicit consent "for one or more specified purposes". Which basis covers the recording, and in what form patients agree to it, is for your data-protection officer to decide. National rules on professional secrecy and on recording conversations can add requirements, which your legal counsel confirms.

Article 35(1) requires a data protection impact assessment (DPIA) before processing that "is likely to result in a high risk to the rights and freedoms of natural persons", and Article 35(3)(b) names processing "on a large scale" of special categories among the cases. An organisation that records thousands of consultations a month should plan for one. Our guide to a DPIA for an internal LLM lists what IT contributes: the data flow, where each model runs, how long audio and transcripts are kept and who can see them.

Running the scribe on your own server changes one part of that assessment. With a cloud scribe, the provider processes the recordings on the practices' behalf, so the contract, the subprocessors and the location of processing come into the DPIA, as our guide to GDPR, cloud hosting and subprocessors explains. On your own server those questions fall away for the AI step, while Article 32 still requires measures appropriate to the risk, such as "the pseudonymisation and encryption of personal data" and the ability to restore access after an incident.

When the scribe would become a medical device

The Medical Device Regulation, Regulation (EU) 2017/745, turns software into a medical device through its intended purpose. The guidance MDCG 2019-11 rev.1 of the Medical Device Coordination Group, published by the Commission in June 2025, gives the test: software that performs an action beyond "storage, archival, communication, simple search, lossless compression" may be medical device software, and the guidance says that hospital information systems "are not in themselves qualified as medical devices". It adds that "modules integrated into or operating alongside EHR systems may qualify as MDSW if they fulfil a medical purpose". Under Rule 11, software that provides "information which is used to take decisions with diagnosis or therapeutic purposes" is class IIa or higher.

For a practice group this draws a clear line. A scribe that writes down what was said and formats it in the practice's template is a documentation tool. A feature that suggests a diagnosis, flags a treatment or rates a risk moves towards a medical purpose, and with it towards the MDR and, where a notified body is involved, the high-risk rules of the AI Act, as our page on a private LLM for hospitals explains. Keep such features out of the scribe you build, or use a certified product for them. Whether your configuration stays on the documentation side is a question for your regulatory adviser.

How large the server must be

Speech recognition is the lightest part. Our guide to speech-to-text servers for Whisper and Parakeet reports a published test in which Parakeet TDT 0.6B v3 processed audio 238.9 times faster than its duration on one NVIDIA L4. By our arithmetic, 400 hours of consultations a day would take that card about 1.7 hours. We found no such figure for the RTX PRO 4500 Server Edition, and diarisation adds time for which pyannote publishes no speed, so a test with your recordings confirms the speech part.

The drafting model sets the size. StorageReview measured NVIDIA's NVFP4 Qwen3.8-27B on one RTX PRO 4500 Server Edition at 153 output tokens per second with 32 parallel requests of 8,192 tokens in and 1,024 out, on vLLM 0.27.1 (review of 14 October 2026). By our arithmetic that is about 540 requests of that size per hour per card. A consultation note is smaller than that request, so the figure is a cautious planning value.

Example values to replace by your own: 30 consultations per doctor per day, 8 minutes of recorded speech each, notes spread over 8 hours with a peak factor of 2. Drafting capacity about 540 requests per hour per RTX PRO 4500 Server Edition, our arithmetic from StorageReview's measurement (14 October 2026); notes and letters of the same consultation counted as one request.

In the four-card server, card 1 carries speech recognition and diarisation, cards 2 and 3 each hold a copy of the drafting model, and card 4 drafts letters and runs a search over the practice's templates and guidelines. With one card drafting, 50 doctors use about 70 per cent of it at the example peak; at 100 doctors, two cards share the load. By our users-per-card rule (90 per cent of the memory the driver reports, less 3 GiB, less the weights), one card with the NVFP4 build keeps 5.28 GiB for conversations, room for 13 drafts of 8,192 tokens at the same time.

A larger model, such as gpt-oss-120b at about 65 GB, fits our entry server with two RTX PRO 6000 Server Edition cards, two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives, at about 1.8 kW by our estimate.

What the server needs in the building and the network

One central server serves all practices. They send the audio after each consultation over their shared network, and the drafts come back into the practice software. By our arithmetic, speech recorded as 16-bit mono audio at 16 kHz is 256 kbit/s, so an eight-minute consultation is about 15 MB before compression, little load for a practice connection.

The NVIDIA RTX PRO 4500 Blackwell Server Edition is a passive, single-slot data-centre card with 32 GB of GDDR7 at 800 GB/s and 165 W. Four of them in a 2U server draw about 1.4 kW by our estimate, within one 230 V 16 A feed, which carries 2.94 kW at 80 per cent; a measurement on the delivered system confirms the figure. The server is loud and gives off its power as heat, so it belongs in a server room or a cooled rack, not in a consulting room or the reception; our server-room checklist covers the preparation. Practices without a server room can have our Private AI/ML service run the platform on dedicated hardware in a Tier-3 data centre in Lithuania, inside the EU; whether that suits health data is part of the DPIA.

Plan what happens when the server is down. Doctors then write their notes by hand as before; patient care does not depend on the scribe. An organisation that comes to depend on it can add a second server at another site; our guide to an AI server without an IT department explains who looks after updates and faults when there are no IT staff in-house.

Notes for the IT team or service provider

1. Integration. The practice software stays the system of record. The scribe writes drafts into it as unsigned documents through its import interface, and only the doctor's signature turns a draft into a record. vLLM's structured outputs return the note in the fields of the template.
2. Retention. Delete the audio once the note is signed, unless your DPIA sets another period, and keep transcripts only as long as the DPIA allows. The gateway log of prompts and answers holds health data; restrict access to it and set its retention.
3. Thinking mode. Qwen3.8-27B reasons before it answers by default. Switch thinking off for note drafts, which keeps answers short and the throughput close to the measured figure.
4. Versions and security. Load models from a pinned revision in safetensors format, run a current vLLM release (0.32.0 of 18 October 2026 at the time of writing) on an internal network behind a gateway with single sign-on and multi-factor authentication, and keep the NVIDIA driver at a release that NVIDIA's bulletin of 30 September 2026 lists as fixed, as our guide to securing a GPU server explains. The model servers need no outbound internet access.
5. Evaluation. Before the first live consultation, compare drafts with notes that doctors wrote themselves on a set of recordings made with consent, per language and per specialty, and record omissions and additions separately.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the RTX PRO 4500 Server Edition and RTX PRO 6000 Server Edition and build AI servers to order with the number of cards your practices need, assembled and tested before delivery, delivered anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service, delivered with our engineering partner Vixen.UNO, sets up speech recognition, diarisation, the drafting model and the connection to your practice software, under an NDA and with a data processing agreement on request; describe your practices and your documentation today, and we propose a pilot in one practice.

FAQ