Live captions on-premise: real-time subtitles for meetings and events with local AI

Live captions without a cloud captioning service

Live captions can run entirely on your own hardware. The audio from the microphones goes from the mixing desk to a small computer, a streaming speech model on a GPU in your building turns it into text in steps of a fraction of a second, and a browser page shows the text on a screen or projector. Open tools for this exist in October 2026, WhisperLiveKit and Collabora's WhisperLive, and a workstation with one NVIDIA RTX PRO 4000 Blackwell is enough to try them in one room. With the audio taken from the room, neither the sound nor the text leaves the building.

This guide is for event organisers, universities with lecture halls, HR and inclusion teams that want captions for colleagues with hearing loss, and public bodies with council meetings. A few terms first. A streaming model is a speech recognition model that works on the audio while it arrives, instead of waiting for the end of a recording. The delay, or latency, is the time between a word being spoken and its appearing on screen. The word error rate (WER) counts the wrong, missing and added words per 100 words of a correct transcript. Recordings after the event are a different task, covered in our guide to meeting minutes with a local AI; live subtitles for broadcast channels are in our guide to the AI server for media.

How live captioning works in a room

Our summary; client and server roles as described in the WhisperLiveKit and WhisperLive documentation, read on 30 October 2026.

The quality of the captions starts with the audio. NVIDIA's model card for Parakeet TDT 0.6B v3 shows how much: on the AMI corpus of recorded meetings its WER is 11.31 per cent, against 1.93 per cent on LibriSpeech test-clean, which is read speech recorded close to the microphone. The card's noise test points the same way, with the average error rate across its test sets rising from 6.34 per cent on clean audio to 11.66 per cent when noise is as loud as the speech. In a hall, take the signal from the desk, where the speakers' headsets and lectern microphones are already mixed, and keep audience microphones and music off that output. For a meeting room without a desk, a conference microphone in the middle of the table is the minimum.

WhisperLiveKit's bundled page records the microphone input of the computer on which it is opened, through the browser, and shows the captions on the same page. A laptop next to the mixing desk, with the desk's output on its audio input, can therefore send the audio and drive the projector at once. The recognition itself runs on the caption server, which sits in the server room or in a side room.

Delay against accuracy in German

A streaming model sees less context than a model that reads a finished recording, because it must commit to words before the sentence ends. The longer it may wait, the fewer mistakes it makes. Two current models publish this trade-off for German.

Hugging Face model cards of Voxtral Mini 4B Realtime 2602 and Nemotron 3.5 ASR Streaming 0.6B (automatic language detection), read on 30 October 2026; FLEURS consists of read sentences, so room audio gives higher figures.

Mistral calls 480 ms "the sweet spot of performance and low latency" for its Voxtral model, and the same card gives 3.54 per cent for its offline model Voxtral Mini Transcribe 2.0, so a caption system set to about half a second gives up roughly 2.6 points against a transcript made after the event. Our arithmetic from these figures: at 6.19 per cent, about one word in 16 is wrong on clean read speech. For a lecture where viewers read along over a longer time, one to two seconds of delay buys accuracy; for a discussion where people react to what was said, half a second keeps the text close to the speaker.

Whisper is no streaming model. It works on windows of 30 seconds, and its card states that "a chunk length of 30-seconds is optimal" for large-v3. Live tools wrap it in a streaming policy that re-reads the growing audio and decides which words are stable: WhisperLiveKit uses SimulStreaming by default, and WhisperLive calls itself "a nearly-live implementation". WhisperLiveKit's README adds that Whisper's language detection "tends to bias towards English", so set German explicitly for German events. The published German accuracy of the offline models, and what dialect does to it, are in our comparison of German speech recognition models.

Open tools for live captions as of October 2026

PyPI pages of WhisperLiveKit and WhisperLive, GitHub page of WhisperLiveKit, Hugging Face model cards; read on 30 October 2026.

WhisperLiveKit describes itself as an "ultra-low-latency, self-hosted speech-to-text pipeline". It runs a choice of recognition back ends, from faster-whisper to Voxtral, Qwen3-ASR and NVIDIA's Canary, serves its own web page on port 8000, offers a Chrome extension that captures the audio of a web page, and provides an OpenAI-compatible REST endpoint and WebSocket interfaces for other programs. Its README states that "the backend supports multiple concurrent users". Access can be protected with an API token. The maintainers ran their benchmarks on an NVIDIA H100 and publish them as charts only, without numbers in the text.

WhisperLive, from Collabora, runs Whisper through faster-whisper, TensorRT or OpenVINO and has Chrome and Firefox extensions that transcribe audio straight from the browser. Two defaults matter for events: --max_clients is 4 and --max_connection_time is 600 seconds, so a session ends after ten minutes unless the setting is raised.

Voxtral Mini 4B Realtime 2602 is Mistral's streaming model under Apache 2.0, with about 3.4 billion parameters in its language model and 970 million in its audio encoder. It covers Arabic, German, English, Spanish, French, Hindi, Italian, Dutch, Portuguese, Chinese, Japanese, Korean and Russian, its delay can be set in steps of 80 ms from 80 to 1,200 ms or to 2,400 ms, and vLLM serves it through a /v1/realtime endpoint over WebSockets. WhisperLiveKit lists a Voxtral back end. Nemotron 3.5 ASR Streaming 0.6B, NVIDIA's multilingual streaming model, lists German among its 19 locales for "highest-accuracy ASR, ready out of the box", with punctuation and capitalisation built in and chunk sizes from 80 ms to 1.12 s. It runs in NeMo or in Transformers 5.13.0 or later; neither caption tool names it as a back end, so connecting it to a caption page is integration work.

You can order the caption server or workstation alone, the setup of WhisperLiveKit or WhisperLive with a German streaming model alone, or both together: the system delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

Speaker labels and translated captions

Who is speaking. For a panel or a council meeting, captions are easier to follow with speaker labels. WhisperLiveKit labels speakers live with NVIDIA's Streaming Sortformer for one to four speakers; its README notes that it assigns each moment to one speaker, so overlapping voices are not shown, and that extra speakers may get an existing label. NVIDIA's Nemotron 3 Diarization, published on 23 September 2026 under the OpenMDW License 1.1, handles up to eight speakers and has an ultra-low-latency profile of 0.32 seconds; on AMI meetings recorded with one distant microphone its card gives a diarisation error rate of 12.95 per cent for that profile, against 11.14 per cent offline. Where each panellist has a microphone of their own, sending each microphone as a separate stream tells the system who speaks without any model. Our guide to speaker diarisation on-premise covers the models in depth.

Captions in a second language. WhisperLive can translate any language into English, and a server-side translation thread can target another language. WhisperLiveKit's --target-language uses NLLW, which its README describes as based on distilled NLLB and covering 200 languages. Check the licence before a company event: the model card of NLLB-200 distilled 600M carries the CC BY-NC 4.0 licence tag and states that "NLLB-200 is a research model and is not released for production deployment". For translated captions in a business setting, put a translation model whose licence allows commercial use behind the recogniser, for instance through WhisperLiveKit's alignatt back end, which translates with a language model instead of NLLB; our guide to on-premise machine translation compares them. Two caption languages mean two caption pages, one per screen.

Captions for hard-of-hearing colleagues, in the room and in hybrid meetings

People who rely on captions are the best judges of them. Agree with deaf and hard-of-hearing colleagues before the event which delay and screen position work for them, and whether automatic captions are enough for the occasion. Automatic captions contain errors; where accessibility is required by law or by your own policy, your accessibility officer or legal counsel confirms whether they meet it. Live captioning processes what identifiable people say even when nothing is stored, so inform speakers and participants beforehand, as our meeting-minutes guide describes. If the transcript is kept afterwards, it is a record of what they said, and the retention rules from that guide apply as well; your data-protection officer confirms the details.

On a projector or a second screen, the caption page runs in full-screen mode with large type. Viewers who should follow on their own phones need a page that relays one room's captions to many devices; neither project's README describes such a viewer page, so it is part of the set-up work.

For hybrid meetings held in a browser, both projects offer browser extensions that capture the audio playing in the browser, so the captions can come from the meeting itself. What stays local is then precisely the caption processing: the meeting audio still travels through the conferencing provider's service, and the captions are made on your server.

Hardware from one meeting room to a conference centre

A live caption stream only has to keep pace with speech, and the number of viewers adds no load on the GPU. What decides the hardware is the number of rooms captioned at the same time, the model and its delay setting, and whether speaker labels and translation run as well. We found no published stream counts for German or for the RTX PRO cards. The nearest published figure is English: NVIDIA's ASR NIM performance page, updated on 7 October 2026, gives one L40S a maximum of 190 live streams with Parakeet 0.6B CTC at 160 ms chunks, and 64 streams at 33.9 ms average latency, measured on a LibriSpeech file. That model has 0.6 billion parameters; a 4-billion-parameter model like Voxtral needs far more computing per stream.

Card memory and power from NVIDIA as listed on our GPU page; usable 18.5 GiB on 24 GB and 40.0 GiB on 48 GB by our users-per-card rule; model memory from the cards cited above; room counts are our reading, to be confirmed in a test.

Entry. The RTX PRO 4000 Blackwell leaves 18.5 GiB by our rule, enough for Voxtral Mini 4B Realtime, which by its card "can run on a single GPU with >= 16GB memory", or for Whisper large-v3 at about 10 GB. How many parallel rooms one card carries depends on the model and the delay; for one of its streaming back ends, WhisperLiveKit advises planning one realtime session per GPU. A 0.6B model such as Nemotron 3.5 ASR Streaming leaves more room: NVIDIA's speech NIM support matrix 26.05.0 lists 6 GB of GPU memory for the Nemotron ASR Streaming profile at batch 32.

Mid. The RTX PRO 5000 Blackwell with 48 GB leaves 40.0 GiB by our rule, which by our reading holds a streaming model next to live speaker labels and a translation model; the diarisation card states no memory figure, so the split is confirmed in a test. Nemotron 3 Diarization's card publishes its speed on this GPU: at the 0.32-second profile, 12.5 times real time at batch 1 without compilation and 292 times at batch 32 with a compiled model, so by our reading speaker labels add little load for a few rooms.

Server. For a campus or a conference centre, a rack server with several L4 cards, each 24 GB at 72 W in a low-profile slot, or L40S cards with 48 GB, gives every hall its own card. When captions are one service among several, the entry server on our AI servers page, two RTX PRO 6000 Server Edition with 96 GB each and about 1.8 kW at the wall by our estimate, runs them in a MIG instance next to the company chat assistant. The streams-per-GPU arithmetic for many parallel calls is in our guide to Whisper GPU requirements, and a server that also answers in speech is in our guide to the voice agent server on-premise.

What IT sets up on the caption server

• Network. The audio and the captions travel over WebSockets. WhisperLiveKit's deployment notes suggest Gunicorn with Uvicorn workers behind Nginx with WebSocket upgrade headers, wss:// with --ssl-certfile and --ssl-keyfile, plus --api-token, --cors-origins and --forwarded-allow-ips.
• Session limits. In WhisperLive, raise --max_clients and --max_connection_time for a full day of sessions.
• Installation. WhisperLiveKit ships a cu129 extra for CUDA 12.9 and a GPU Dockerfile run with docker run --gpus all -p 8000:8000; its Qwen3 vLLM back end needs an environment of its own, and some extras conflict with one another.
• Voxtral Realtime needs a current vLLM and mistral_common 1.9.0 or later; Mistral strongly recommends WebSockets for audio sessions. If speaker labels or translation share the card, lower vLLM's --gpu-memory-utilization from its default of 0.92, because vLLM reserves that share of the memory when it starts.
• Licences of add-ons. The AmberNet language-identification model comes from NVIDIA NGC under its own terms, SenseVoiceSmall under its own model licence, and the default Sortformer speaker model under CC BY 4.0, which asks for attribution.
• Rehearsal. Test with the real microphones and the real room: measure the delay on screen, read a list of speaker and product names aloud, and decide the font size from the back row.

Send us the number of rooms, the languages and how your audio is mixed today, and we suggest the best solution for your tasks and budget.

FAQ