AI video dubbing with lip sync on your own GPU: open models, licences and the workstation

Dubbing a company video with lip sync, without uploading it

A company can translate and dub its own videos with lip sync on a local workstation using open models: Whisper writes down what is said, a translation model such as EuroLLM-9B-Instruct translates the script, Chatterbox Multilingual speaks the translation in the original speaker's voice, and LatentSync, MuseTalk or InfiniteTalk adjust the lips in the existing footage. All of them allow commercial use under the licences we read on 30 October 2026, LatentSync's weights with use restrictions. By our arithmetic from the published memory figures, a workstation with one 32 GB NVIDIA RTX PRO 4500 Blackwell runs the chain one step at a time, and a 72 GB RTX PRO 5000 Blackwell keeps every model loaded at once.

The typical job: a product video, a message from the managing director or a training module exists in German and is needed in English, French, Italian, Polish or Spanish, in the speaker's own voice. This guide is for marketing, training and communication teams who do that regularly and want to keep unreleased footage in the company. The hardware is in one section near the end.

Dubbing replaces the spoken track of a video with a new one in another language. Lip sync means the mouth moves in time with the new words. A model is the trained AI itself, a set of files that the software loads onto the graphics card (GPU); its licence says what a company may do with it. Voice cloning means a speech model imitates a person's voice from a short recording.

Cloud services offer this in one step. HeyGen's video translation page, read on 30 October 2026, offers to "Translate videos into 175+ languages with realistic AI voices, lip sync, and subtitles" and has an "Edit & Review" step to proofread the translated script before the video is rendered. A local workstation has fewer languages, and someone on the team runs the steps. In return the footage, the voice and the scripts stay in the company network.

The four steps from original to dubbed video

1. Transcribe. Whisper turns the original speech into text with timestamps. OpenAI's repository states that "Whisper's code and model weights are released under the MIT License" and gives about 10 GB of GPU memory for the large model and about 6 GB for turbo, an optimised version of large-v3. Our Whisper GPU guide and the comparison of faster-whisper, WhisperX and whisper.cpp cover this step in depth.
2. Translate. A language model translates the transcript sentence by sentence. EuroLLM-9B-Instruct is under Apache 2.0, has 9.154 billion parameters and, by its card, covers the official EU languages; its instruction data focuses on "general instruction-following and machine translation", and the card warns that the model has not been aligned to human preferences and can produce false statements. In 16-bit precision its weights take about 18.3 GB by our arithmetic (9.154 billion × 2 bytes). A colleague who speaks the target language reviews the script before any audio is made and shortens sentences that run longer than the scene.
3. Speak. A text-to-speech model reads the translation in the speaker's voice. Chatterbox Multilingual V3 from Resemble AI is a 0.5-billion-parameter model under the MIT License with 23 languages, among them German, English, French, Italian, Spanish, Polish and Dutch; Czech, Hungarian and Romanian are not on its list. Its card describes "one voice cloning model that works across many languages" and warns that with a reference recording in another language the output "may inherit the accent of the reference clip's language"; it suggests setting the CFG weight to 0 in that case. Every file it generates carries Resemble AI's Perth watermark. Our comparison of local alternatives to ElevenLabs covers the voice models and voice cloning in more depth.
4. Lip sync. A lip-sync model changes the mouth, or the whole performance, to match the new audio track. The three usable options follow below.

If the video has music or sound effects under the voice, the voice is separated first so the background can be laid under the new speech. Demucs, an open separation model under the MIT License, does this, but its repository was archived on 1 January 2025 and is read-only, and its author states it is no longer maintained.

Which models a company may use: licences, October 2026

GitHub repositories and Hugging Face cards of openai, utter-project, facebook, ResembleAI, coqui, bytedance, TMElyralab, MeiGen-AI and the Wav2Lip project, read 30 October 2026.

Three popular models drop out. The Wav2Lip README states that "any form of commercial use is strictly prohibited", because its models were trained on the LRS2 dataset. The XTTS-v2 card puts the model under the Coqui Public Model License; the LICENSE.txt in its repository, version 1.0.0, grants use "for any non-commercial purpose" and counts a use as non-commercial only if no direct or indirect payment arises from it. NLLB-200 is under CC-BY-NC 4.0, and its card calls it "a research model" that "is not released for production deployment". Our guide to on-premise machine translation lists further translation models with licences that do allow company use.

LatentSync needs a closer look, because its code and its weights carry different licences. ByteDance publishes the code on GitHub under Apache 2.0, and the README there says nothing about the licence of the weights or about commercial use. The LatentSync 1.6 weights on Hugging Face carry only the licence tag openrail++, which Hugging Face's licence list maps to the "Open Rail++-M License"; the weights repository contains no licence file. The best-known text of that licence, which Stability AI publishes as the CreativeML Open RAIL++-M License, grants a "perpetual, worldwide, non-exclusive, no-charge, royalty-free" licence, says the licensor "claims no rights in the Output You generate", and adds use restrictions in its Attachment A, for example no use "To generate or disseminate verifiably false information and/or content with the purpose of harming others". Whoever passes the weights on, for example preinstalled on a workstation, must include these restrictions as an enforceable provision in the agreement and tell the recipient that they apply. For that reason LatentSync is not part of the software we install; your team can install it under its licence. Your legal counsel confirms the licence for your use.

MuseTalk's model card says its code "is released under the MIT License. There is no limitation for both academic and commercial usage", and that "The trained model are available for any purpose, even commercially". It adds that other models it uses, such as Whisper, DWPose and S3FD, keep their own licences.

Mouth only or the whole performance: LatentSync, MuseTalk and InfiniteTalk compared

LatentSync from ByteDance is a latent diffusion model, the same kind of model as image generators, that redraws the mouth to match an audio track. Version 1.6 of 11 June 2025 was trained on 512 × 512 video to reduce blurriness, and ByteDance's README gives a minimum of 18 GB of GPU memory for inference (8 GB for version 1.5). Its checkpoints include Whisper tiny, which turns the audio into embeddings for the model. Two settings decide quality: 20 to 50 inference steps, where more steps give better image quality and take longer, and a guidance scale of 1.0 to 3.0, where higher values improve lip-sync accuracy but can cause distortion or jitter. The README names better results on Chinese video for version 1.5 and makes no claim about other languages. For ComfyUI there is an unofficial wrapper under Apache 2.0 that its author describes as "Optimized to run on 20GB VRAM"; its latest changelog entry is from 14 June 2025.

MuseTalk 1.5 from Tencent Music Entertainment, released on 28 March 2025, changes a face region of 256 × 256 pixels to match the audio. The repository reports "30fps+ on an NVIDIA Tesla V100", which is real time on an older data-centre card, and names Chinese, English and Japanese among the audio languages it supports. It publishes no memory figure. The smaller face region is the trade-off for its speed: on a close-up in full HD, compare its output with LatentSync's before you choose.

InfiniteTalk from MeiGen-AI works differently. Its video-to-video mode synchronises the lips, head, body and expressions with the new audio, and the repository says it "Enables unlimited length generation". It generates the frames anew at 480p or 720p and "mimics the original video's camera movement, though not identically". The SDEdit option makes the camera movement more accurate but "introduces color shift and is best suited for short clips". InfiniteTalk is built on the Wan2.1 video model with 14 billion parameters; our guide to local AI avatar videos describes it in more detail, including its photo-to-video mode. Because InfiniteTalk generates 480p or 720p, a full-HD master needs an upscaler afterwards, as in our article on AI video upscaling.

For an interview or presentation recorded in good quality, LatentSync or MuseTalk keep the original image and change only the face. For a short social clip, InfiniteTalk adapts the whole performance. A test with one minute of your own material settles it.

Languages. None of the three projects publishes a lip-sync evaluation in German, French or another European language. Dub one minute into each target language and have a native speaker review it before you plan a series.

Speed. Apart from MuseTalk's V100 figure, no publisher states how long a minute of video takes, on RTX PRO cards or elsewhere. It depends on the model, resolution, steps and clip length; have one of your usual videos timed on the delivered workstation before you promise dates to a client.

You can order the workstation alone, the setup of Whisper, a translation model, Chatterbox, MuseTalk, InfiniteTalk and ComfyUI alone, or both together: the workstation delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

Consent, GDPR and AI Act labelling

A dubbed video puts new words into a real person's mouth, in a cloned voice. The European Commission defines personal data as "any information that relates to an identified or identifiable living individual", which covers the face and voice of the speaker. Get a written agreement from each person shown for dubbing and voice cloning, name the languages and the purpose, and agree what happens to the dubbed versions if it is withdrawn. For employees, the Commission notes that consent is not freely given where there is a clear imbalance, for example between employer and employee, so your data-protection officer confirms the legal basis, and our guide on local AI and the GDPR explains what own hosting changes.

Article 50(4) of the AI Act requires deployers of an AI system that generates or manipulates "image, audio or video content constituting a deep fake" to "disclose that the content has been artificially generated or manipulated". A dubbed video in which a real person appears to speak a language they did not speak can fall under that definition. Chatterbox's watermark is an imperceptible technical mark in the audio and tells the viewer nothing; Article 50(5) asks for the information at the latest at the first exposure, so a visible note from the start of the video is one way to disclose. Article 50 applies from 2 August 2026, as our article on the EU AI Act explains; a company that sets up the chain for its own use can also count as its provider, which adds the machine-readable marking of Article 50(2). Your legal counsel confirms what each video needs.

Memory per dubbing step and the workstation tiers

Not every model publishes a memory figure, so the table combines published figures with the weight of the model files. The steps run one after another: a card needs room for the largest step, and keeping all models loaded saves loading time.

repositories and cards read 30 October 2026; InfiniteTalk chain as calculated in our guide to local AI avatar videos. Working memory for long contexts and video frames comes on top of weights.

Whisper, EuroLLM, Chatterbox and LatentSync loaded together come to about 49.5 GB, or 46 GiB, by our arithmetic (10 + 18.3 + 3.2 + 18). Our sizing rule counts 0.9 × the memory the driver reports, minus 3 GiB, as usable: about 25.7 GiB on a 32 GB card, 40.0 GiB on 48 GB, 61.5 GiB on 72 GB and 83.0 GiB on 96 GB.

our sizing from the table above; card memory, bandwidth and power from NVIDIA's datasheets.

Entry. The RTX PRO 4500 Blackwell has 32 GB and draws 200 W. Each step fits on its own within the card's 25.7 GiB, the largest being the translation model (about 18.3 GB, or 17.0 GiB, of weights) and LatentSync (at least 18 GB). A 24 GB RTX PRO 4000 holds LatentSync too, but with the ComfyUI wrapper's target of 20 GB little room remains, so we start at 32 GB.

Step up. The RTX PRO 5000 Blackwell has 48 or 72 GB at 1,344 GB/s and draws 300 W. The 72 GB card keeps the whole LatentSync chain of about 46 GiB loaded within its 61.5 GiB. Either size holds the InfiniteTalk chain in FP8, about 28.5 GB or 26.5 GiB of weights, which leaves about 13.5 GiB for 480p frames on the 48 GB card. That makes the RTX PRO 5000 our recommendation for a team that wants both kinds of lip sync, the mouth only and the whole performance.

Mid. One RTX PRO 6000 Blackwell with 96 GB holds InfiniteTalk in 16-bit for 720p next to the translation model: about 67.9 GB, or 63.2 GiB, of weights within 83.0 GiB, by our arithmetic. The Workstation Edition draws 600 W, the Max-Q 300 W with a blower cooler. Two Workstation Edition cards are the realistic maximum for that edition, at about 1.8 kW at the wall by the estimate in our article on two RTX PRO 6000 Workstation Edition. Three or four Max-Q cards belong only in a chassis built for them with strong directed airflow: Puget Systems (July 2025) found that directly adjacent Max-Q cards have little room to draw air, and Exxact (updated March 2026) reports that four adjacent cards throttled without its custom airflow.

Large. A server with four RTX PRO 6000 Server Edition cards runs one dubbing job per card. By our estimate it draws about 3.4 kW and belongs in a server room with a 32 A or three-phase feed, as our page on the four-card Server Edition server explains.

What we install, and what stays on the workstation

When we set up the software, the workstation gets Whisper, EuroLLM, Chatterbox, MuseTalk and InfiniteTalk, and the video steps run in ComfyUI, an interface under GPL-3.0 with several new releases in October 2026. ComfyUI's App Mode shows a prepared workflow as a simple form, so the person who dubs picks the files and starts the job without touching the node graph. All-in-one dubbing apps such as SoniTranslate (Apache 2.0) exist on GitHub. Its install guide targets CUDA 11.8, while RTX PRO Blackwell cards need CUDA 12.8 or later, as our guide to Blackwell software support explains, and its README warns that "the models or weights may have commercial restrictions" although the code is Apache 2.0.

The original videos, transcripts, translations, voice recordings and dubbed versions stay on the workstation. The internet is needed only to download models and updates; dubbing runs offline. If subtitles are enough for part of your material, our guide to an AI server for media covers them.

We deliver dubbing workstations from one RTX PRO 4500 to two RTX PRO 6000 Workstation Edition or Max-Q and servers for localisation teams; send us the number of videos per month, their length and target languages, and we propose the configuration and the models that fit them and your budget.

FAQ