Mistral Large 3 on 8× NVIDIA RTX PRO 6000 Blackwell Server Edition or on H200 NVL

Can Mistral Large 3 run without H200 NVL

Mistral Large 3 runs without H200 NVL for contexts below 64K tokens: eight NVIDIA RTX PRO 6000 Blackwell Server Edition cards run its 403.1 GB NVFP4 version with native FP4 arithmetic and, by our users-per-card rule, 16 concurrent conversations of 32K tokens. For longer contexts, Mistral recommends the FP8 version, and that one needs eight H200 NVL for more than a single conversation. Four H200 NVL are the smallest system for the NVFP4 version, with 13 conversations of 32K.

Our estimates: concurrent conversations of 32,768 tokens by the users-per-card rule, 16-bit KV cache held in full on every card under tensor parallelism; context guidance from Mistral's NVFP4 card and vLLM's recipe; wall power from our eight-card and four-card server pages, cards at 600 W.

Our family guide on Mistral hardware requirements sizes Large 3 on H200 NVL for 1, 20 and 100 users. This page compares the three systems a buyer chooses between for Large 3 and explains what each one costs in context length, speed and power.

Mistral Large 3 in October 2026

Mistral Large 3 is a mixture-of-experts model with a 2.5B vision encoder, published as the 2512 versions. Its Hugging Face card gives "675B total parameters" and "41B active parameters" in the header, and "673B params and 39B active" in its feature list; we use the header figures.

Mistral AI, model cards, params.json, index files and file lists of Mistral-Large-3-675B-Instruct-2512 and its NVFP4 version on Hugging Face, read on 24 October 2026. Sizes are the tensor totals in the two index files, 681,499,857,920 bytes in 272 FP8 shards and 403,125,497,052 bytes in 273 NVFP4 shards.

The NVFP4 version was quantised from the FP8 model with llm-compressor. Its card was updated on 9 September 2026: "Weights have been recalibrated and updated", which should bring long-context results close to the original. Both cards state a 256K context window and that "This model is licensed under the Apache 2.0 License", linking the Apache text; the repositories carry no LICENSE file of their own. Apache 2.0 permits commercial use and has no revenue clause.

Mistral's own hardware lines read: "FP8 on a single node of B200s or H200s" and "NVFP4 on a single node of H100s or A100s". vLLM's Mistral Large 3 recipe, updated on 15 October 2026, says the FP8 format "can be used on one 8xH200 node" and serves the NVFP4 version on four B200 with --tensor-parallel-size 4.

Memory: 375 GiB of NVFP4 against 635 GiB of FP8

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card: 83.0 GiB per RTX PRO 6000 Server Edition, 123.4 GiB per H200 NVL. Eight Server Edition cards give 664.3 GiB, four H200 NVL 493.4 GiB and eight 986.9 GiB. The NVFP4 weights are 375.4 GiB and the FP8 weights 634.7 GiB.

Large 3 uses multi-head latent attention: each layer stores one compressed vector of 512 values plus a 64-value positional part per token. Across 61 layers that is 68.6 KiB per token in 16-bit, 2.14 GiB per conversation of 32K by our estimate. vLLM keeps this latent cache in full on every card under tensor parallelism, as our Mistral guide cites from vLLM's blog, so what counts is the memory left per card.

Our estimate by the users-per-card rule: (usable memory − weights) ÷ cards, divided by the 16-bit latent cache per conversation at full length, 0.54 GiB at 8K and 2.14 GiB at 32K. Four RTX PRO 6000 Server Edition cards (332.2 GiB) do not hold the NVFP4 weights.

At 64K, the upper limit Mistral gives for NVFP4, eight Server Edition cards hold 8 conversations and four H200 NVL 6. vLLM's context-parallel documentation describes decode context parallelism (--decode-context-parallel-size) for MLA models, which shards this cache across the cards instead of copying it. With eight shards the Server Edition server would hold up to about 134 conversations of 32K by our estimate; we found no document that runs this layout for Large 3 on these cards, so test it before you size a service on it.

We build the eight-card Server Edition server and four- and eight-card H200 NVL servers for this model; send us the context lengths and peak requests you expect, and our team replies with the system that fits and its figures.

FP4 in hardware or weight-only

The RTX PRO 6000 Server Edition has Tensor Cores with FP4, compute capability 12.0. vLLM's release notes for 0.30.0 of 22 September 2026 list W4A4 NVFP4 "over weight-only kernels on SM120/121", so the NVFP4 weights and activations are computed in FP4. Mistral's card says NVFP4 gets its "full advantage" on B200 and reports "a significant speed-up and a minor regression on vision datasets" there; it reports nothing for RTX cards.

The H200 NVL computes FP8 but has no FP4 arithmetic. vLLM's recipe says it can "fall back to Marlin FP4" on older generations, and adds: "You won't notice a speed-up in comparison with FP8 quantization but still benefit a memory gain." Four H200 NVL therefore run Large 3 at about the speed the FP8 version would have, with half the cards. Our guide to FP4 models on the H200 NVL and Ada GPUs explains the weight-only path.

The FP8 version uses 128 × 128 block scales. On Hopper that is a standard format. On the Server Edition, vLLM lists an SM12x blockwise FP8 kernel and TensorRT-LLM lists no FP8 block scaling for sm120, as our guide to Blackwell RTX (SM120) software support sets out; with one conversation of room, FP8 on eight Server Edition cards is a test setup in any case.

Context length: the 64K line

Mistral's NVFP4 card states that for large contexts above 64K "we observed a subsequent drop of performance" and recommends the FP8 weights in that case. vLLM's recipe lists FP8 "up to 256k context length" and NVFP4 "for < 64k context length". By the card's update note, the September recalibration should bring long-context results close to the original, but both pages still carry the 64K guidance.

If your sessions stay below 64K, NVFP4 is the efficient format. Contract review over long files, large code repositories or agents that keep long histories can cross the line. They need FP8, and FP8 at service size means eight H200 NVL. The recipe also recommends FP8 if you plan to fine-tune, "as it can be more precise than NVFP4 in some situations". Our guide to long-context LLM hardware covers the cache side of long sessions.

Splitting the model: PCIe against NVLink

Eight Server Edition cards have no NVLink, so a TP=8 copy exchanges every layer's results over PCIe. vLLM disables its own custom all-reduce for more than two PCIe GPUs by default and uses NCCL; the opt-in FlashInfer PCIe IPC all-reduce that vLLM 0.30.0 added for systems without NVLink (release notes of 22 September 2026, #53576) covers eight cards. For a mixture-of-experts model, --enable-expert-parallel places whole experts on cards and sends tokens to them instead, a layout our article on RTX PRO 6000 without NVLink compares with tensor parallelism.

Four H200 NVL on a 4-way bridge form one NVLink domain at 900 GB/s per GPU, which suits a TP=4 layout like the one vLLM's recipe shows for four B200. Eight H200 NVL form two such domains joined over PCIe, as our guide to H200 NVL card counts for large models explains. We found no published throughput of Large 3 on any of these three systems as of 24 October 2026.

Power, rack and chassis

All three systems use cards rated at 600 W. By our estimate, with two AMD EPYC processors and 15 per cent for the rest of the server, eight cards draw about 6.3 kW at the wall and four H200 NVL about 3.4 kW. Dell's data sheet gives about 8 kW as an indicative maximum for an eight-card node, and feeds should be planned on the server maker's figure. Eight-card servers need a 400 V three-phase position; a four-card H200 NVL server fits a 230 V 32 A feed. Our page on eight RTX PRO 6000 Server Edition cards lists the chassis and the other models that fit.

For a buyer, the trade is simple to state. The four-H200 NVL server uses half the power and rack space and holds 13 conversations of 32K; the eight-card Server Edition server holds 16, computes FP4 natively and keeps 664 GiB for later models, such as larger NVFP4 checkpoints. Host memory follows NVIDIA's certified-systems guide, at least twice the GPU memory: 1,536 GB for eight Server Edition cards and 1,152 GB for four H200 NVL.

Which system for which buyer

• Contexts below 64K, Blackwell cards, room for other models: eight RTX PRO 6000 Server Edition with NVFP4. Our page to buy an 8× RTX PRO 6000 Server Edition server has the configuration.
• The smallest footprint: four H200 NVL with NVFP4, weight-only, at about 3.4 kW; see our four H200 NVL server.
• Long contexts, fine-tuning or the release precision: eight H200 NVL with FP8, as on our eight H200 NVL server page.
• Eight Server Edition cards in FP8: for a functional test only, with room for one conversation of 32K.

Mistral Large 4: what Mistral has announced

Mistral announced Mistral Large 4 on 6 October 2026 as "a 1 trillion-parameter natively multimodal model with 52 billion active parameters" and wrote: "We will release the weights by the end of the month." Its Hugging Face placeholder, Mistral-Large-4-1T-A52B, shows "Upcoming release" with a current ETA of 31 October 2026 and states 49 billion active parameters per token, 52 billion including embeddings and output layers. Neither page names a format, a file size or a licence as of 24 October 2026. Until the files appear, Large 3 is the largest Mistral model you can download; we will size Large 4 from its published files.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with eight RTX PRO 6000 Server Edition cards or with four or eight H200 NVL and their NVLink bridges, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. Setting up Large 3, retrieval and the platform around it is our Private AI/ML service.

FAQ