300B MoE models on 4× NVIDIA RTX PRO 6000 Blackwell Server Edition compared

Five models that fit four Server Edition cards

Four NVIDIA RTX PRO 6000 Blackwell Server Edition cards give 332.2 GiB for weights and cache by our users-per-card rule, and five open mixture-of-experts models of 150B to 500B parameters fit them. MiMo-V2.6-Flash holds the most conversations by our count, and GLM-5.3-Flash in NVIDIA's NVFP4 version comes next with an FP8 cache; Beam-1.0, the largest, fits only in NVFP4 and leaves the least memory.

DeepSeek-V4-Flash is the best documented on these cards, with a vLLM recipe verified on eight of them. Qwen3.8-Flash-Next activates the fewest parameters per token, under a licence with a clause for service providers.

This page compares the models for a four-card server you have already chosen or are about to order, such as the inference node on our AI servers page. Each model has one row here; the per-model pages carry the detail: Beam-1.0 on 4× RTX PRO 6000 Server Edition, GLM-5.3-Flash on 4× RTX PRO 6000 Server Edition, our DeepSeek hardware guide and our Qwen hardware guide.

Our estimates by the users-per-card rule, 83.04 GiB per card, one tensor-parallel copy over four cards, conversations of 32,768 tokens at full length with a 16-bit cache unless stated. Checkpoints: sums of the safetensors files on Hugging Face, read on 24 October 2026; Qwen3.8-Flash-Next from our Qwen guide (vLLM recipe of 13 October 2026).

The conversation counts follow each model's cache layout, which differs more than the weights do. GLM-5.3-Flash keeps a compressed latent that vLLM stores in full on every card of a tensor-parallel copy, so its count depends on the memory of one card. Beam-1.0 and MiMo-V2.6-Flash keep a normal grouped-query cache in a few full-attention layers, which splits across the cards. DeepSeek-V4-Flash counts are under review in our articles and left out here; its 44.2 GiB per card is memory only.

The models in numbers

Model cards, config.json and file lists on Hugging Face (zai-org, nvidia, deepseek-ai, Qwen, reflection, XiaomiMiMo), read on 24 October 2026; DeepSeek-V4-Flash parameters as in our DeepSeek guide (the Hugging Face metadata shows 304B).

All five offer contexts of at least 262,144 tokens; GLM-5.3-Flash, Beam-1.0 and MiMo-V2.6-Flash-RL state 1M. The checkpoint column decides whether a model fits; the active column decides how much each generated token costs. A MoE model reads its shared layers and only the experts the router picks, so Qwen3.8-Flash-Next with 6B active reads far less per token than Beam-1.0 with 23.4B. For Beam's NVFP4 checkpoint, the file sizes give about 23 GB per token by our estimate, an upper bound of about 275 tokens per second for one stream on four cards. Batched requests hit more experts per step, so such ceilings are upper bounds by our estimate. Our guide to MoE models: total against active parameters explains the arithmetic.

Each of the five models in detail

GLM-5.3-Flash. Z.ai's 320B model with 18B active, 34 linear-attention and 11 sparse-attention layers, and image and video input. NVIDIA's NVFP4 checkpoint of 204.4 GB puts 47.6 GiB on each of the four cards and leaves 35.4 GiB. Z.ai's FP8 checkpoint also fits, with only 6.6 GiB per card left. The GLM-5.3-Flash page above works through both checkpoints, the cache per card and the H200 NVL alternative.

DeepSeek-V4-Flash-0731. The official July release of DeepSeek's 284B model with 13B active, a 166.9 GB checkpoint with FP4 experts and FP8 for the rest. It is the smallest of the five and leaves 44.2 GiB per card. Our DeepSeek guide quotes vLLM's recipe of 29 September 2026: serving "was verified on 8× RTX PRO 6000 (PCIe, no NVLink)". Our page on the server with four RTX PRO 6000 Server Edition cites a published TP=4 run on four of these cards.

Qwen3.8-Flash-Next. Qwen's model with 125B parameters, 6B active, a 51B n-gram embedding table and a 4B multi-token-prediction module. Qwen's FP8 checkpoint is 185.5 GB, 172.78 GiB. Our Qwen guide reads vLLM's recipe of 13 October 2026, which moves the table to host memory by default: four cards then keep 51.7 GiB each, enough for 100 conversations of 32K by that guide's estimate. With the table on every card they keep 4.2 GiB. NVIDIA's NVFP4 version of 132.7 GB is smaller, but its card names B200 and B300 at TP=8 as test hardware.

Beam-1.0-501B-A23B. Reflection's 501.3B model with 23.4B active, the largest of the five and the only one under Apache 2.0. Only the NVFP4 checkpoint fits four cards; it leaves 14.7 GiB per card, about 97 conversations of 32K with the FP8 cache that vLLM's recipe sets. Nine of its 52 layers keep the full context, which keeps the cache small.

MiMo-V2.6-Flash-RL. Xiaomi's 309B model with 15B active, released on 22 September 2026 as reported by the trade press. Its config.json gives 48 layers, nine with global attention over 4 KV heads and 39 with a sliding window of 128 tokens. Its 172.9 GB checkpoint is FP8 in 128 × 128 blocks with an MXFP4 storage type. By our arithmetic from config.json, with values stored at their own width of 128, a 32K conversation needs about 0.73 GiB of 16-bit cache, split over four cards, which with 42.8 GiB per card gives the largest count in the table. An engine that pads the values to the key width of 192 needs about a fifth more; read the cache size vLLM reports at start-up. The model card's vLLM command uses tensor parallelism of four.

Software on the RTX PRO 6000 (SM120)

The RTX PRO 6000 has compute capability 12.0, which software calls SM120, and a recipe written for B200 (SM100) does not carry over automatically. The status for these five models differs.

vLLM release notes v0.31.0 (5 October 2026) and v0.32.0 (18 October 2026); vLLM recipes for GLM-5.3-Flash (8 October), Beam-1.0 (16 October), Qwen3.8-Flash-Next (13 October, via our Qwen guide) and DeepSeek-V4-Flash (29 September, via our DeepSeek guide); model cards on Hugging Face; read on 24 October 2026.

vLLM 0.30.0 made native W4A4 NVFP4 kernels the default on SM120, so the NVFP4 checkpoints of GLM-5.3-Flash, Qwen3.8-Flash-Next and Beam-1.0 compute in FP4 on these cards when the model's other kernels have SM120 paths. Sparse MLA, the attention of GLM-5.3-Flash and DeepSeek-V4-Flash, is where SM120 support arrived last; our SM120 software guide records the sequence from August to October 2026. MiMo's card warns that "Stable vLLM may lag" and points to a dedicated image, and Beam needs the reflection-beam image until a vLLM release includes it. For any of the five, read the attention and MoE backends vLLM chooses in the start-up log of the delivered server.

We supply the four-card server and set up the model you choose on it; send us the models on your shortlist and your context length, and our team replies with the configuration and the memory figures for each one.

Licences for company use

None of the licence texts and model cards we read on 24 October 2026 excludes a territory, so none rules out a company in the EU running the model on its own servers. They differ in conditions.

• Apache 2.0, Beam-1.0: the standard text, with licence and notices kept in redistributed copies.
• MIT, GLM-5.3-Flash and DeepSeek-V4-Flash: Z.ai's and DeepSeek's licence files are the standard MIT text, with the copyright and permission notice kept in copies.
• MIT tag, MiMo-V2.6-Flash-RL: the model card is tagged MIT and the press reports MIT, but the repository has no LICENSE file, so ask for the licence text before production use.
• Qwen Community License 1.0, Qwen3.8-Flash-Next: clause 2 requires a separate licence from Qwen for licensees that run "a Model as a Service or AI Work Assistant business", with an exception for internal use that makes neither the software, its outputs nor its capabilities available to a third party. Clause 1 asks products above 100 million monthly active users or US$20 million monthly revenue to display the model name. NVIDIA's NVFP4 version adds the NVIDIA Open Model License.

Whether a clause applies to your deployment is for your legal department to confirm. Our guide to open LLM licences for company use covers the licence types.

Which model for which job

• Most conversations on one four-card server: MiMo-V2.6-Flash-RL by memory, after a start-up test, since no vendor states SM120 support, and once Xiaomi's licence text is confirmed; GLM-5.3-Flash in NVFP4 with its documented SM120 backend and MIT licence file.
• Best documented on these cards: DeepSeek-V4-Flash-0731, with vLLM's verification on eight RTX PRO 6000 and the smallest checkpoint.
• Fewest active parameters per token: Qwen3.8-Flash-Next with 6B active, for internal use or after a licence review if you provide a service to third parties.
• The largest model: Beam-1.0 in NVFP4, with 14.7 GiB per card left and a dedicated vLLM image; for more users or FP8, eight cards, as the Beam page sizes.
• More than one of them: two of these models do not share four cards; an eight-card server or two four-card servers carry two, as our article on one 8-GPU server or two 4-GPU servers compares.

Benchmark scores are left out on purpose: the model cards report them with different settings, and the right model depends on your tasks and language. Test two candidates on your own data before you decide; our guide to choosing an open LLM for a company assistant describes such a test.

Server, host memory and power for all five models

Every row assumes the same hardware: four Server Edition cards on PCIe Gen5 without NVLink, two AMD EPYC processors and 768 GB of DDR5 ECC, twice the GPU memory as NVIDIA's certified-systems guide asks. With the cards at 600 W and two EPYC 9355 at 280 W, we estimate about 3.4 kW at the wall, or about 2.7 kW with the cards capped at 450 W, which decides between a 16 A and a 32 A or three-phase feed. A measurement on the delivered system confirms or replaces these figures. Qwen3.8-Flash-Next with its n-gram table in host memory needs at least 51 GB of system memory for the table, which 768 GB covers.

Tensor parallelism over four cards sends an all-reduce over PCIe in every layer, and expert parallelism sends tokens to the cards that hold their experts. Our article on RTX PRO 6000 without NVLink compares the layouts. If a model needs more memory per card than these leave, the comparison of 4× RTX PRO 6000 or 4× H200 NVL shows what the H200 NVL server changes.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with four or eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Choosing and setting up the model, retrieval and the serving stack is part of our Private AI/ML service; describe the tasks the model should handle, and we propose a model and a layout for your server.

FAQ