NVIDIA DGX Station GB300: which open models run in 252 GB of HBM3e, and what spills over

Which open models run on the DGX Station GB300

Open models with up to about 190 GiB of weights run on the DGX Station GB300 entirely in its 252 GB of HBM3e, with room for the KV cache. That covers gpt-oss-120b, Llama 3.3 70B in FP8, Qwen3.8-Flash-Next and GLM-5.3-Flash in NVFP4, and DeepSeek-V4-Flash-0731. Larger models such as Beam-1.0 or Kimi-K2.6 load only with part of their weights in the 496 GB of LPDDR5X on the Grace CPU, which delivers about an eighteenth of the bandwidth, so they generate several times more slowly.

NVIDIA's product page, read on 24 October 2026, lists "252 GB HBM3e ECC | 7.1 TB/s" for the GPU, "496 GB LPDDR5X ECC | 396 GB/s" for the CPU and 900 GB/s for NVLink-C2C between them, and says the system supports "models up to 1 trillion parameters". The specifications, the 784 GB figure of 2025 and the alternatives we build are in our article on DGX Station GB300 specs and alternatives. This page goes model by model.

How much of the 252 GB a model can use

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB for activations and the runtime. NVIDIA has not published what nvidia-smi reports on the Station's GPU, so we plan with 252 GB, which is 234.7 GiB, as the reported figure. For HBM that is conservative: an H200 NVL listed at 141 GB reports 140.4 GiB. The result is 0.9 × 234.7 − 3 = 208.2 GiB for weights and cache. NVIDIA's SGLang playbook for the Station starts the server with --mem-fraction-static 0.85, a similar share.

The 496 GB of LPDDR5X is not part of that figure. It holds the operating system, the engine's processes and whatever the engine places there on purpose. The GPU reaches it coherently over NVLink-C2C, but at no more than the 396 GB/s the memory itself delivers.

Models that stay in HBM3e

The table lists current open models whose weights fit the 208.2 GiB with room left for cache. Sizes are the safetensors files on Hugging Face.

Hugging Face file lists, read on 24 October 2026: sum of the safetensors files, or the repository total where the list is paged (Nemotron 3.5 Super VL, DeepSeek-V4-Flash-0731). Cache left by our users-per-card rule with 208.2 GiB usable.

For gpt-oss-120b, each conversation of 32,768 tokens takes 1.125 GiB of 16-bit cache, so 147.4 GiB holds about 131 at once. Llama 3.3 70B in FP8 holds 56 conversations of 8K with a 16-bit cache and 112 with an FP8 cache, against 12 with an FP8 cache on one RTX PRO 6000 Server Edition. The cache per conversation of the hybrid-attention models differs per model; our GLM-5.3-Flash on four RTX PRO 6000 Server Edition, Qwen3.8-Flash-Next on two GPUs and DeepSeek hardware guides work it out.

Two NVFP4 versions of Qwen3.8-Flash-Next exist. NVIDIA's repository is 132.6 GB, with 53.7 GB of it in one file that, by its name, holds the FP8 MTP and n-gram embedding weights; the one by Inferact is 182.4 GB, and it is the checkpoint NVIDIA's vLLM playbook for the Station downloads. GLM-5.3-Flash in NVFP4 fits with 17.7 GiB for cache, enough for development and a few users, not for a department.

These counts are requests in flight at the same moment, at full length. A Station with gpt-oss-120b serves a team well beyond 131 people, because few of them generate at the same second. We size Stations and GPU servers by the model and the peak load; tell us the checkpoint and the context you plan, and our team replies with the memory figures for it.

Speed while the weights stay in HBM3e

Each new token reads the active weights once, so the bandwidth of the memory that holds them sets a ceiling for one request. At 7.1 TB/s, Llama 3.3 70B in FP8, about 70.6 GB per token, is held to about 100 tokens per second, and in NVFP4, about 40.6 GB, to about 175. gpt-oss-120b reads about 5 GB per token, which puts its ceiling above 1,400. These are upper bounds by our estimate; for mixture-of-experts models a batch reads more experts than one request does.

For comparison, one RTX PRO 6000 Server Edition at 1,597 GB/s gives the same 70B model in NVFP4 a ceiling of about 39 tokens per second. NVIDIA's specification table lists 20 PFLOPS of FP4 for the Station, under a note that Tensor Core figures are with sparsity unless stated otherwise, and the Blackwell Ultra GPU runs NVFP4 and FP8 natively, so NVFP4 checkpoints are the natural choice on it.

Models that spill into LPDDR5X

A model larger than about 190 GiB still loads, but the engine has to place part of the weights in the CPU's memory. The table assumes 178 GiB of weights stay in HBM3e, leaving 30 GiB there for cache, and puts the rest in LPDDR5X.

Repository totals on Hugging Face, read on 24 October 2026; Meta's Muse Spark Open card asks for "a GPU with at least 700GB of VRAM for NVFP4". Ceiling against all weights in HBM3e is our arithmetic, assuming each token reads its weights in the same proportion from both tiers.

The arithmetic: a token that reads a share h of its weights from HBM3e and l from LPDDR5X takes h ÷ 7.1 + l ÷ 0.396 time units, against 1 ÷ 7.1 with everything in HBM3e. LPDDR5X is 17.9 times slower, so with 35 per cent spilled the token takes 0.65 + 0.35 × 17.9 = 6.9 times as long. Even 10 per cent in LPDDR5X makes it 2.7 times. For Kimi-K2.6, which has 32B active parameters per token and 1T in total, 376 GiB of the 462 GiB of LPDDR5X would hold experts, leaving about 86 GiB for the operating system and the engine.

How an engine does the split matters. vLLM documents --cpu-offload-gb, "the space in GiB to offload to CPU", with a UVA zero-copy backend, and a separate --kv-offloading-size for KV cache in CPU memory. The Kimi-K2.6 card recommends vLLM, SGLang and KTransformers; how KTransformers keeps experts in CPU memory is covered in our guide to MoE expert offload with KTransformers. NVIDIA's Station playbooks for vLLM and SGLang describe the hardware only as "Large HBM + Grace DRAM" and say nothing about offloading, and we found no published token rates for a model split across the two tiers. Test the model you plan on the engine you plan before buying for it.

Software on the GB300: SM103, Arm and NVIDIA's playbooks

NVIDIA lists "Ubuntu with NVIDIA AI Developer Tools" as the operating system, with CUDA-X libraries installed, and the Grace CPU is Arm. vLLM 0.32.0 of 18 October publishes CUDA 13.0 and 12.9 wheels for x86_64 and arm64. NVIDIA's playbook library for the Station includes "Serve LLMs with vLLM on One Device", updated on 19 October 2026, which runs Inferact/Qwen3.8-Flash-Next-NVFP4 from the image vllm/vllm-openai:qwen38-flash-next, and "Serve LLMs with SGLang", updated on 31 July 2026.

The SGLang playbook adds three Station notes: "Add --ipc host and --cap-add SYS_NICE", "--gpus all uses the GB300 when it is the only GPU" and "Use --attention-backend flashinfer (required on GB300 / SM103)". Its model list names DeepSeek-V4-Flash as a "Large local MoE for high-memory hardware platforms" and adds --quantization modelopt_fp4 for NVFP4 checkpoints. A further playbook quantises models to NVFP4 with NVIDIA Model Optimizer. For NIM, NVIDIA's product page, read on 24 October 2026, still says support is "planned in June", with production-branch support planned for October.

GB300 is SM103, compute capability 10.3, a different target from the 12.0 of the RTX PRO Blackwell cards. A kernel or a container built for one does not prove the other; our guide to Blackwell RTX software support on SM120 covers the RTX side. Running models, retrieval and the platform on top on a Station or a server is part of our Private AI/ML service; send us the models you plan and the number of users, and we reply whether a Station or a server holds them in GPU memory.

Two DGX Stations for models above 252 GB

NVIDIA's playbook "Connect Two DGX Stations for Distributed Workloads", updated on 4 September 2026, joins two systems with "two ConnectX-8 (CX8) QSFP cables" and no switch, each cable "an independent 400 Gb/s RoCEv2 path". It sets up the network and tells you to validate it with "NCCL, Ray, vLLM, or other distributed applications"; it names no model and no parallel layout.

By our rule, two Stations hold 416.5 GiB in HBM3e. Beam-1.0 in NVFP4, at 273.8 GiB, then fits entirely in GPU memory with about 143 GiB for cache. Kimi-K2.6, at 554.1 GiB, still does not. The two 400 Gb/s links carry about 100 GB/s together, so a model split across both Stations exchanges data far more slowly than inside one GPU, and pipeline parallelism, which sends only activations between the halves, is the layout to test first.

Power, circuit and MIG

NVIDIA rates the DGX Station at 1,600 W; its US page adds "20A circuit required", a figure for North American mains, while the European page gives the 1,600 W without a circuit requirement. At 230 V, 1,600 W is about 7 A by our arithmetic; the electrician who plans the office or technical room confirms the circuit under local rules, and our article on a GPU server in an office covers heat and noise. The GPU splits into up to seven MIG instances, so several small models or developers can share it with isolation, while a model that needs the full 252 GB uses the whole GPU.

When an eight-card server fits better

When the model must stay entirely in GPU memory above 208 GiB, or when many people use it at once, a rack server holds more. Eight RTX PRO 6000 Server Edition give 664.3 GiB by our rule, enough for Kimi-K2.6 with 110 GiB for cache, as our page on Kimi K2.7-Code on eight RTX PRO 6000 Server Edition works out for a model of the same size. Eight H200 NVL give 986.9 GiB in two NVLink domains, as our 8× H200 NVL server guide shows. Both draw about 6.3 kW at the wall by our estimate and need a server room.

The Station stays the better fit for one developer or a small team who want one large model on one GPU at 7.1 TB/s at a desk, and for models up to about 190 GiB that would otherwise be split over several PCIe cards. Our comparison of the DGX Station and the DGX Spark places it against the smaller GB10 system.

What we supply

The DGX Station GB300 is built by the manufacturers NVIDIA lists, and we supply their versions per project: before we can quote, the distribution asks for the company, the project and the intended use, as our guide to buying a DGX Station GB300 in the EU explains. We also build AI servers to order with RTX PRO 6000 Server Edition or H200 NVL, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. You can buy either from us: our team sizes the configuration to your workload and budget and replies within one business day. Describe the models and the team that will use them, and we propose the Station or the server that fits.

FAQ