GLM-5.3 in NVFP4 on eight NVIDIA RTX PRO 6000 Blackwell Server Edition cards

Eight Server Edition cards with NVIDIA's NVFP4 GLM-5.3

GLM-5.3 runs on a server with eight NVIDIA RTX PRO 6000 Blackwell Server Edition cards in NVIDIA's NVFP4 version. Its 464.1 GB of weights take 432.3 GiB, which leaves 29.0 GiB per card for cache within the 664.3 GiB of our users-per-card rule. In two pipeline stages of four cards that is room for about 18 conversations of 32K tokens with a 16-bit cache, or 30 with an FP8 cache, by our estimate, close to what eight H200 NVL hold with the FP8 checkpoint.

The open item is software. NVIDIA tested the checkpoint on the B300, a data-centre Blackwell GPU with compute capability 10.x, and we found no published run on the RTX PRO 6000, which is compute capability 12.0. Our GLM hardware requirements guide sizes GLM-5.3 in FP8 on eight H200 NVL and explains why the FP8 weights do not fit eight RTX PRO 6000; this page covers the NVFP4 route on eight Server Edition cards.

What NVIDIA's GLM-5.3-NVFP4 checkpoint contains

NVIDIA created the repository nvidia/GLM-5.3-NVFP4 on 14 September 2026. Its card describes GLM-5.3 as "a Mixture-of-Experts (MoE) model for reasoning, coding, and agentic tasks" with "753B in total and 40B activated" parameters, quantised with NVIDIA Model Optimizer 0.47.0.

nvidia/GLM-5.3-NVFP4 model card, config.json and file list (sum of 50 safetensors files, 464,143,549,144 bytes), Hugging Face API (created 14 September 2026, last modified 18 September 2026), read on 24 October 2026.

The card states: "Only the weights and activations of the linear operators within transformer blocks in MoE are quantized." Its config.json uses blocks of 16 values for the NVFP4 experts in layers 3 to 77, stores the dense MLP of layers 0 to 2 in FP8 and excludes layer 78, the multi-token prediction (MTP) layer, from quantisation. The serving command uses --tp-size 8, --quantization modelopt_fp4, --context-length 400000 and --mem-fraction-static 0.90.

vLLM's GLM-5.3 recipe, updated on 15 October 2026, points to a second NVFP4 checkpoint, Inferact/GLM-5.3-NVFP4, which re-quantises only the MoE expert linears; the recipe gives "~465 GB", calls it "Blackwell-only" with the heading "(B200/B300)", and serves it with --tensor-parallel-size 8, --enable-expert-parallel and --kv-cache-dtype fp8_e4m3. Neither source names an RTX PRO card. Our list of NVFP4 models for the RTX PRO 6000 tracks such checkpoints.

Accuracy: NVFP4 against FP8 on NVIDIA's card

NVIDIA reports the NVFP4 checkpoint against the FP8 base on eight benchmarks. The largest gap is 1.13 points, on Terminal Bench 2.1, in favour of NVFP4; SciCode loses 0.62 and AA-LCR 0.12.

nvidia/GLM-5.3-NVFP4 model card, read on 24 October 2026; "Evaluation runtime: SGLang with tensor parallelism 8 across two 4-GPU nodes" on NVIDIA's test hardware.

These are NVIDIA's own measurements on its own hardware. They show that the 4-bit experts keep the model's quality on public benchmarks. Your documents, code and languages are a separate test, and a pilot on the delivered server runs it. Our guide on how to quantise an LLM to NVFP4 or FP8 explains what the formats store.

Memory and conversations on eight Server Edition cards

Our users-per-card rule takes 90 per cent of the 95.6 GiB the driver reports and subtracts 3 GiB per card: 83.0 GiB per card, 664.3 GiB for eight. The NVFP4 weights take 432.3 GiB, 54.0 GiB per card, and leave 232.1 GiB, 29.0 GiB per card. The FP8 checkpoint would need 704 GiB and does not fit.

GLM-5.3 stores one compressed latent per layer and token, 512 values plus a 64-value positional part, and a key for its sparse-attention indexer. Our GLM guide counts 97.8 KiB per token in 16-bit and 60.0 KiB in FP8 over the 78 layers: 3.06 and 1.88 GiB per conversation of 32,768 tokens. Under tensor parallelism vLLM keeps this cache in full on every card, and two pipeline stages halve it per card.

Our estimates by arithmetic, not measurements: concurrent conversations at full length, whole numbers rounded down; 29.0 GiB of cache per Server Edition card with NVFP4 weights, 35.35 GiB per H200 NVL with FP8 weights (our GLM guide); 400K is the context of NVIDIA's command.

The Server Edition figures are upper bounds. vLLM measures its own runtime memory at start-up, and our Kimi K2 guide records about 7.4 GiB per card less cache on eight H200 than our rule leaves; the token count vLLM prints at start-up on the delivered server gives the real figure. At the full 1M context, one FP8 conversation needs about 30.0 GiB per card in two stages, just more than the 29.0 GiB left. Our article on long-context LLM hardware covers the offloading options for such lengths.

vLLM and SGLang kernels for GLM-5.3 on Blackwell RTX (SM120)

Two parts of GLM-5.3 decide whether it runs on compute capability 12.0: the NVFP4 experts and the sparse latent attention. For the experts, vLLM 0.30.0 of 22 September 2026 made "W4A4 NVFP4 preferred on SM120/121", and SGLang's quantisation documentation selects its FlashInfer CUTLASS backend for NVFP4 on SM120, as our page on Blackwell RTX SM120 software support records.

The attention is the narrower part. vLLM's attention backend table, read on 24 October 2026, lists FLASHINFER_MLA_SPARSE_SM120 for 12.x with the cache types auto, fp8, fp8_e4m3 and fp8_ds_mla, and without decode context parallelism (DCP). GLM-5.3 has the 512-plus-64 latent of DeepSeek-V3.2. The smaller GLM-5.3-Flash has no positional part, and vLLM issue 53963, opened on 26 August 2026, reports for it "no SM120 sparse-MLA path for rope-free MLA" on the RTX PRO 6000. vLLM 0.31.0 of 5 October 2026 then listed "NoPE sparse MLA on the FlashInfer SM120 backend" for that model, but on 24 October the issue was still open, with user reports in October of problems with MTP drafts on that backend. Helix served the Flash model on a node with eight Server Edition cards through a pinned, experimental SGLang build after its vLLM attempt failed (blog of 27 August, updated 24 September 2026). Neither report covers the full GLM-5.3, so its sparse path on this card is unverified in public sources.

DCP is the second consequence. vLLM 0.30.0 lists "PCP+DCP on sparse-MLA models" and requires that "Attention implementations must explicitly declare DCP support", and the SM120 sparse backend does not declare it in the table. The table also lists FLASH_ATTN_MLA_SPARSE_FA4 for "Any" capability with DCP, but only with a 16-bit cache, and we found no published run of it on this card. Plan this server with pipeline stages, which halve the cache per card, and leave DCP out until a release lists it for 12.x.

We build eight-card Server Edition servers to order and check the engine path for your model before we quote; tell us the GLM variant, context and concurrent users, and our team replies with the layout and the conversations it holds.

Layouts: two pipeline stages, TP=8 or expert parallel

The Server Edition cards have no NVLink, so every split runs over PCIe Gen5. Tensor parallelism over eight cards sends two all-reduces per layer across all cards and both processor sockets, and by default vLLM disables its custom all-reduce for more than two GPUs without NVLink; release 0.30.0 (22 September 2026) added an opt-in FlashInfer PCIe IPC all-reduce for such systems. Two pipeline stages of four keep the per-layer traffic inside each half of the server, and in a two-socket server each stage can sit under one processor; only the activations cross between the stages. Our comparison of TP=2, TP=4 and replicas without NVLink covers the settings.

In vLLM this is --tensor-parallel-size 4 --pipeline-parallel-size 2. Release 0.30.0 lists "speculative decoding under pipeline parallelism", which matters because vLLM's recipe runs GLM-5.3 with MTP and five speculative tokens, and NVIDIA's checkpoint keeps the MTP layer unquantised. The recipe's NVFP4 command uses expert parallelism over eight cards, which places whole experts on cards; on a PCIe server it sends tokens between cards at every MoE layer. Measure both layouts on the delivered server with your prompt lengths; our guide to GPU server acceptance testing describes such a load test.

The same 432.3 GiB also fits two servers with four Server Edition cards each. NVIDIA's own evaluation ran tensor parallelism of eight across two 4-GPU nodes; over Ethernet we would split by pipeline stages instead, as our guide to a 600 GB model across two 4-GPU servers explains.

Eight Server Edition with NVFP4 or eight H200 NVL with FP8

Our estimates and the sources above; H200 NVL figures from our GLM guide; power by our estimate as in our eight-card power guide (eight cards at 600 W, two processors, plus 15 per cent, for either card); host memory by NVIDIA's configuration guide for NVIDIA-Certified Systems (30 September 2026).

For a company that wants GLM-5.3 on one server, the Server Edition route holds about four fifths of the conversations of the H200 NVL route by our estimate, with a format the publisher has not tested on this card. The H200 NVL route runs the original FP8 weights with a recipe that names that memory size; our guide to an eight-card H200 NVL server sizes it. By memory alone, the NVFP4 weights would also fit four H200 NVL with about 61 GiB left, but the H200 has no FP4 arithmetic and would run them through a weight-only fallback that neither publisher covers, so we do not plan with it.

If a smaller model serves your use case, GLM-5.3-Flash needs only four cards; our article on GLM-5.3-Flash on four Server Edition cards sizes it.

GLM-5.3 License and NVIDIA's terms

The GLM-5.3 License grants the right "to deal in the Software without restriction" under its conditions. Its clause 2 defines Model as a Service as "giving a third party access to language model inference or fine-tuning (e.g., via API)" with meaningful control over inputs, parameters or training data, and excludes "end-user products with model capabilities solely embedded within specific features or harnesses". If a licensee operating such a business exceeds 10 billion US dollars of revenue with its affiliates over any consecutive 12 months, it "must pass Z.AI's security review before using the Software or its derivative works for any commercial purpose". We found no territorial restriction in the text. NVIDIA's checkpoint is governed by the NVIDIA Open Model Agreement, and its card states: "The NVIDIA Open Model Agreement does not modify or supersede the GLM-5.3 License or its conditions." Whether a clause applies to your deployment is for your legal department to assess.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with eight RTX PRO 6000 Server Edition cards, or with eight H200 NVL and their NVLink bridges, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. Quantities and lead time are confirmed in writing with the quote. Our page to buy an 8× RTX PRO 6000 Server Edition server lists the variants, and large LLMs on eight Server Edition cards places GLM-5.3 beside other models. Deploying GLM-5.3 with vLLM or SGLang is part of our Private AI/ML service, with engineering by our partner Vixen.UNO.

FAQ