RTX PRO Blackwell (SM120) software: vLLM, SGLang, TensorRT-LLM and NVFP4 in October 2026

Which software runs on SM120 today

The NVIDIA RTX PRO Blackwell cards, from the RTX PRO 6000 Blackwell Server Edition, Max-Q and Workstation Edition to the RTX PRO 5000, 4500, 4000 and 2000, all report compute capability 12.0, which the engines call SM120. As of 24 October 2026, vLLM 0.32.0, SGLang 0.5.22 and TensorRT-LLM run them with native FP4 and FP8 arithmetic; the gaps sit in a few attention and mixture-of-experts kernels, in FP8 block scaling under TensorRT-LLM and in multi-GPU communication without NVLink. Pin versions per model, and read the start-up log for fallback warnings before you sign off a server.

SM120 is not the same target as the B200 and B300, which are compute capability 10.0 and 10.3. A kernel written for SM100 does not run on SM120 just because both are called Blackwell: vLLM lists its trtllm-gen attention kernels for 10.x only, and release notes name SM100, SM120 and SM12x separately. DGX Spark with its GB10 chip is 12.1, so a line about SM121 or SM12x usually covers it too. Every version and date below comes from release notes and documentation read on 24 October 2026. The projects move fast: vLLM published 0.30.0, 0.31.0 and 0.32.0 within four weeks.

Versions and dates as of October 2026

vLLM release notes of v0.30.0 (22 September), v0.31.0 (5 October) and v0.32.0 (18 October 2026); SGLang release pages of v0.5.20 (18 September) and v0.5.22 (19 October); TensorRT-LLM release pages of v1.2.1 (marked Latest) and v1.4.0rc1, and its quantisation page, documentation 1.4.0rc1, last updated 19 October 2026; FlashInfer README; CUDA and driver facts as in our driver guide. All read 24 October 2026.

The CUDA version matters more than it used to. pip install vllm and the default vLLM 0.32.0 image are built on CUDA 13.0, which NVIDIA pairs with the R580 driver branch; vLLM also publishes CUDA 12.9 wheels. SGLang went further: its 0.5.20 release notes say "The CUDA 12 lane is retired", and 0.5.19 was the last release with CUDA 12 wheels and images. A host on a branch older than R580 therefore needs a driver update before it can run the default vLLM image or a current SGLang image. On Blackwell, NVIDIA supports only the open GPU kernel modules. Our guide to NVIDIA driver branches and CUDA versions sets out R580 against R595 and the compatibility rules for containers built on newer CUDA releases.

NVFP4, MXFP4 and FP8 per engine

Blackwell Tensor Cores compute FP4 natively. Of the cards we supply, the RTX PRO Blackwell cards and DGX Spark do; the H200 NVL, L40S and L4 compute FP8 but not FP4. Native support in the hardware still needs a kernel in the engine for each format and each layer type, and here the three engines differ.

vLLM release notes v0.30.0 (pull requests 55170 and the SM12x blockwise FP8 entry for DGX Spark) and v0.32.0 (46963, 58877), FP8 from compute capability 8.9 per vLLM's documentation; SGLang quantisation guide (undated page); TensorRT-LLM quantisation matrix, documentation 1.4.0rc1, 19 October 2026; LMSYS blog post on the NVFP4 KV cache in SGLang, 16 September 2026. All read 24 October 2026.

vLLM. The release notes of 0.30.0 list "W4A4 NVFP4 over weight-only kernels on SM120/121" among the new defaults, so NVFP4 checkpoints now compute in FP4 on these cards instead of unpacking to 16-bit. Release 0.32.0 adds "NVFP4 KV cache now runs on SM8x and SM12x" and "OAI Triton MXFP4 MoE on SM12x", the path for the MXFP4 experts of gpt-oss. It also changes a default that shows up in logs: "MXFP4 W4A4 checkpoints fall back to W4A16 with a warning when no W4A4 kernel exists". vLLM's documentation describes the same pattern for NVFP4 on GPUs without a native FP4 kernel, a weight-only fallback through Marlin with a warning. The checkpoint then still saves memory, but the speed is that of a 16-bit kernel.

SGLang. Its quantisation guide has a table of NVFP4 GEMM backends: auto picks flashinfer_cutedsl on SM100, Marlin on SM80 to SM90 and flashinfer_cutlass "otherwise (including SM120)", and a cuDNN backend for SM100 and SM120 needs CUDA 13 and cuDNN 9.15 or later. NVFP4 GEMM "requires FlashInfer to be installed". For block-scaled FP8, the guide lists a cutlass backend for SM120 and flashinfer_cutlass for SM100 and SM120. A feature request on SGLang's GitHub had asked for block FP8 on SM120, naming the RTX 5090 and RTX PRO 6000; the current guide lists a backend for it. The nvfp4_online method, which quantises at load time, is listed for SM100 and SM103 only.

TensorRT-LLM. NVIDIA's matrix gives sm120 its own row: NVFP4, MXFP4, FP8 per tensor and the FP8 KV cache are supported, FP8 block scaling, FP8 rowwise, the NVFP4 KV cache and the W4A8 and W4A16 AWQ and GPTQ modes are not, while the sm100/103 row has most of them. A DeepSeek-style FP8 checkpoint with 128 × 128 block scales therefore belongs in SGLang or vLLM on these cards, whose 0.30.0 notes tune a blockwise FP8 kernel on SM12x for DGX Spark, and an INT4 AWQ checkpoint does not run in TensorRT-LLM on them at all. Our engine comparison of vLLM, SGLang, TensorRT-LLM and Ollama has the TensorRT-LLM release status, and our guide to FP8, NVFP4, MXFP4 and INT4 explains the formats.

Attention kernels: what vLLM selects on 12.x

vLLM's attention backend page, dated 24 October 2026, lists compute capabilities per backend. FlashAttention 2 is listed from 8.0 and FlashAttention 3 for 9.x only. FlashAttention 4 is listed from 10.0, and the page does not say whether that includes 12.x. The native FlashInfer backend is listed for 8.x to 9.x and its trtllm-gen variant for 10.x, Triton attention for any GPU. For the MLA attention of DeepSeek-class models, FlashMLA and FlashMLA sparse are listed for 9.x to 10.x, and a separate FLASHINFER_MLA_SPARSE_SM120 backend for 12.x. The page gives default priority lists for SM 8.x to 9.x and for SM 10.x; vLLM picks "the first compatible backend" from them.

SM120 has also gained its own kernels. Release 0.30.0 added "B12X causal paged attention for SM120/SM121", an optional backend that the documentation installs with the vllm[b12x] extra and selects with --attention-backend b12x. It covers causal decoder attention in BF16 with head sizes 64 to 256, and not decode context parallelism. Release 0.31.0 added "NoPE sparse MLA on the FlashInfer SM120 backend", and 0.32.0 lists "SM120 split-K attention" and NVFP4 KV cache "with XQA decode on SM12x". For a dense model with standard attention, none of this needs a setting; for MLA models and long contexts, check in the log which backend vLLM chose.

We deliver RTX PRO 6000 Server Edition servers with the operating system, NVIDIA driver, CUDA and a container runtime installed on request; send us the model, checkpoint format and engine, and our team replies with a configuration and the driver branch it runs on.

DeepSeek-class MoE models on SM120: from workaround to release

Large mixture-of-experts models with sparse MLA attention, DeepSeek-V4-Flash among them, were where SM120 support lagged most in 2026. The record, in date order:

vLLM forum post "SM120 (RTX PRO 6000) NVFP4 MoE Performance Report" (WSL2, nightly builds, edition not stated); Helix engineering blog "Running DS4 on RTX PRO 6000" (Server Edition, driver 595.84); SGLang v0.5.20 release notes (pull request 29927, opt-in environment flags, batch 1); vLLM release notes v0.30.0 (56876) and v0.31.0 (57218). All read 24 October 2026.

In April a user reported that all 80 TMA warp-specialised grouped GEMM tactics of the native CUTLASS NVFP4 MoE kernel failed at initialisation on four RTX PRO 6000 cards, and that the best result came from the Marlin W4A16 fallback at TP=4, 50.5 tokens per second for one user on Qwen3.5-397B. The setup ran under WSL2 with P2P disabled, so read it as a sign of the state of kernels then, not as a speed figure. In August Helix ran DeepSeek-V4-Flash on eight Server Edition cards and named "the missing FlashMLA sparse-decode kernel on SM120" and "nvfp4 asserts on SM120" for the KV cache; it loaded a community-built CUDA extension for the first and switched the KV cache to fp8_ds_mla for the second, and its recipe required a driver of branch 590 or later.

SGLang 0.5.20 then moved the SM120 sparse-MLA indexer to DeepGEMM's paged-MQA kernel and enabled the DeepGEMM FP4 MoE backend, "replacing the torch fallback that was the only working path". Its notes report decode time per output token falling from 36.1 to 10.5 ms at batch 1 on four RTX PRO 6000, opt-in through environment flags. vLLM pinned a DeepGEMM fork with SM120 paged-MQA ports in 0.30.0 and bumped it with SM120 fixes in 0.31.0. For a model of this class, start from the engine's current recipe, not from a guide written in the spring.

Multi-GPU on SM120 without NVLink

No RTX PRO Blackwell card has NVLink, so tensor parallelism runs its all-reduces over PCIe. By default vLLM uses its own custom all-reduce for two PCIe-only GPUs and leaves four or eight to NCCL: its source disables the custom kernel when the GPUs share a node, the world size is above two and they are not fully connected. The release notes of 0.30.0 list "an opt-in FlashInfer PCIe IPC all-reduce for NVLink-less boxes" (pull request 53576), switched on with VLLM_ALLREDUCE_USE_FLASHINFER_PCIE_IPC=1. The pull request names the RTX PRO 6000 and SM120 as its motivation.

The backend supports tensor-parallel sizes 2, 4 and 8, BF16 and FP16 tensors and one node, and falls back with a warning when the installed FlashInfer lacks the PcieIpcAllReduceWorkspace class. vLLM 0.31.0 lists FlashInfer 0.7.0.post1 among its dependencies, and the class appears in FlashInfer's 0.7.2.post1 documentation; we found no statement of the first FlashInfer release that ships it, so check the log after switching it on. Published gains are small and limited to TP=2, as our guide to RTX PRO 6000 without NVLink shows. P2P between the cards, and the IOMMU and ACS settings that break it, are covered in our guide to RTX PRO 6000 P2P on Linux.

A stack to pin for a production server

For a server with RTX PRO 6000 Server Edition cards that runs one or a few models for months, these choices follow from the release notes above.

1. Driver. R580, the long-term support branch, or R595, the production branch. Both list every RTX PRO 6000 edition, and both are above 575.51.03, the floor NVIDIA's MIG guide sets for MIG on the RTX PRO 6000. Stay at or above the fixed versions of NVIDIA's security bulletin of 30 September 2026, 580.178.04 or 595.91.07.
2. Engine release. vLLM 0.31.0 or later on a shared network, because 0.31.0 fixed a remote code execution issue rated High, which affects servers started with --trust-remote-code (GitHub advisory GHSA-h3rc-6mm3-gc2m of 6 October 2026); 0.32.0 for the NVFP4 KV cache and the MXFP4 MoE path on SM12x. SGLang 0.5.20 or later for DeepSeek-class sparse-MLA models.
3. Checkpoint format. NVFP4 or FP8 per tensor run natively in all three engines. FP8 with block scales runs in SGLang by its guide and has an SM12x kernel in vLLM, and TensorRT-LLM lists no block scaling for sm120. INT4 AWQ or GPTQ runs through Marlin in vLLM and SGLang, not in TensorRT-LLM on these cards.
4. Start-up log. Search it for the words fallback, Marlin, W4A16 and weight-only, for the attention backend vLLM picked and, with the opt-in all-reduce, for its fallback warning.
5. Accuracy. An NVFP4 KV cache needs about 56 per cent of the memory of an FP8 cache, 4.5 against 8 bits per value by our arithmetic from the format. LMSYS published accuracy figures for SGLang on 16 September 2026, with performance runs on the RTX PRO 6000 Blackwell Server Edition: for Qwen3.8-27B, an NVFP4 cache scored 0.31 percentage points lower on GSM8K and 1.01 lower on GPQA-Diamond than an FP8 cache, and resolved 76.20 against 77.80 per cent of SWE-bench Verified. The post calls SGLang's support experimental and covers two models, so compare answers against an FP8 cache on your own evaluation set first; our page on the NVFP4 KV cache on the RTX PRO 6000 Server Edition has the figures and the flags.
6. Reproducibility. Pin the engine image by digest, with the FlashInfer and DeepGEMM versions it carries, and rerun the load test after every change.

Our engineering partner Vixen.UNO deploys open models with vLLM on these servers as part of our Private AI/ML service; tell us which models you want to run and in which format, and we check them against the release notes before you order.

What changes on the H200 NVL

The H200 NVL is compute capability 9.0, Hopper, and sits on the other side of most of these lines. vLLM lists FlashAttention 3 and FlashMLA for 9.x, and TensorRT-LLM's matrix lists FP8 block scaling and INT4 AWQ for Hopper. There are no FP4 Tensor Cores: NVFP4 and MXFP4 checkpoints load through weight-only kernels and compute in 16-bit, as our guide to FP4 models on the H200 NVL explains. For an FP8 model that depends on Hopper kernels, the H200 NVL with NVLink bridges is the conservative choice; for NVFP4 checkpoints and 96 GB per card with native FP4, the RTX PRO 6000 Server Edition, with the checks above. Our comparison of RTX PRO 6000 and H200 NVL for LLM inference weighs the two cards, and our page to buy a GPU server for LLMs lists the configurations we build.

FAQ