NVIDIA RTX PRO 6000 Blackwell Server Edition vs H100 NVL and H100 SXM

Which card does which job

The NVIDIA RTX PRO 6000 Blackwell Server Edition can replace an H100 for serving language models that fit 96 GB, particularly in FP4, and for any server that also renders, encodes video or hosts virtual workstations. The H100 stays ahead where memory bandwidth, double precision or NVLink decide: interactive serving of large dense models, FP64 simulation and training across several GPUs. In their PCIe versions both are data-centre cards for standard servers, but they were designed for different jobs, and the datasheets show it in almost every row.

Our reading of NVIDIA's H100 and RTX PRO 6000 Server Edition product pages and NVIDIA's MIG user guide (updated 13 October 2026), all read on 24 October 2026; details in the sections below.

An H100 NVL pair is NVIDIA's PCIe answer for models that need more than one card, joined by a bridge at 600 GB/s. The H100 SXM is a module that NVIDIA lists for HGX H100 systems with four or eight GPUs and for the DGX H100, at up to 700 W per GPU. The RTX PRO 6000 Server Edition is a passive, dual-slot PCIe card that NVIDIA's product brief lists without NVLink, so several cards in one server talk over PCIe 5.0.

Specifications side by side

NVIDIA H100 product page (updated 1 July 2026) and RTX PRO 6000 Blackwell Server Edition product page (updated 10 March 2026), read on 24 October 2026. The H100 page marks its Tensor figures "with sparsity"; NVIDIA prints the Server Edition's FP8 and FP4 figures without a sparsity basis. FP64 on the Server Edition is our arithmetic, 1/64 of the FP32 rate per NVIDIA's RTX Blackwell PRO architecture whitepaper. CC is CUDA compute capability.

Two rows need care. The H100's Tensor figures are with structured sparsity, so its dense FP8 rate is half: 1,671 TFLOPS on the NVL and 1,979 on the SXM. NVIDIA gives the Server Edition 2 PFLOPS in FP8 without saying whether that is sparse, so the datasheets do not settle which card computes faster in FP8. A published measurement does, below. The FP32 row runs the other way: the Blackwell card lists 120 TFLOPS against 60 and 67, which matters for rendering and for codes that stay in single precision.

Measured on the same benchmark: Kempner Institute

The Kempner Institute at Harvard ran one set of single-GPU tests on the GPUs of its cluster and published the means of three runs. The handbook states neither the H100's form factor nor the RTX PRO 6000 edition, so read the H100 row as an 80 GB Hopper card, not as a specific NVL or SXM result.

Kempner Institute Computing Handbook, "Choosing a GPU partition", last updated 15 July 2026, read on 24 October 2026. PyTorch 2.8.0, CUDA 12.8; 16384 × 16384 matrix products timed with CUDA events; 4 GiB device-to-device copy; one training step of a GPT-style model with 16 layers, hidden size 2048, sequence length 1024, batch 12, about 0.94B parameters, in BF16.

By our division of these results, the RTX PRO 6000 reached 54 per cent of the H100's BF16 rate, 52 per cent of its FP8 rate, 48 per cent of its copy bandwidth and 56 per cent of its training throughput. It was ahead of the A100 in every column. The handbook also measured 749 TFLOPS for the RTX PRO 6000 in NVFP4 through torchao, which it marks as not directly comparable with the raw matrix figures; the H100 has no FP4 column at all.

The handbook's own recommendations follow the numbers: the H100 or H200 for FP8 training at scale and for sharding one model over many GPUs, the RTX PRO 6000 as "a strong general-purpose choice" for single-GPU training, fine-tuning and inference, and for FP4 and RT Core work.

LLM inference: memory, FP4 and the decode ceiling

For serving, three things decide: whether the model fits with room for its KV cache, which number formats the card computes in, and how fast it reads its weights for each generated token. We size with our users-per-card rule: 90 per cent of the memory the driver reports, less 3 GiB per card, less the weights, as our guide to how many users one RTX PRO 6000 serves derives. For the RTX PRO 6000 the driver reports 95.6 GiB. We have no driver-reported figure for either H100, so the H100 rows start from the listed 80 and 94 GB, which makes them an upper bound.

Our estimate. Conversations of 8,192 tokens with an FP8 KV cache, 1.25 GiB each; NVIDIA's FP8 and NVFP4 checkpoints of Llama 3.3 70B (72.7 and 42.7 GB). Ceiling for one user: bandwidth ÷ bytes read per token, 70.6 GB in FP8 and 40.6 GB in NVFP4. Ceilings, not measurements.

For the H100 SXM, 0.9 × 74.5 − 3 is 64.1 GiB, less than the 67.7 GiB of FP8 weights, so by our rule the model needs two cards. The H100 NVL holds it with 8.1 GiB to spare at most, six conversations; for models up to 70B parameters NVIDIA's H100 page names an NVLink-bridged pair of NVL cards, 188 GB in total. One RTX PRO 6000 holds it in FP8 with twelve conversations, or in NVFP4 with 34, at the accuracy cost NVIDIA's model card reports (MMLU 81.1 against 83.3 for BF16, as our RTX PRO 6000 vs H200 NVL comparison shows). On Hopper, TensorRT-LLM's support matrix lists FP8 and INT4 weights through AWQ or GPTQ with 16-bit or FP8 activations, but no FP4 arithmetic.

For one user typing, the H100 generates tokens about twice as fast at the same precision, because decode reads every weight once per token and the H100 reads memory 2.1 to 2.4 times faster. NVFP4 narrows the gap: the Blackwell card reads 40.6 GB per token instead of 70.6 GB, which lifts its ceiling from about 23 to about 39 tokens per second. With many users batched together, decode becomes more compute-bound, and the Kempner matrix figures then put the H100 at about twice the FP8 rate. For a mixture-of-experts model such as gpt-oss-120b, which reads about 5 GB per token, the ceilings are about 320 tokens per second on the Server Edition, 670 on the H100 SXM and 780 on the H100 NVL, by our estimate an upper bound, since batched tokens hit different experts.

We supply the RTX PRO 6000 Server Edition and build servers with two to eight of them; send us the models, precision and peak users you serve on H100 today, and our team replies with the equivalent configuration and its figures.

Fine-tuning and training

For one GPU, the Kempner training run puts the RTX PRO 6000 at 56 per cent of the H100's throughput. A LoRA or QLoRA job that fits one card therefore takes about twice as long, with 16 GB more memory than an H100 SXM: a 70B QLoRA job of about 42.8 GB leaves about 53 GB on 96 GB, against about 37 GB on 80 GB and 51 GB on 94 GB, by our arithmetic from listed capacity. Our fine-tuning server guide sizes these jobs per card count.

Across several GPUs the interconnect takes over. The H100 SXM runs on HGX baseboards with four or eight GPUs at 900 GB/s of NVLink each, and the H100 NVL bridges two cards at 600 GB/s. Eight RTX PRO 6000 Server Edition cards exchange gradients and activations over PCIe Gen5 x16 at 128 GB/s per card, both directions counted. For data-parallel training of a model that fits one card, that costs one gradient all-reduce per step; for tensor-parallel or fully sharded training of a large model, it is the main cost. Training a model that does not fit one card belongs on NVLink hardware, which among the cards we supply means the H200 NVL in 4-way bridged domains, as our comparison of H200 NVL and RTX PRO 6000 for fine-tuning explains.

FP64, graphics and video

Double precision separates the two cards most. NVIDIA rates the H100 NVL at 30 TFLOPS of FP64 and 60 on its Tensor Cores, the SXM at 34 and 67. NVIDIA's RTX Blackwell PRO whitepaper puts FP64 at 1/64 of the FP32 rate, about 1.9 TFLOPS on the Server Edition by our arithmetic, enough for code to run correctly and too little for an FP64 solver. A team that runs CFD, molecular dynamics or VASP on H100s should look at the H200 NVL, as our FP64 GPU server guide shows.

The other way round, the H100 has no RT Cores and no video encoders. NVIDIA's Isaac Sim requirements, updated 14 October 2026, state that "GPUs without RT Cores (A100, H100) are not supported", and list the RTX PRO 6000 Blackwell in their top tier. The Server Edition has 188 RT Cores, four NVENC and four NVDEC engines and MIG profiles with graphics support, and it is the RTX PRO 6000 edition on NVIDIA's vGPU list, from release 19.0, with virtual workstation profiles. Our GPU server for robotics and Isaac Sim and vGPU profiles guide cover those uses.

MIG divides both, in different ways. The H100 SXM splits into up to seven instances of 10 GB, the NVL into seven of 12 GB, compute only. The RTX PRO 6000 splits into four of 24 GB, two of 48 GB or one of 96 GB, with +gfx variants for graphics APIs; our RTX PRO 6000 MIG profiles list them.

Power, cooling and the servers they go into

The H100 NVL is the frugal card: 350 to 400 W, configurable, dual-slot PCIe. The RTX PRO 6000 Server Edition draws up to 600 W, configurable down to 300 W by NVIDIA's product brief SP-12355-001, with a 450 W mode that server makers use to fit four cards into 2U. The H100 SXM draws up to 700 W, but only on an HGX board, which arrives as a complete eight- or four-GPU system rather than as a card for an existing server.

Board power from NVIDIA's product pages and brief; GPU power is our multiplication. Whole server for the Server Edition: our estimate with two AMD EPYC 9555 and 15 per cent for the rest, as in our guide to the eight-card Server Edition server; Dell gives about 8 kW as an indicative maximum per eight-card node, and feeds are planned on the maker's figure.

Server makers also rate the two cards differently for air. Dell's PowerEdge XE7745 technical guide (revision A09, September 2026), in its thermal table for the configuration with eight E3.S drives, lists a maximum inlet of 35 °C for the H100 NVL and 30 °C for the RTX PRO 6000 Server Edition. An eight-card Server Edition server at about 6.3 kW needs a three-phase position, as our guide to 16 A, 32 A and three-phase feeds explains.

Licences, lifecycle and what we supply

The H100 NVL includes a five-year NVIDIA AI Enterprise subscription; for the SXM, NVIDIA lists it as an add-on. The RTX PRO 6000 Server Edition is licensed separately, per GPU, where a setup needs NVIDIA AI Enterprise; MIG on bare metal needs no licence. On the hardware side, a reseller's end-of-life notice gave 1 November 2025 as the last time buy for the H100 NVL. NVIDIA's vGPU lifecycle page, updated 2 October 2026, still lists the H100 PCIe and the RTX PRO 6000 Server Edition among GPUs that have not reached the end of full support.

We do not supply the H100 in any form. Where an H100 estate needs a PCIe successor with the same Hopper software, HBM and NVLink, that is the H200 NVL, as our H200 NVL vs H100 NVL comparison sets out; for A100 servers, see replacing A100 cards.

Recommendation by use case

• Inference of models up to about 70B for many users, batch or chat: the RTX PRO 6000 Server Edition, in NVFP4 where your evaluation accepts it, as one copy per card.
• Interactive serving where per-token latency is in a service level: HBM. Among the cards we supply that is the H200 NVL, which reads memory at 4.8 TB/s.
• FP64 simulation: the H200 NVL, at 30 TFLOPS FP64; not the RTX PRO 6000.
• Training or fine-tuning across eight GPUs with tensor or full sharding: NVLink hardware; the H200 NVL in 4-way domains or a DGX B300 per project.
• Rendering, Isaac Sim, video pipelines and virtual workstations next to AI: the RTX PRO 6000 Server Edition; the H100 cannot run them.

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with two, four or eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. Describe the jobs your H100 servers run today, and we say which of them move to the RTX PRO 6000 Server Edition and which belong on an H200 NVL.

FAQ