4× NVIDIA RTX PRO 6000 Blackwell Server Edition or 4× H200 NVL: four-card servers compared

Two four-card servers at the same power

Choose four H200 NVL when one 70B to 400B model has to serve many users interactively, when the job includes sharded fine-tuning or double-precision simulation, or when the NVIDIA AI Enterprise subscription is part of the plan. Choose four NVIDIA RTX PRO 6000 Blackwell Server Edition when models run in NVFP4, when the work is batch throughput, several models side by side or virtual workstations next to AI, and when 96 GB per card holds the largest model. Both servers draw 2.4 kW of board power.

Both are passive, dual-slot PCIe Gen5 cards for rack servers, rated up to 600 W, and both run in the same 4U chassis from Dell, HPE and Cisco. They differ in the memory system and the link between cards. The H200 NVL carries 141 GB of HBM3e at 4.8 TB/s and joins up to four cards with an NVLink bridge at 900 GB/s per GPU. The RTX PRO 6000 Server Edition carries 96 GB of GDDR7 at 1,597 GB/s and has no NVLink, so its four cards talk over PCIe. Our card-level comparison, RTX PRO 6000 vs H200 NVL for LLM inference, covers one card of each; this page compares the four-card server, which is how most buyers order them.

The decision table

Conversations for Llama 3.3 70B in FP8 at 8K with an FP8 KV cache, one copy over four cards, and memory left for Llama 4 Maverick in FP8: our estimates by the users-per-card rule below. FP64: NVIDIA's H200 page (30 TFLOPS per card) and our arithmetic for the Server Edition (1/64 of 120 TFLOPS FP32, per NVIDIA's RTX Blackwell PRO architecture whitepaper).

Memory and the link between cards

NVIDIA RTX PRO 6000 Blackwell Server Edition product page and product brief SP-12355-001_v02; NVIDIA H200 page and product brief PB-12128-001_v01; NVIDIA MIG user guide (13 October 2026); read on 24 October 2026. Link rates are NVIDIA's totals for both directions.

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card for the CUDA context and activations: 83.0 GiB per RTX PRO 6000 Server Edition and 123.4 GiB per H200 NVL. What is left after the weights holds the KV cache. The rule is derived in our guide to how many users one RTX PRO 6000 can serve.

The link decides how well a model split over all four cards runs. vLLM's source enables its custom all-reduce on more than two GPUs only when they are fully connected over NVLink; on more than two PCIe-only GPUs it logs "Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs" and uses NCCL. Four H200 NVL on one 4-way bridge meet that condition, four RTX PRO 6000 Server Edition cards do not. This is the default behaviour: vLLM 0.30.0 (release notes of 22 September 2026, #53576) added an opt-in FlashInfer all-reduce over PCIe IPC for systems without NVLink, which a four-card Server Edition server can test. The Server Edition server works best with one copy per card or two TP=2 pairs, as our article on RTX PRO 6000 without NVLink explains.

Which models fit, and for how many users

Our estimates by the users-per-card rule. Weights from the Hugging Face repositories as listed in our model guides: Llama 3.3 70B FP8 72.7 GB, gpt-oss-120b 65.3 GB, Qwen3-235B-A22B-Instruct-2507-FP8 236 GB, Llama 4 Maverick FP8 about 417 GB, DeepSeek-V3.2 about 690 GB.

The 4-way NVLink domain is what makes the 400B class possible on four PCIe cards. On the Server Edition server, one copy over all four cards has 332.2 GiB for weights and cache together, so the larger the weights, the less room is left for conversations. DeepSeek-V3.2 needs eight cards of either kind, as our guide to LLM hardware requirements by model lists.

For users, take Llama 3.3 70B in FP8, the model most companies name first, at an 8K context with an FP8 KV cache of 1.25 GiB per conversation.

Our estimate, concurrent conversations of 8,192 tokens at full length: (usable memory of the copy − 67.7 GiB) ÷ 1.25 GiB, rounded down per copy.

The gap is largest with four copies: 55.7 GiB of cache per H200 NVL against 15.3 GiB per RTX PRO 6000 Server Edition, so 44 conversations per card against 12. With an NVFP4 checkpoint of 39.8 GiB, which runs natively on Blackwell only, each Server Edition card holds 34 conversations and four copies 136. These are requests in flight at the same moment; our guide to private ChatGPT server sizing by company size converts headcount into that number.

Speed per user follows bandwidth. Reading the 70.6 GB of FP8 weights per token gives a ceiling of about 23 tokens per second on one RTX PRO 6000 Server Edition and 68 on one H200 NVL, by our arithmetic. StorageReview measured 32.89 tokens per second for one Llama 2 70B user with TP=4 on four Server Edition cards in an HPE ProLiant Compute DL380a Gen12 (review of 6 November 2025). We supply both servers; send us the model, the context length and the peak requests in flight, and our team returns both configurations with the figures per layout.

Published results: MLPerf® v6.0 per card

The cleanest comparison of the two cards is one chassis in one round: Dell's PowerEdge XE7740 with eight H200 NVL and with eight RTX PRO 6000 Server Edition in MLPerf® Inference v6.0.

MLPerf® Inference: Datacenter v6.0, closed division, available category, Llama 2 70B (llama2-70b-99), entries 6.0-0021 (Dell PowerEdge XE7740, 8 × H200 NVL) and 6.0-0022 (Dell PowerEdge XE7740, 8 × RTX PRO 6000 Server Edition), retrieved from mlcommons.org on 24 September 2026. Results verified by MLCommons Association. Per card is our division of the system result, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

By our division the H200 NVL leads by 18 per cent per card offline and 14 per cent in the server scenario. These are eight-card systems; a four-card server has one NVLink domain on the H200 side and no cross-domain traffic, so per-card rates of four-card servers may differ. MLPerf® Inference v6.1 of 16 September 2026 has no H200 NVL entry, so a like-for-like pair stays at v6.0. Our H200 NVL benchmarks and RTX PRO 6000 benchmarks list the other entries, including an interactive result for the Server Edition in another maker's server.

StorageReview's enterprise GPU roundup of 14 October 2026, with vLLM 0.27.1, tested one H200 NVL at 450 W, "the most a Dell PowerEdge R770 GPU slot delivers", against an RTX PRO 6000 Workstation Edition at 600 W, which has the same GPU and more bandwidth than the Server Edition. On Ministral 3 14B in FP8 at 32 streams, the H200 NVL reached 3,435 output tokens per second and the Workstation Edition 2,283; on an NVFP4 build of Qwen3.8-27B the order reversed, 467 against 616, because Hopper has no FP4 arithmetic.

Fine-tuning on four cards

Our H200 NVL vs RTX PRO 6000 for fine-tuning comparison counts the model states per method. QLoRA on a 70B model, 42.8 GB, fits one card of either kind, so a four-card server runs four such jobs in parallel. LoRA on a 70B model with a BF16 base, 144.4 GB, needs two cards of either kind. A full fine-tune of an 8B model with 32-bit Adam, 128 GB of model states, fits one H200 NVL and needs two RTX PRO 6000 Server Edition cards with FSDP. A full fine-tune of a 70B model, about 1,129 GB, exceeds both servers' GPU memory and needs CPU offload or eight cards and more.

Once a run is sharded, the link carries the weights. In a 70B LoRA sharded over four cards, that article counts about 212 GB received per card for each forward and backward pass. At nominal link rates that takes at least 3.3 seconds over PCIe Gen5 at 64 GB/s each way and 0.47 seconds over NVLink at 450 GB/s each way, by our arithmetic, before FSDP hides part of it behind computation. NVIDIA rates the H200 NVL at 835.5 dense BF16 TFLOPS, half its sparse figure; for the Server Edition NVIDIA publishes no dense rate, and the Workstation Edition's is 503.8. For sharded training the H200 NVL server is the stronger platform, and for one job per card the gap narrows to that tensor rate.

FP64 and simulation

NVIDIA rates the H200 NVL at 30 TFLOPS of FP64 and 60 on its Tensor Cores, 120 TFLOPS of FP64 for four cards. NVIDIA's RTX Blackwell PRO architecture whitepaper gives the GB202 chip an FP64 rate of 1/64 of FP32, which by our arithmetic is about 1.9 TFLOPS per Server Edition and 7.5 for four. Codes that compute in double precision, many CFD, chemistry and structural solvers among them, belong on the H200 NVL; single- and mixed-precision solvers run well on the RTX PRO 6000 Server Edition. Our article on GPUs for CFD simulation goes through the solvers.

Power, chassis and host

Both cards take one 16-pin connector and run up to 600 W. NVIDIA's briefs give the Server Edition a 300 W minimum and the H200 NVL a 200 W minimum. With two AMD EPYC 9355 at 280 W and 15 per cent for the rest of the server, both four-card servers come to about 3.4 kW at the wall at 600 W per card, and about 2.7 kW at 450 W, by our estimate. That is above the 2.94 kW we plan per 230 V 16 A feed at 600 W, so the rack position needs 32 A or three-phase, or the cards run capped. StorageReview found that the RTX PRO 6000 Workstation Edition at 450 W "gave up 2 to 5 percent at short context" and "4 to 9 percent at long context"; for the H200 NVL at 450 W we found no published comparison with 600 W.

HPE QuickSpecs a50004300enw and a00047453enw; Lenovo Press LP2263; Supermicro SYS-212GB-NR product page; Dell XE7745 technical guide A09 and XE7740 technical guide A04; Cisco C845A M8 data sheet and spec sheet A.11; NVIDIA-Certified Systems configuration guide (30 September 2026); read on 24 October 2026.

NVIDIA's certified-systems guide asks for at least twice the total GPU memory as system memory and six cores per GPU, so the H200 NVL server needs about 360 GB more host RAM. Our reference inference node on AI servers carries four Server Edition cards with two AMD EPYC and 768 GB of DDR5 ECC; for four H200 NVL we fit 1,152 GB, 24 modules of 48 GB. Our guide to the four-card H200 NVL server covers bridges, chassis and host for that build, and the four-card RTX PRO 6000 Server Edition server the other one.

Licences and virtualisation

NVIDIA's licensing guide, updated on 2 September 2026, states: "Each NVIDIA H200 NVL Tensor Core GPU includes a five-year NVIDIA AI Enterprise subscription." The term starts 90 days after the card ships to the server maker and is activated with the GPU serial number, so four cards bring four subscriptions. For the RTX PRO 6000 Server Edition, NVIDIA AI Enterprise is licensed separately, per GPU. Open-source engines such as vLLM run on either server without it.

For virtual machines, the H200 NVL has compute vGPU (C-series) through NVIDIA AI Enterprise only, and NVIDIA lists it in the AI Enterprise 8.2 support matrix, whose compute vGPU hypervisors include VMware vSphere 8.0 and later; it has no graphics vGPU. The Server Edition is on NVIDIA's vGPU list from release 19.0 for graphics and compute profiles, with ESXi 8.0 Update 3g or later on vSphere; graphics vGPU editions such as vWS and vPC are licensed per concurrent user, some also per GPU as a subscription. A server that hosts CAD desktops by day and inference at night is a Server Edition server.

Which four-card server for which job

• Company assistant on a 70B model for 1,000 staff and more: four H200 NVL, with 286 to 340 conversations in flight and three times the bandwidth per user.
• One 400B-class model on one server: four H200 NVL on a 4-way bridge.
• Document and batch pipelines on NVFP4 models: four RTX PRO 6000 Server Edition, which run FP4 natively; the RTX PRO 6000 Workstation Edition led the NVFP4 test in StorageReview's roundup.
• Several models of up to 70B, one per card, plus embeddings and rerankers: either; the Server Edition server needs less host RAM, the H200 NVL server leaves three to four times the cache per card.
• Virtual workstations, rendering or video next to AI: four RTX PRO 6000 Server Edition.
• LoRA and full fine-tuning across cards, FP64 solvers: four H200 NVL.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with four RTX PRO 6000 Server Edition or four H200 NVL with their NVLink bridges, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. Quantities and lead time are confirmed in writing with the quote. Describe your models, users and power feed, and we quote both servers where both fit.

FAQ