H200 NVL vs H200 SXM in HGX H200: NVLink reach, power, TP=8 and which one to choose

The same Hopper GPU in two form factors

The H200 NVL and the H200 SXM carry the same GPU with 141 GB of HBM3e at 4.8 TB/s and differ mainly in how they are built into a server. For a model that fits within four cards the NVL gives the same capacity in a standard server, while one NVLink domain of eight needs the SXM platform.

The H200 NVL is a dual-slot, air-cooled PCIe card rated up to 600 W that NVLink bridges join in sets of two or four. The H200 SXM is a module rated up to 700 W on an HGX H200 baseboard, where NVSwitch joins all eight GPUs at 900 GB/s each.

NVIDIA H200 page (specifications marked preliminary, tensor figures with sparsity); NVIDIA HGX AI Factory reference architecture, components (updated 31 August 2026); NVIDIA AI Enterprise licensing guide (updated 2 September 2026); read on 24 October 2026.

The decision usually turns on whether a model, or a training job, needs tensor parallelism over more than four GPUs at NVLink speed. If it does not, the NVL fits a standard air-cooled rack server with two, four or eight cards. If it does, the eight-GPU NVLink domain of an SXM system is the platform, and for that case we supply the DGX B300 per project, since we do not supply HGX H200.

Compute and power: 600 W against 700 W

NVIDIA's H200 page gives the SXM 3,958 TFLOPS of FP8 and the NVL 3,341, both with sparsity; dense figures are half. By our arithmetic the SXM is about 18 per cent faster on paper and draws up to about 17 per cent more power, so per rated watt the two are within 2 per cent of each other: 5.57 sparse FP8 TFLOPS per watt for the NVL and 5.65 for the SXM. BF16 shows the same ratio, 1,671 against 1,979 TFLOPS; FP64 is 30 against 34 TFLOPS. Lenovo's product guide LP1944 lists the same split, 600 W for the H200 NVL PCIe card and 700 W for the H200 SXM.

Memory and bandwidth are identical, and token generation for a moderate number of users waits mainly on memory bandwidth. By our estimate a single user of Llama 3.3 70B in FP8 sees the same ceiling of about 68 tokens per second on either GPU, since both read 70.6 GB of weights per token at 4.8 TB/s. The SXM's higher rating shows in compute-bound work: prefill of long prompts, large batches and training.

Both are configurable. NVIDIA's product brief for the NVL, PB-12128-001, lists 600 W as the default maximum and 200 W as the minimum; servers such as Dell's PowerEdge R770 cap it at 450 W, which our article on the H200 NVL at 450 W covers.

NVLink reach: bridges of up to four, NVSwitch for eight

The main difference is the reach of NVLink. NVIDIA's product brief states that "the NVIDIA H200 NVL uses a single wide NVLink bridge per card", either a 2-slot bridge for two cards or a 4-slot bridge for four. Eight cards therefore form two domains. Dell's technical guide for the PowerEdge XE7740 (Rev. A04, April 2026) requires "two bridges" for eight H200 NVL, Cisco's UCS C845A M8 spec sheet allows two 4-way or four 2-way bridges, and OVHcloud's guide for its eight-card H200 NVL instances describes NVLink only inside each set of four and PCIe between them.

On the SXM side, NVIDIA's HGX AI Factory reference architecture, updated 31 August 2026, states that "HGX H100 and H200 8-GPU Baseboards use a combination of third generation NVSwitch and fourth generation NVLink", with 900 GB/s of GPU-to-GPU bandwidth and 7.2 TB/s in total, and that "H200 variant has up to 1128 GB of GPU memory". Every GPU on the baseboard reaches every other at the full NVLink rate.

NVIDIA H200 page; NVIDIA H200 NVL product brief PB-12128-001_v01; NVIDIA HGX AI Factory reference architecture, components (31 August 2026); NVIDIA AI Enterprise reference architecture, compute node; Dell XE7740 technical guide A04; Cisco C845A M8 spec sheet A.11; OVHcloud guide of 1 October 2026; read on 24 October 2026.

Both systems hold the same 1,128 GB, so every model that fits one fits the other. The topology changes how fast a model spread over all eight GPUs runs, and how the cluster network is sized: NVIDIA's reference designs put one 400G SuperNIC on each pair of H200 NVL cards and one SuperNIC on each SXM GPU. Our article on the H200 NVL NVLink bridge, 2-way or 4-way covers the bridge mechanics.

What the topology means for TP=8

Tensor parallelism splits every layer across GPUs and joins the partial results with an all-reduce after each one. Within one NVLink domain that exchange runs at 900 GB/s per GPU; in an eight-card NVL server with TP=8, part of every exchange crosses the PCIe link between the two domains at 128 GB/s. OVHcloud's guide for eight H200 NVL says to "avoid using --tensor-parallel-size 8" and runs a large model as two pipeline stages instead, --pipeline-parallel-size 2 with --tensor-parallel-size 4, so that each stage stays inside one NVLink domain and only the activations between the stages cross PCIe, once per step. By default, vLLM's own fast all-reduce path is enabled on more than two GPUs only when they are fully connected over NVLink, which eight NVL cards are not; vLLM 0.30.0 (release notes of 22 September 2026, #53576) added an opt-in FlashInfer all-reduce over PCIe IPC for systems without NVLink.

The practical consequences by workload:

• A 70B model for many users runs best as two TP=4 copies, one per domain, on eight NVL cards: 680 concurrent conversations of 8K with an FP8 cache by our users-per-card rule, as our 8× H200 NVL server shows. The HGX's NVSwitch adds little here, since no copy leaves its domain.
• A dense model of 500 to 900 GB has to span both domains. On the NVL it runs as TP=4 × PP=2. On an HGX H200 it can run as TP=8, which uses all eight GPUs on every token instead of one stage at a time.
• Large MoE models with expert parallelism send tokens to experts on other GPUs in every layer. On the NVL server part of that all-to-all traffic crosses PCIe; on the HGX it stays on NVSwitch. vLLM's recipe for DeepSeek-V3.2, as our DeepSeek hardware guide reports, recommends data parallelism with expert parallelism on eight GPUs, and the model fits eight NVL cards with about 88 conversations of 32K by our rule.
• Training across eight GPUs, such as a full fine-tune with FSDP, exchanges gradients and shards between all GPUs every step. On two NVL domains that traffic crosses PCIe, which is the case where an eight-GPU NVLink domain pays off most.

No published measurement compares TP=4 × PP=2 on eight NVL cards with TP=8 on an HGX H200 for the same model, so we give no speed ratio. If your model fits within four cards, the NVL topology costs nothing. We supply servers with two, four or eight H200 NVL; send us the model, its precision and the layout you plan, and our team checks whether it stays inside one NVLink domain.

Published MLPerf® results

The H200 NVL's multi-card result comes from MLPerf® Inference: Datacenter v6.0, closed division, Llama 2 70B (llama2-70b-99): Dell's PowerEdge XE7740 with eight H200 NVL reached 32,004.0 tokens per second in the offline scenario and 29,084.9 in the server scenario, entry 6.0-0021, retrieved from mlcommons.org on 24 September 2026, result verified by MLCommons Association. That is about 4,000 tokens per second per card offline, our division of the system result, not an MLPerf metric. The v6.1 round of 16 September 2026 has no H200 NVL entry. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

We do not place an HGX H200 result next to it, because we could not verify an HGX H200 entry ID in MLCommons' result tables on 24 October 2026, and we publish MLPerf® figures only with a verified entry ID. Llama 2 70B in FP8 fits on one GPU of either kind, so this benchmark mostly reflects compute and memory per GPU, where the paper gap is the 18 per cent above, and says little about the NVLink difference. Our H200 NVL benchmarks put the result in context per user.

Servers, cooling and rack power

The NVL is a standard PCIe part. NVIDIA lists it for "MGX H200 NVL partner and NVIDIA-Certified Systems with up to 8 GPUs", and the makers' documents list it in 2U servers with two cards, such as Lenovo's SR650a V4, and in 4U servers with up to eight, such as Dell's XE7740, HPE's DL380a Gen12 and Cisco's C845A M8, as our list of H200 NVL compatible servers shows. A company can start with two cards and add more in the same chassis. The SXM comes on HGX baseboards with four or eight GPUs, in systems built around the baseboard, and Lenovo's guide notes that "SXM5 GPUs are either air-cooled or water-cooled, depending on the server".

Board power from NVIDIA's H200 page; NVL system estimate from our 8× H200 NVL guide (two AMD EPYC 9555 at 360 W plus 15 per cent); DGX H200 maximum from NVIDIA's DGX H100/H200 user guide as recorded in our H200 buying guide; Lenovo Press LP1944 (7 November 2025).

NVIDIA's H200 NVL announcement of November 2024 put about 70 per cent of enterprise racks at 20 kW or below with air cooling. Such a rack takes two eight-card NVL servers by our estimate, and a DGX H200 at its 10.2 kW maximum uses half of it alone. Dell sets a 30 °C inlet limit for eight H200 NVL in the XE7740, so the room still matters for the NVL. Our guide to GPU rack power and cooling works through the feeds.

NVIDIA AI Enterprise: included with the NVL

NVIDIA's licensing guide, updated 2 September 2026, states: "Each NVIDIA H200 NVL Tensor Core GPU includes a five-year NVIDIA AI Enterprise subscription." The term starts 90 days after the board ships to the server maker, cannot be changed, and is activated with the GPU serial number. NVIDIA's H200 page, read on 24 October 2026, lists NVIDIA AI Enterprise as "Included" for the H200 NVL and as "Add-on" for the H200 SXM. An eight-card NVL server therefore brings eight subscriptions, while an HGX H200 system needs licences bought separately, as our article on activating NVIDIA AI Enterprise explains. Open-source engines such as vLLM run without the subscription on either platform.

Where Blackwell fits: DGX B300 for one domain of eight

For a workload that needs one NVLink domain of eight GPUs, we point to Blackwell rather than to HGX H200, which we do not supply. The NVIDIA DGX B300 has eight Blackwell Ultra SXM GPUs with 2.1 TB of HBM3e, and two NVLink switches with 14.4 TB/s of aggregate bandwidth connect all eight, 1.8 TB/s per GPU by NVIDIA's Blackwell Ultra blog. It computes FP4, holds almost twice the memory of an eight-GPU H200 system and draws about 14 kW by NVIDIA's product page, as our DGX B300 specs detail. We supply the DGX B300 per project: the distribution asks for company, project and intended use, and we quote per project, as our guide to buying a DGX B300 in the EU describes.

The B300 has one limit that matters for some H200 buyers. NVIDIA's HGX page lists FP64 at 10 TFLOPS for an HGX B300 with eight GPUs, against 30 TFLOPS for a single H200 NVL by NVIDIA's H200 page, so double-precision codes stay on Hopper, as our guide to GPUs for CFD simulation shows.

Which to choose by workload

Our recommendations from the figures on this page and our 4× and 8× H200 NVL guides.

For FP4 models, the RTX PRO 6000 Blackwell Server Edition computes NVFP4 in a PCIe server, as our RTX PRO 6000 vs H200 NVL comparison explains. We size the choice per model and rack; describe the model, the users and the power at the rack position, and we reply with the platform and configuration that fit.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the H200 NVL as cards with their 2-way and 4-way NVLink bridges for servers you already run, and build AI servers to order with two, four or eight cards, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. The DGX B300 we supply per project. Our guide to buying H200 NVL in the EU covers the purchase route.

FAQ