H200 NVL NVLink bridge, 2-way or 4-way: NVLink domains, topology and what changes for LLMs

H200 NVL NVLink bridge: 2-way and 4-way in short

Each H200 NVL card takes one NVLink bridge. A 2-way bridge joins two adjacent cards and a 4-way bridge joins four, and NVIDIA’s H200 page gives both 900 GB/s per GPU, against 128 GB/s for PCIe Gen5. The bridged cards form an NVLink domain of two or four cards with 282 or 564 GB of HBM3e between them. Since a bridge stops at four cards, an eight-card server holds two 4-way domains or four 2-way domains, and traffic between the domains runs over PCIe. The bridges are separate parts, and the server maker has to list the bridge type for the exact server model.

For a language model, the domain sets how far tensor parallelism reaches at NVLink speed and how much memory one copy of a model can use without crossing PCIe. The traffic of each parallelism method is set out in our article on one model on several GPUs over PCIe and NVLink, and the card counts per model in our H200 NVL guide for 235B to 671B models.

NVLink bandwidth per GPU against PCIe Gen5

NVIDIA’s H200 page lists the interconnect of the H200 NVL as “2- or 4-way NVIDIA NVLink bridge: 900GB/s per GPU” and “PCIe Gen5: 128GB/s”, in a specification table that NVIDIA marks as preliminary. Both figures count the two directions together, so a card sends at most 450 GB/s over the bridge and 64 GB/s over a Gen5 x16 slot in each direction. NVIDIA’s product brief (PB-12128-001_v01, April 2025) allows one NVLink bridge per card with 18 NVLink links, with a “Total maximum NVLink bandwidth” of 900 GB/s per card.

NVIDIA’s developer blog of 11 December 2024 calls the 2-way bridge “7x faster than PCIe Gen5”, total against total, while the product brief’s “14×” sets 900 GB/s against one direction of PCIe Gen5 x16. The brief and the blog also give 1.8 TB/s for the 4-way bridge, while the H200 page keeps 900 GB/s per GPU for both bridges. Since the brief caps each card at 900 GB/s, plan with that figure per GPU for either bridge. Our checklist for an H200 NVL in an existing server covers where NVIDIA’s tables differ and the clearance rules for the bridge.

We found no NVIDIA document that states how a card’s 18 links are divided among its three peers on a 4-way bridge. ServeTheHome reported in November 2024 that the bridge uses no NVLink switch, so by our reading each card shares its links among its peers, and a copy between two cards of a 4-way set uses fewer links than inside a 2-way pair. On a delivered server, the matrix of nvidia-smi topo -m shows the count per pair.

NVLink domains in two-, four- and eight-card servers

Memory is 141 GB per card; bridge spans from NVIDIA’s H200 NVL Product Brief PB-12128-001_v01, Table 4-2. Typical use is our reading of the card counts in our H200 NVL sizing guide and fine-tuning comparison, by weights alone, before the KV cache.

The driver reports 140.4 GiB per card, so a 2-way domain offers 280.8 GiB and a 4-way domain 561.6 GiB for weights, KV cache and runtime. Section 4.5 of NVIDIA’s product brief sets the layouts. It states that “bridged NVIDIA H200 NVL cards should be within the same CPU domain”. Its recommended topology bridges GPUs “under the same CPU or PCIe switch”, places “the same (even) number of GPUs under each CPU socket” and keeps “identical bridge slot-spans for all CPU domains”. The brief lists bridging GPUs under different CPUs as allowed but not recommended.

In a two-socket server with eight cards, these rules give one 4-way set or two 2-way pairs under each processor. In a two-socket server with four cards, two under each processor, they give two 2-way pairs. A 4-way bridge across those four cards would span both sockets, which the brief calls “less optimal for many workloads” but allows “if a balanced topology is maintained”. NVIDIA’s enterprise reference architecture for the H200 NVL uses a “PCIe Optimized 2-8-5” server: two CPUs, eight GPUs and five network adapters. Since the brief allows one bridge per card, spanning at most four, the two sets of an eight-card server exchange data over PCIe.

Bridge part numbers and server qualification

The bridges are ordered as parts of their own. NVIDIA’s product brief lists the 2-way bridge as 900-23945-xx00-xxx and the 4-way bridge as 900-23946-xx00-xxx, at 49 g and 128 g. NVIDIA lists the server options for the card as “NVIDIA MGX H200 NVL partner and NVIDIA-Certified Systems with up to 8 GPUs”, and the server maker’s own list decides which bridge a given model takes.

HPE’s QuickSpecs for NVIDIA accelerators (version 83, 8 September 2026) show the card and its bridges as separate options. They list the H200 NVL for the ProLiant DL380a Gen12, the DL380 Gen12 and the DL385 Gen11. The 2-way bridge, S4A90C, is listed for the DL380a Gen12 and the DL385 Gen11, and the 4-way bridge, S4A91C, for the DL380a Gen12 only. A server listed for the card is therefore not necessarily listed for a bridge. HPE’s DL380a Gen12 QuickSpecs add that the bridges “can only be supported with H200 NVL GPUs” and that the “10DW configuration does not support GPU NVL bridges”. Moving from 2-way pairs to a 4-way set later needs four adjacent cards, so the slot layout follows the bridge type. Order the bridges with the cards; our guide to buying the H200 NVL in the EU covers the other decisions before the order.

We supply the H200 NVL with its 2-way and 4-way bridges, as cards or in AI servers built to order. Tell us the server model and how many cards each bridge should join, and we check the rack, power and airflow before we quote.

What a domain changes for LLM inference

Inside a domain, the cards can run tensor parallelism over NVLink: a tensor-parallel size of 2 on a 2-way pair and 4 on a 4-way set. Tensor parallelism all-reduces activations twice per transformer layer in every forward pass. For Llama 3.3 70B, our PCIe and NVLink article puts that at 2.5 MiB per token per GPU at a size of 2 and 3.75 MiB at a size of 4. A 2,000-token prompt then moves about 4.9 and 7.3 GiB per GPU, some 82 and 123 ms at the 64 GB/s that PCIe Gen5 x16 carries each way. Over the bridge, at 450 GB/s each way, the same transfers take no less than about 12 and 17 ms, by our arithmetic.

A 4-way set moves more data per card than a 2-way pair. NVIDIA’s nccl-tests documentation computes the all-reduce bus bandwidth with the factor 2 × (n − 1) / n, which is 1 for two GPUs and 1.5 for four. In return, four cards share the weight reads of each token, and one copy of the model gets 561.6 GiB. For a model that fits 280.8 GiB, the choice is between two 2-way pairs, each serving its own copy, and one 4-way set whose one copy leaves the memory of two more cards to the KV cache.

Between the two domains of an eight-card server, tensor parallelism would put PCIe into every layer. vLLM’s documentation shows --tensor-parallel-size 4 with --pipeline-parallel-size 2 for eight GPUs and advises pipeline parallelism where GPUs “do not have NVLINK interconnect”. Applied to two bridged sets, by our reading, this keeps tensor parallelism inside each set and sends only stage-boundary activations over PCIe. The other option is two copies, one per domain, which exchange nothing; pipeline parallelism across both is needed only when one copy does not fit 564 GB, as with DeepSeek-V3 in FP8.

We size H200 NVL servers from the model and its users, replicas or bridged sets. Send us the model, its precision and the peak number of users, and the configuration and quote, with the bridge layout, follow within one business day.

Fine-tuning across bridged cards

In data-parallel fine-tuning, every card holds a full copy and only the gradients of the trained weights are all-reduced at each step, which for a LoRA adapter is little traffic. Sharded runs differ, because FSDP gathers the sharded weights before the forward and before the backward pass. Our fine-tuning comparison of the H200 NVL and RTX PRO 6000 puts a 70B LoRA sharded over two cards at about 141 GB received per card for every forward and backward pass, at least 2.2 seconds over PCIe Gen5 and 0.3 seconds over a 2-way bridge at nominal rates.

On four cards, each card receives about 212 GB per pass in the same example, at least about 0.47 seconds at 450 GB/s each way, by our arithmetic; FSDP hides part of these transfers behind computation. Keep a sharded run inside one domain where its memory allows, because collectives across two domains pass over PCIe. An eight-card server with two 4-way sets can run two sharded jobs side by side, one per domain.

How to check the NVLink topology on a delivered server

Run these checks before the frameworks go on, and keep the output.

Commands and output terms from NVIDIA’s nvidia-smi documentation and the nccl-tests README and PERFORMANCE.md; expected results are our reading of them and of NVIDIA’s bridge rules.

The nvidia-smi legend defines NV# as a “Connection traversing a bonded set of # NVLinks” and SYS as a path across “the SMP interconnect between NUMA nodes”. On a 4-way set, the number after NV shows how many links each pair has. The -mp matrix leaves NVLink out, so it shows whether the cards of one set sit under the same processor, as NVIDIA’s brief recommends.

Address the cards of one set by PCI bus ID or UUID, as the nvidia-smi documentation recommends, because the enumeration order can change between reboots. CUDA numbers the GPUs by a heuristic from fastest to slowest unless CUDA_DEVICE_ORDER=PCI_BUS_ID is set, which enumerates them by PCI bus ID; set it before choosing the cards of a set with CUDA_VISIBLE_DEVICES. Limit nccl-tests to the cards of one set with that variable, run it per set, then across all eight. The nccl-tests documentation states that its bus bandwidth can be compared “with the hardware peak bandwidth, independently of the number of ranks used”. It adds that bus bandwidth “should reflect the speed of the hardware bottleneck”, so by our reading a result for one set at or below the 64 GB/s of PCIe means the traffic is not using the bridge.

What we supply

We supply the NVIDIA H200 NVL with its 2-way and 4-way NVLink bridges and its five-year NVIDIA AI Enterprise subscription, as cards for a server whose maker lists them or in AI servers built to order with 2, 4 or 8 cards. Bridges go into the same quote as the cards. A new server is planned with the slot layout the bridge type needs, assembled and load-tested before shipment, with manufacturer warranty on every component. For a server you already run, we check the platform, power and cooling before you order. Operating system, drivers, CUDA and a container runtime are installed on request, and cards from our professional GPU range come on the same EU contract and invoice where a platform mixes them.

FAQ