Eight NVIDIA H200 NVL in one server or an NVIDIA DGX B300 for a first large AI system

H200 NVL vs B300: the decision in one table

For a first large AI system, a server with eight NVIDIA H200 NVL fits when each model copy stays under about 490 GiB, when the room offers an ordinary three-phase rack position, or when the same machine also runs FP64 simulation. An NVIDIA DGX B300 fits when a model of 700 GB or more must serve many users, when FP4 checkpoints should run natively, or when the team fully fine-tunes 70B models and trains across all eight GPUs. The DGX delivers about 3.4 times the throughput of the eight-card server on the one benchmark both have published, and it needs a rack built for about 20 kW per system.

Both machines hold eight GPUs, and that is where the likeness ends. The table gathers the figures that decide between them; the sections below explain each row.

NVIDIA H200 product page (H200 NVL column) and DGX B300 product page (updated 15 October 2026), read on 24 October 2026; 62 TB/s and 14.5 to 15.1 kW from NVIDIA's DGX B300 datasheet and user guide as quoted in our DGX B300 specs article; HGX B300 FP64 from NVIDIA's HGX page as quoted in our FP64 article. The DGX B300 product page gives "~14 kW"; we plan with the higher datasheet and user-guide figures. The 2.1 TB is NVIDIA's system figure; the user guide adds the 288 GB per GPU up to 2.3 TB, and we plan with 2.1 TB. Dense FP8 of the H200 NVL is our halving of NVIDIA's sparse 3,341 TFLOPS per card; usable memory and wall power are our estimates, explained below.

Two NVLink domains or one

The eight H200 NVL in a server are not one pool. NVIDIA's NVLink bridge for this card joins two or four cards, so eight cards form two domains of four, and the two domains exchange data over PCIe Gen5. NVIDIA lists 900 GB/s per GPU over the bridge and 128 GB/s over PCIe Gen5. Dell's technical guide for the PowerEdge XE7740 states that eight double-wide H200 cards need two 4-way bridges, as our article on eight H200 NVL in one server quotes.

The DGX B300 connects its eight Blackwell Ultra GPUs through two NVLink switches, 14.4 TB/s in aggregate and 1.8 TB/s per GPU by NVIDIA's Blackwell Ultra figures. Every GPU reaches every other at the same speed, so a model split eight ways with tensor parallelism keeps all of its traffic on NVLink.

The difference shows in the layouts each machine runs well:

• On the H200 NVL server, a model up to one domain runs as TP=4, and two copies share the server. A larger model runs as TP=4 × PP=2, one pipeline stage per domain, so only the activations between the two halves cross PCIe once per step. OVHcloud's guide for its eight-card H200 NVL instances advises against a tensor-parallel size of eight.
• On the DGX B300, TP=8 is the plain layout, and gradient exchange in training runs over NVLink between all eight GPUs.

For serving models that fit one domain, the two-domain layout costs little: each copy runs inside its own NVLink bridge. It costs more when one model must span both domains at interactive speed, and most when eight GPUs train one model together, because every step then exchanges gradients across the PCIe link. Our article on the H200 NVL NVLink bridge, 2-way or 4-way covers the bridge rules.

Memory: which models need the DGX B300

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per GPU; what remains after the weights holds KV cache. For the H200 NVL, 0.9 × 140.4 − 3 = 123.4 GiB per card, 986.9 GiB for eight. For the B300 we read the 270 GB that MLPerf® entries give the accelerator cautiously as 10⁹ bytes, 251.5 GiB, which leaves 223.3 GiB per GPU and about 1,786 GiB per system. The rule is explained in our guide to how many users one GPU serves.

Concurrent conversations at full length by our users-per-card rule; Llama 3.3 70B with an FP8 cache of 1.25 GiB per 8K conversation, the others with a 16-bit cache unless stated. Weights: sums of the Hugging Face safetensors files as in our model guides, read on 24 October 2026. DGX B300 at 223.3 GiB per GPU, our cautious reading. Llama row for the DGX: 223.3 − 67.7 = 155.6 GiB per GPU, 124 conversations, eight copies.

The first row is a model both machines serve well. The H200 NVL server holds 680 conversations of 8K as two copies; the DGX holds about half as many again with one copy per GPU. For a company whose largest model is a 70B or a 120B model, the extra memory buys users, not new models.

The middle rows are the models of 700 to 830 GiB. They fit the H200 NVL server only as one copy over both domains, with cache left for a few dozen conversations of 32K in the case of GLM-5.3. On the DGX the same weights leave room for several times as many. The last row does not fit eight H200 NVL at all. The figures for these models, their cache types and layouts come from our guides to GLM, DeepSeek-V4-Pro on eight H200 NVL or a DGX B300 and Muse Spark Open.

FP4, FP8 and FP64 on each system

Precision separates the two machines as clearly as memory does. Hopper has no FP4 arithmetic. An NVFP4 checkpoint built for Blackwell runs on the H200 NVL only as a weight-only format, stored in 4 bits and computed at 16-bit, as our guide to FP4 models on H200 NVL and Ada GPUs explains. The DGX B300 datasheet lists 108 PFLOPS of dense FP4 for the system.

In FP8, NVIDIA lists 3,341 TFLOPS per H200 NVL with sparsity, which is about 1,671 dense; eight cards give about 13.4 PFLOPS dense by our arithmetic. The DGX B300 lists 72 PFLOPS of FP8 with sparsity, and the page's footnote says dense is half: 36 PFLOPS, 2.7 times the H200 NVL server.

FP64 runs the other way. NVIDIA rates each H200 NVL at 30 TFLOPS of FP64 and 60 TFLOPS on its FP64 Tensor Cores, so eight cards list 240 TFLOPS. NVIDIA's HGX table gives 10 TFLOPS of FP64 for all eight B300 GPUs together. A research institute that runs VASP, Quantum ESPRESSO or an in-house double-precision solver next to its AI work keeps that work on Hopper, as our FP64 GPU server comparison shows.

Published MLPerf® results for both systems

Both systems have entries in MLPerf® Inference: Datacenter, closed division, on Llama 2 70B, but from different rounds: the DGX B300 in v6.1 of 16 September 2026, eight H200 NVL in v6.0, because v6.1 contains no H200 NVL entry.

MLPerf® Inference: Datacenter v6.1 and v6.0, closed division, available category, Llama 2 70B, offline and server scenarios, entries 6.1-0071 (NVIDIA DGX B300) (retrieved from mlcommons.org on 24 October 2026) and 6.0-0021 (Dell PowerEdge XE7740 with eight H200 NVL, llama2-70b-99, retrieved from mlcommons.org on 24 September 2026). Results verified by MLCommons Association. The ratios are our division of the system results, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Per GPU, our division gives 13,660 tokens per second for the B300 and 4,001 for the H200 NVL, again not an MLPerf metric. Two rounds apart, the software stacks differ as well as the hardware, so read the ratio as an order of magnitude. The DGX B300 also has a gpt-oss-120b result in v6.1, 106,637 tokens per second offline, for which no H200 NVL counterpart exists.

Throughput per kilowatt narrows the gap. Dividing the offline results by the DGX datasheet's 15.1 kW and by our 6.3 kW estimate for the H200 NVL server gives about 7,200 and 5,100 tokens per second per kW, a factor of 1.4. Neither power figure is a measurement from these runs, so this is a rough indication by our arithmetic. Our article on DGX B300 and a server with eight RTX PRO 6000 compares the DGX with the PCIe Blackwell server on the same benchmarks.

We build the eight-card H200 NVL server to order and quote the DGX B300 per project; send us your models, their precision and the peak number of users, and our team replies with the layout and the figures for both systems.

Power, weight and the room

The room often decides before the benchmarks do. An eight-card H200 NVL server is a 4U PCIe machine, such as Dell's PowerEdge XE7740 or Cisco's UCS C845A M8. By our estimate, eight cards at 600 W and two AMD EPYC 9555 at 360 W each, plus 15 per cent for the rest, draw about 6.3 kW at the wall, which needs a 400 V three-phase position; a measurement on the delivered system confirms or corrects the estimate.

Our estimate for the H200 NVL server (cards, processors, 15 per cent for the rest; 160 CFM per kW at an 11 °C rise from our rack guide); Dell XE7740 technical guide Rev. A04 for the inlet limit. DGX B300 user guide (updated 5 October 2026) and NVIDIA's "Data Center Best Practices with DGX B300" (version 1.0, 27 February 2026), as quoted in our DGX B300 specs article.

At its 19.7 kW peak, one DGX B300 draws about as much as three eight-card H200 NVL servers by our estimate. NVIDIA's data-centre guide plans two AC systems per rack at 39.4 kW peak and recommends active rear-door heat exchangers for such racks, and its user guide states that the system "must be installed by NVIDIA partner network personnel or NVIDIA field service engineers". The H200 NVL server fits a rack with a 20 kW budget, two per rack with room for switches. Our guides to GPU server power on 16 A, 32 A and three-phase and the DGX B300 specs and power requirements give the feed and rack details.

Fine-tuning and training on eight GPUs

Training needs far more memory than serving. Mixed-precision Adam holds about 16 bytes per parameter before activations, so the model states of Llama 3.3 70B come to about 1,129 GB. On eight H200 NVL that is 141.1 GB per card, as much as the card holds, so a full fine-tune needs CPU offload, as our article on the eight-card H200 NVL server works out. On a DGX B300 with about 2.1 TB, the same states leave about 970 GB for activations by our arithmetic, roughly 120 GB per GPU.

LoRA and QLoRA of a 70B model fit both machines with room to spare. The difference for those runs is the link: a job that spans more than four H200 NVL sends its gradient traffic across PCIe between the two domains, while the DGX keeps it on NVLink. A team that mostly adapts open models with LoRA runs well on the H200 NVL server, keeping each job inside one domain. A team that pre-trains, fully fine-tunes 70B models or trains across eight GPUs every day gains most from the DGX.

MIG, software and support

Both systems split their GPUs with MIG into up to seven instances each, 56 per system. NVIDIA's MIG user guide, updated 13 October 2026, lists 1g.18gb up to seven times on the H200 141 GB product; NVIDIA AI Enterprise 8.2 lists 1g.34gb up to seven times on the B300. A DGX instance of the smallest size has about twice the memory. Our H200 NVL MIG profiles list every size.

The software bundles differ in a way that belongs in the quote. NVIDIA's H200 product page lists the NVIDIA AI Enterprise add-on as "Included" with the H200 NVL, a five-year subscription per card. The DGX B300 product page lists DGX OS as included and NVIDIA AI Enterprise and NVIDIA Mission Control as "Recommended", and NVIDIA's licensing guide states that AI Enterprise licences are purchased separately for DGX systems with Blackwell GPUs. The DGX comes with three-year business-standard support from NVIDIA. For the H200 NVL server, we provide manufacturer warranty on every component, handled through us, and install the operating system, drivers, CUDA and a container runtime on request.

How each system is ordered in the EU

The two orders run differently. The H200 NVL server is built to order: we send a written quote, the server is assembled and tested before delivery, and the order runs on one EU contract and invoice; quantities and lead time are confirmed in writing with the quote. Our page to buy an 8× H200 NVL server shows the configuration and variants.

The DGX B300 we supply per project. Before we can quote, the distribution asks us for the company, the project and the intended use, and the approval takes time. We check price and availability for your project, and our guide to buying a DGX B300 in the EU lists what the enquiry should contain and what a DGX quote should state.

Which one fits your use case

• A company assistant or RAG service on 70B to 235B models for hundreds of users: the H200 NVL server. Each copy fits one NVLink domain, and two copies share the server.
• AI next to FP64 simulation in one institute: the H200 NVL server, with 24 times the FP64 rating of the B300 system.
• A room with 16 to 20 kW per rack and no rear-door cooling: the H200 NVL server.
• Models of 700 GB or more for many users, or FP4 checkpoints at full speed: the DGX B300.
• Full fine-tuning of 70B models or daily training across eight GPUs: the DGX B300.

Between the two lies a second H200 NVL server. Two eight-card servers run one large model as four pipeline stages, or survive the loss of one server when they run two copies, as our comparison of 1T models for 100 users on a DGX B300 or two H200 NVL servers works out. The step beyond the DGX B300, and how it differs from the HGX B300 servers that makers build, is in DGX B300 vs DGX B200 vs HGX B300. If you are unsure which side of the line your models fall on, tell us the models, the context length and the rack position, and we compare both systems for your case.

What we supply

You can buy the eight-card H200 NVL server from us: our team sizes the configuration to your workload and budget and replies within one business day. Our training node has eight H200 NVL with two 4-way NVLink bridges, two AMD EPYC processors, 2.3 TB of DDR5 ECC, eight NVMe drives and 400G networking, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. The cards are on our H200 NVL page. The NVIDIA DGX B300 we supply per project; we check rack, power and airflow before we quote either system, and recommend an acceptance test on site, as our guide to GPU server acceptance testing describes.

FAQ