2, 4 or 8 NVIDIA RTX PRO 6000 Blackwell Server Edition cards: choosing the server size

The decision in one table

For a pilot or one department, two NVIDIA RTX PRO 6000 Blackwell Server Edition cards are enough: they run one model of up to about 150 GB, or two copies of a 70B model, from one 16 A feed. Four cards serve a company-wide assistant or a model of up to about 300 GB. Eight cards are for the large open mixture-of-experts models of 400 to 600 GB, or for many models on one server, and they need a three-phase feed.

Our estimates. Usable memory by our users-per-card rule; users are concurrent conversations of 8,192 tokens with an FP8 KV cache on NVIDIA's 72.7 GB FP8 checkpoint, from one copy per card to one TP=2 pair (two cards), one TP=4 copy (four) or two TP=4 copies (eight); wall power with cards at 600 W, derived in the power section below.

Which size for which company

• Two cards for a pilot, one department or a development and test server next to production: one 70B model as two copies, or one model of up to about 150 GB as a pair, in 2U on a 16 A feed. Our page on a server with two RTX PRO 6000 Server Edition cards covers this size.
• Four cards for a company-wide assistant with RAG, several models side by side, or one model of up to about 300 GB such as Qwen3-235B-A22B. This is our inference node, and two of them give failover.
• Eight cards for the large open MoE models, Llama 4 Maverick, Kimi-K2.6 or DeepSeek-V4.1-Flash, or for a platform team that runs many models and MIG instances on one host. It needs a three-phase feed and a rack that takes 6 to 8 kW in one position.
• Another card when one model must serve many users at low latency across several GPUs: the H200 NVL has HBM3e and NVLink bridges, as our RTX PRO 6000 vs H200 NVL comparison explains.

The card is the same in every size: 96 GB of GDDR7 at 1,597 GB/s, up to 600 W, passive, PCIe Gen5 x16 and no NVLink, as NVIDIA's product brief SP-12355-001_v02 states. What changes with the count is how much memory one model can use, how many copies run side by side, and what the server needs from the room.

Memory: what each size holds

Our users-per-card rule takes 90 per cent of the 95.6 GiB the driver reports and subtracts 3 GiB per card: 83.04 GiB per card for weights and cache. The rule is explained in our guide to how many users one RTX PRO 6000 can serve. The cards do not pool their memory: a model larger than 96 GB is split over two, four or eight cards with tensor or pipeline parallelism, and every copy of a model stores its own weights.

Our estimates by arithmetic, not measurements: concurrent conversations of 32,768 tokens; 16-bit cache for gpt-oss-120b and Kimi-K2.6, FP8 cache for the others; DeepSeek-V4-Flash shown by fit only. Checkpoints: gpt-oss-120b 65.3 GB, DeepSeek-V4-Flash-0731 167 GB, Qwen3-235B-A22B FP8 236 GB, Llama 4 Maverick FP8 about 417 GB, Kimi-K2.6 595 GB, from Hugging Face as in our model guides, read 9 to 24 October 2026.

DeepSeek-V4-Flash-0731 (167 GB) loads on two cards but leaves only about 5 GiB per card for its cache, so we plan it on four. Four cards add Qwen3-235B-A22B and room for several hundred conversations on the smaller models. Eight cards are the only size of the three that holds Llama 4 Maverick or Kimi-K2.6, and our article on which LLMs fit eight RTX PRO 6000 Server Edition cards lists the models up to the 664 GiB limit and the layouts for them.

User numbers grow faster than card numbers when a model is split, because one copy stores its weights once and the rest of the memory becomes cache. Llama 3.3 70B in FP8 holds 12 conversations of 8K per card as one copy per card, 78 per pair and 211 on four cards as one copy. These are requests in flight at the same moment; a company of 500 staff with an assistant in every browser produces far fewer at once.

What MLPerf® shows from two to ten cards

HPE has published MLPerf® results for the same card in servers with two, four, eight and ten cards. Llama 2 70B fits one card in the FP4 weights these runs used, so the results show how throughput grows with the card count for a model that fits one card.

MLPerf® Inference: Datacenter v6.0, closed division, available category, Llama 2 70B (llama2-70b-99), offline scenario, entries 6.0-0052, 6.0-0054, 6.0-0047 and 6.0-0045, retrieved from mlcommons.org on 24 September 2026. Results verified by MLCommons Association. Per card is our division of the system result by its card count, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

By our division, the per-card rate stays between 3,340 and 3,739 tokens per second from two to ten cards. The two smaller systems are a 2U DL385 Gen11, the larger two a 4U DL380a Gen12, and the four-card run had its cards capped at 450 W, so the differences per card also reflect the server model, the power cap and the driver build. HPE's current QuickSpecs for the DL380a Gen12 limit the RTX PRO 6000 Server Edition to eight cards, so treat the ten-card entry as a published result, not an orderable configuration, until HPE lists it. In MLPerf® Inference v6.1 (closed division, Llama 2 70B llama2-70b-99.9, offline, retrieved from mlcommons.org on 24 October 2026, results verified by MLCommons Association), four eight-card Server Edition servers from MiTAC, Cisco, Dell and HPE reached 29,284.8 to 30,034.2 tokens per second, entries 6.1-0070, 6.1-0013, 6.1-0036 and 6.1-0050. Our RTX PRO 6000 benchmarks add the server and interactive scenarios.

For a director, this means: with one copy of a model per card, throughput scales with the card count, so the choice between two, four and eight is made by model size, the number of concurrent users and the room. A model split over all cards scales less well, because every layer waits for traffic over PCIe.

We supply servers with two, four or eight RTX PRO 6000 Server Edition cards; send us your models and the number of concurrent users, and our team replies with the card count and a configuration.

Rack space and chassis for each count

The card runs only in a server whose maker lists it for that model, slot, cable and fan set. These are configurations from the makers' documents.

Lenovo Press LP2263 (28 July 2026), HPE QuickSpecs a50004300enw and a00047453enw, Dell PowerEdge XE7745 technical guide, Cisco UCS C845A M8 data sheet (24 August 2026), Supermicro RTX PRO 6000 systems page and SYS-422GA-NRT datasheet, read on 24 October 2026; the SR675 V3 height as in our list of compatible servers.

A two-card 2U server is the smallest step. Check before buying whether it should take four cards later: Lenovo's SR650a V4 takes four only with the slot-capped 450 W part, so a server bought with two 600 W cards grows to four with capped cards. Lenovo lists eight cards in its ThinkSystem SR675 V3, with 5th Gen AMD EPYC processors only. Dell's technical guide for the XE7745 limits configurations with the RTX PRO 6000 Server Edition to 30 °C ambient air. Our article on a 2U or 4U server for four cards compares the chassis, and RTX PRO 6000 Server Edition compatible servers lists more.

Power per server and the feed it needs

For two cards we assume two AMD EPYC 9135 at a default TDP of 200 W each, as in our entry configuration, for four cards two EPYC 9355 at 280 W and for eight cards two EPYC 9555 at 360 W, and add 15 per cent for memory, drives, fans and conversion losses.

Our estimate: board power from NVIDIA's brief, processor TDP from AMD's EPYC 9135, 9355 and 9555 pages, plus 15 per cent. Feeds planned at 80 per cent: 2.94 kW at 230 V 16 A, 5.89 kW at 230 V 32 A, 8.87 kW at 400 V three-phase 16 A.

Dell's datasheet for its eight-card PowerEdge XE7740 and XE7745 gives about 8 kW per node at full load, more than our estimate, which shows how much fans and processors add in a dense chassis; a measurement on the delivered system replaces either figure. With redundant power supplies on A and B feeds, each side has to carry the whole server. The 450 W cap is what lets a four-card server stay on one 16 A feed, by our estimate, and our guide to RTX PRO 6000 Server Edition power at 600, 450 and 300 W explains what the cap costs. Feeds and plugs are covered in GPU server power: 16 A, 32 A and three-phase.

Processors, memory and NVMe per count

NVIDIA's configuration guide for NVIDIA-Certified Systems, updated on 30 September 2026, covers servers with 2, 4 or 8 GPUs and sets the same rules for each: at least two sockets, six physical cores per GPU, system memory of at least twice the GPU memory, and one NVMe drive of at least 1 TB per socket. NVIDIA's RTX PRO AI Factory reference architecture of 18 May 2026 asks for seven cores and 128 GB per GPU, plus two cores per GPU for the operating system or virtualisation and two per MIG instance.

NVIDIA-Certified Systems configuration guide (30 September 2026) and RTX PRO AI Factory reference architecture (18 May 2026), our multiplication by card count; AI Factory base cores before the extra cores for the operating system and MIG.

Our reference configurations on AI servers follow the Certified guide: the entry server has two Server Edition cards, two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives, and the inference node four cards, 768 GB and four NVMe drives. An eight-card Server Edition server is built to order with at least 1,536 GB. Our guide to how much RAM a GPU server needs explains when more pays off.

Failure domain: one large server or several smaller ones

A two-card server that loses a card loses half its capacity. A four-card server running four copies keeps three quarters, and an eight-card server seven eighths. A host fault, a firmware update or a driver change, however, stops every card in the server at once, whatever its size.

That makes the card count a question of availability as well. One eight-card server is the densest choice and one failure domain. Two four-card servers cost more rack units and network ports, but a load balancer in front of them keeps half the service running through a fault or a maintenance window, and a model of up to 300 GB still fits each of them. A large model that needs eight cards cannot be split that way, so a service that has to survive a host fault with such a model needs two eight-card servers. Our comparison of one 8-GPU server or two 4-GPU servers and the guide to LLM high availability with two nodes work through the designs.

For isolated workloads, MIG splits each card into up to four instances of 24 GB, so the three sizes offer 8, 16 or 32 isolated GPUs. Our team plans card count, chassis and feed together; describe your users, models and availability target, and we propose one or two servers with the figures for each.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with two to eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, on one EU contract and invoice, with manufacturer warranty on every component. Quantities and lead time are confirmed in writing with the quote. The pages for two cards, four cards and eight cards go into each size, and buy a 4× RTX PRO 6000 Server Edition server describes the ordering steps.

FAQ