2× NVIDIA RTX PRO 6000 Blackwell Server Edition in one server: models, users and power

What two Server Edition cards give a server

A server with two NVIDIA RTX PRO 6000 Blackwell Server Edition cards has 192 GB of GPU memory, of which 166.1 GiB holds model weights and KV cache by our users-per-card rule. That runs Llama 3.3 70B in FP8 for up to 78 conversations of 8K as one tensor-parallel pair, gpt-oss-120b as two independent copies, or a mid-sized model next to a RAG pipeline. The whole server draws about 1.8 kW by our estimate, which one 230 V 16 A feed carries.

The 192 GB are two pools of 96 GB. NVIDIA's product brief for the Server Edition lists NVLink as "Not supported", so the two cards exchange data over their PCIe Gen5 x16 links. The card is passive, takes its air from the server's fans and runs at up to 600 W, or at 450 W in the capped power mode.

NVIDIA RTX PRO 6000 Blackwell Server Edition product page and product brief SP-12355-001_v02 (June 2025), read on 24 October 2026; 95.6 GiB is the 97,887 MiB nvidia-smi reports.

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card for CUDA graphs and activations: 0.9 × 95.6 − 3 = 83.04 GiB per card. What is left after the weights is KV cache, one slice per conversation in flight. The derivation is in our guide to how many users one RTX PRO 6000 can serve.

NVIDIA announced its RTX PRO Servers on 11 August 2025 in designs "capable of supporting two, four or eight" RTX PRO 6000 Blackwell GPUs, among them 2U mainstream servers such as Dell's PowerEdge R7725 with two cards. Two is also the smallest GPU count in NVIDIA's configuration guide for NVIDIA-Certified Systems, and the two-card server is the entry configuration among our AI servers.

Models that fit 166 GiB, as two copies or one pair

Two cards give a model two layouts. Two copies, one per card, keep the cards independent: nothing crosses PCIe while they serve, and a failed card takes out half the capacity. One copy split over both cards with tensor parallelism, TP=2, stores the weights once and leaves far more memory for conversations.

Our estimates by the users-per-card rule: concurrent conversations each holding a full 32,768-token context, 16-bit KV cache unless stated. Weights from the safetensors files on Hugging Face, read on 24 October 2026. Cache per conversation: 2.14 GiB Qwen3.8-27B, 1.125 GiB gpt-oss-120b, 5 GiB Llama 3.3 70B with FP8 cache.

The gpt-oss-120b pair keeps 2 × 83.04 − 60.8 = 105.3 GiB for cache, which at 1.125 GiB per conversation of 32K is 93. DeepSeek-V4-Flash-0731, a 167 GB checkpoint (155.5 GiB), fits on paper but leaves only about 5.3 GiB per card for its cache, which is held in full on every card; the published runs we found use four cards, and we plan it on four. Qwen3-235B-A22B in FP8 (219.8 GiB) and Llama 4 Maverick do not fit two cards, as our LLM hardware requirements by model show.

Conversations per model, and the staff behind them

Llama 3.3 70B is the model most buyers name first, so here it is at the 8K context of a typical chat. NVIDIA's FP8 checkpoint is 72.7 GB, 67.7 GiB, and each conversation of 8K takes 2.5 GiB of 16-bit cache or 1.25 GiB of FP8 cache.

Our estimate by the users-per-card rule, conversations of 8,192 tokens at full length; weights 67.7 GiB in FP8 and 39.8 GiB in NVFP4, NVIDIA's checkpoints of 72.7 and 42.7 GB.

For the pair, 166.08 − 67.7 leaves 98.4 GiB, and 98.4 ÷ 2.5 gives 39 conversations, or 78 with an FP8 cache. One card alone keeps only 15.3 GiB beside the weights, six conversations, which is why a 70B model in FP8 belongs on a pair.

These are requests in flight at the same moment. With the example values of our guide to a private ChatGPT server by company size, 500 employees produce about 20 requests in flight at the peak, which two copies of gpt-oss-120b hold with room to spare. The same guide places a second server from 500 employees so that the service survives a host failure.

We supply this server with two cards and size the layout per model; send us your models, context length and peak requests in flight, and our team replies with a configuration and the conversations for each layout.

TP=2 over PCIe, the split that needs no NVLink

A pair is the easiest way to split a model without NVLink. vLLM's custom all-reduce, its fast path for exchanging partial results between cards, is switched off by default on more than two GPUs without NVLink, and its source logs "Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs". With exactly two cards the custom kernel stays on, provided the peer-to-peer test at start-up passes; otherwise vLLM falls back to NCCL. vLLM 0.30.0 (release notes of 22 September 2026) added an opt-in FlashInfer PCIe IPC all-reduce for servers without NVLink (pull request 53576), which our guide to RTX PRO 6000 without NVLink describes.

Splitting also raises the speed of a single conversation. Each token reads all the weights once, so one card's 1,597 GB/s against 70.6 GB per token caps Llama 3.3 70B in FP8 at about 22.6 tokens per second for one user, by our arithmetic; a pair reads half the weights per card and raises that ceiling to about 45, less the time spent on the exchange. StorageReview measured gpt-oss-120b in NVFP4 at 176.09 tokens per second for one user on a TP=2 pair of Server Edition cards with vLLM.

Peer-to-peer traffic needs a clean path. NVIDIA's NCCL troubleshooting guide states that the driver stack does not support PCIe peer-to-peer transfers with the IOMMU in translated mode, and that ACS on PCIe switches can redirect the traffic to the root complex with "a significant performance reduction or even a hang". A vLLM forum report of April 2026 describes a TP=2 hang on Server Edition cards that --disable-custom-all-reduce resolved. In a two-socket server, nvidia-smi topo -m shows whether the pair sits under one processor or crosses the link between sockets. Our guide to P2P, IOMMU and ACS for NCCL covers the settings and tests in detail.

Published results for two cards

MLPerf® Inference v6.0 holds a two-card and a four-card result from the same HPE server model, so the step from two to four cards can be read directly. MLPerf® Inference v6.1 holds Server Edition results only for four- and eight-card servers.

MLPerf® Inference: Datacenter v6.0, closed division, available category, Llama 2 70B (llama2-70b-99), offline scenario, entries 6.0-0052 and 6.0-0054, retrieved from mlcommons.org on 24 September 2026. Results verified by MLCommons Association. Per card is our division of the system result, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Offline, all queries arrive at once, so the figure measures the server's throughput at full load. The speed each user sees is measured in the server and interactive scenarios, which our RTX PRO 6000 benchmarks add together with the eight-card results.

Chassis for two cards, and chassis that take four

The card runs only in a server whose maker lists it for that model, slot, power cable and fan set. These two-card configurations are in the makers' documents.

Manufacturer documents read on 24 October 2026; the Dell R7725 sheet ("Up to 2 x 450W DW") as read on 9 October 2026 for our list of compatible servers.

A chassis built for two is the smallest, and it stays at two. A chassis for four costs rack space and power supplies on day one, and saves a second server later. The full list, with cables and fan kits, is in RTX PRO 6000 Server Edition compatible servers.

Processors and memory by NVIDIA's rules

NVIDIA's configuration guide for NVIDIA-Certified Systems, updated on 30 September 2026, asks for at least two sockets, six physical cores per GPU, system memory of at least twice the total GPU memory and one NVMe drive per socket. For two cards that is 12 cores and 384 GB. NVIDIA's RTX PRO AI Factory reference architecture asks for seven cores and 128 GB per GPU, which two cards also meet.

Our entry configuration follows the Certified guide: two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives. Two EPYC 9135, 16 cores and 200 W each, give 32 cores and 24 memory channels, so 24 modules of 16 GB make 384 GB with one module per channel. The 32 cores also cover four cards, which matters for the growth step below. Our guide to CPU, RAM and storage for a 4-GPU server explains each tier.

Power on one 16 A feed

Two cards and two small processors stay well inside a single-phase feed. We add 15 per cent to cards and processors for memory, drives, network cards, fans and conversion losses, an assumption that a measurement on the delivered system confirms or replaces.

Our estimate: board power from NVIDIA's brief, 200 W per processor from AMD's EPYC 9135 page, plus 15 per cent for the rest of the server. With two 280 W EPYC 9355, as in our four-card estimate, add about 0.2 kW.

We plan a 230 V 16 A single-phase feed at 80 per cent of its 3.68 kW, which is 2.94 kW. The two-card server at 600 W needs about 1.8 kW, and with redundant power supplies on an A and a B feed each side carries the whole server comfortably. About 1.8 kW of heat needs roughly 290 CFM of air at an 11 °C rise, by the 160 CFM per kW in our guide to GPU rack power and cooling, and NVIDIA's brief asks for 43 CFM through each card at a 25 °C inlet and 125 CFM at 45 °C.

MIG and vGPU on two cards

MIG splits each card into up to four instances of 24 GB, two of 48 GB or one of 96 GB, so two cards offer up to eight isolated GPUs of 24 GB. MIG is set per card: one card can serve gpt-oss-120b whole while the other runs an embedding model, a reranker, speech-to-text and a small task model in quarters. NVIDIA's MIG guide states that "NCCL is currently not supported with MIG", so a TP=2 pair runs on whole cards. Our RTX PRO 6000 MIG profiles show what fits in each size.

The Server Edition is the RTX PRO 6000 edition NVIDIA supports with vGPU, from release 19.0. On vSphere it needs ESXi 8.0 Update 3g or later, and compute vGPU profiles require NVIDIA AI Enterprise, licensed per GPU, as our vGPU profiles guide explains.

Growing from two cards to four

A two-card server can grow to four if three things are planned on the first day.

• Chassis and power supplies. Pick a chassis listed for four cards. HPE's DL380a Gen12, for example, uses the same five power supplies for two or four double-width GPUs, so the step adds only cards and cables. Lenovo's SR650a V4 takes four cards only in its version "Slot Capped at 450W", so the four-card step there means capped cards.
• Memory layout. Four cards need 768 GB by NVIDIA's rule. With 24 modules of 16 GB, every channel is full and the step means replacing modules. Starting with 12 modules of 32 GB leaves room to add 12 more, at half the memory bandwidth until then, which matters little for serving from GPU memory and more for KV cache offloaded to system memory.
• Power feed. Four cards at 600 W need about 3.2 kW, above the 2.94 kW of a 16 A feed. Either the rack position gets 32 A or three-phase, or the four cards run at the 450 W cap, about 2.5 kW, where StorageReview measured a throughput cost of 2 to 9 per cent depending on context length.

Our guide to adding GPUs to a server later covers slots, cables and firmware. Tell us at the quote stage whether the server should take four cards; describe the growth you expect, and we choose the chassis, supplies and memory layout for that step.

Two servers instead of one, or another card

Two two-card servers in different racks keep a service running when one host fails or takes a firmware update, which one four-card server cannot. Our guide to LLM high availability on two GPU nodes sets out the load balancer and failover. Where one model must serve many users at long context, four cards on one server hold more conversations per copy, as 4× RTX PRO 6000 Server Edition works out.

Two H200 NVL cards give 141 GB of HBM3e each at 4.8 TB/s and an NVLink bridge, the stronger choice for a 70B model at high interactive load; they have no FP4 arithmetic. For a desk-side system, two RTX PRO 6000 Max-Q in a workstation are the workstation route; for continuous operation in a rack we recommend the Server Edition.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with two to eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. The configuration, its variants and the ordering steps are on our page to buy a 2× RTX PRO 6000 Server Edition server. Models, RAG and the platform on top are our Private AI/ML service.

FAQ