Four NVIDIA RTX PRO 5000 Blackwell 72 GB in one workstation: what 288 GB runs

288 GB in four cards: the figures

A workstation with four NVIDIA RTX PRO 5000 Blackwell 72 GB cards has 288 GB of GPU memory and, by our sizing rule, runs models with up to about 220 GiB of weights: Llama 3.3 70B in FP8 for 88 concurrent conversations as two TP=2 pairs, gpt-oss-120b on each pair, or Qwen3-235B-A22B-2507 in FP8 split over all four cards. It draws about 1.8 to 2 kW at the wall by our estimate, which one 230 V office circuit carries. Four RTX PRO 6000 Max-Q in the same tower give a third more memory at the same board power, and the comparison further down shows when the RTX PRO 5000 build is still the right purchase.

Each card has 72 GB of GDDR7 with ECC at 1,344 GB/s and 300 W, in a dual-slot board of 4.4 × 10.5 inches with an active cooler, according to NVIDIA's product page, read on 24 October 2026. Unlike the Max-Q page, which states that the card "enables up to four GPUs in a single system", the RTX PRO 5000 page says nothing about how many cards a system takes, so the count per tower comes from the system makers.

NVIDIA RTX PRO 5000 product page (memory, bandwidth, AI TOPS, 300 W, board size), read 24 October 2026. Driver figure scaled from the 95.6 GiB that a 96 GB card reports; usable memory by our users-per-card rule with ECC off.

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card for activations and the runtime: 0.9 × 71.7 − 3 = 61.5 GiB per card. Whatever the weights leave is KV cache for the conversations in flight. The rule and its derivation are in our guide to how many users one RTX PRO 6000 can serve. With ECC reserving memory, usable space would fall to about 57.5 GiB per card, as our RTX PRO 5000 48 GB or 72 GB comparison explains.

Four pools, no NVLink

The four cards do not merge into one 288 GB GPU. The RTX PRO 5000 has no NVLink, so each card addresses its own 72 GB and the cards exchange data over PCIe 5.0. A model larger than one card has to be split by the serving engine, and that leaves three practical layouts on four cards: four independent copies, two pairs with tensor parallelism (TP=2), or one copy over all four (TP=4). Pipeline parallelism, where each card holds a share of the layers, is the alternative that vLLM's parallelism guide recommends for GPUs without NVLink, "for higher throughput and lower communication overhead".

The layouts differ in how vLLM communicates. By default its source code disables vLLM's own custom all-reduce "on more than two PCIe-only GPUs", so a TP=2 pair keeps the faster kernel and a TP=4 split falls back to NVIDIA's NCCL library. vLLM 0.30.0 (release notes, 22 September 2026) added an opt-in FlashInfer PCIe IPC all-reduce for systems without NVLink (#53576). Both depend on peer-to-peer transfers between the cards; NVIDIA's NCCL documentation warns that IO virtualisation can redirect that traffic through the processor, "causing a significant performance reduction or even a hang". Our articles on TP=2, TP=4 or replicas without NVLink and on P2P checks on Linux cover the BIOS and kernel settings.

Models that fit in 288 GB

Four 72 GB cards open the class of models between about 125 and 220 GiB of weights, which two cards cannot hold, and keep every smaller model on one card or one pair.

Our estimates by the users-per-card rule, 8K tokens with an FP8 KV cache unless stated; gpt-oss-120b at 32K with a 16-bit cache of 1.125 GiB. Weights as in our model guides and our four-card Max-Q article; Qwen3-235B cache from its config.json on Hugging Face (94 layers, 4 KV heads, head size 128), read 24 October 2026: 0.73 GiB per 8K conversation in FP8.

Two rows need a comment. gpt-oss-120b loads on one card only on paper: 0.7 GiB is left, which holds no conversation, so it belongs on a pair. Qwen3-235B-A22B in NVFP4, about 124.8 GiB, misses a pair by 1.7 GiB and therefore takes all four cards, where it leaves 121 GiB for cache. The FP8 version of the same model, about 219.8 GiB, is the largest model in the table: over four cards it leaves 26.3 GiB, enough for about 35 conversations of 8K tokens by our rule, which suits a small team rather than a department. GLM-4.5 in FP8, at 361 GB, and larger models do not fit. For other models and sizes, see our LLM hardware requirements by model.

Four copies, two pairs or one model over four cards

The layout trades capacity against independence. Four copies need no traffic between the cards, and one failed card leaves three quarters of the service running. One copy over four cards stores the weights once, so far more memory is left for conversations.

Our estimates by the users-per-card rule with 61.5 GiB usable per card; 8K tokens with FP8 KV cache, gpt-oss-120b at 32K with a 16-bit cache. Memory arithmetic, not measured throughput.

These are requests in flight at the same moment, far fewer than the staff who use a service. The example values in our guide to a private ChatGPT server by company size put about 20 requests in flight at peak for 500 staff, 40 for 1,000 and 80 for 2,000. By memory, two pairs serving Llama 3.3 70B in FP8 cover the largest of these cases; Qwen3-32B as four copies covers it too, with each card independent.

A mixed layout is often the practical one. One pair serves Llama 3.3 70B in FP8 for 44 conversations, the third card runs Qwen3-32B for coding with 29, and the fourth stays free for embeddings, a reranker or tests. Send us the models your team wants loaded at the same time and the number of people who use them, and we work out the layout on four cards before quoting the workstation.

Speed at 1,344 GB/s per card

Generation speed for one user has a ceiling set by bandwidth, because each token reads all active weights once. By our arithmetic, one RTX PRO 5000 generating Llama 3.3 70B in NVFP4, about 40.6 GB per token, is held to about 33 tokens per second. In FP8 on a TP=2 pair each card reads half of 70.6 GB, so the ceiling is about 38 tokens per second, and under TP=4 about 76 before the cost of the all-reduce over PCIe. Qwen3-32B in FP8 on one card, about 34.4 GB per token, has a ceiling of about 39. Real decoding reaches a fraction of these bounds; they are useful for comparing cards, not as forecasts.

Prompt processing and batch throughput depend on compute. NVIDIA rates the card at 2,064 AI TOPS in FP4 with sparsity, against 3,511 for the RTX PRO 6000 Max-Q, so long RAG prompts and coding-agent contexts take longer on the RTX PRO 5000 at the same card count. We found no published measurement of tensor-parallel LLM serving on four RTX PRO 5000 cards. The published results for single cards are collected in our RTX PRO 5000 benchmarks; a load test with your own model on the configured system is the reliable figure.

MIG halves and vGPU on Red Hat KVM

NVIDIA's product page gives the RTX PRO 5000 "up to two (2) fully isolated instances", and the datasheet sizes them at 36 GB on the 72 GB card, so four cards could offer eight isolated GPUs of 36 GB. NVIDIA's MIG user guide, updated on 13 October 2026, lists only the 48 GB version so far, so the profile names for the 72 GB card are read on the delivered card with nvidia-smi mig -lgip. A 36 GB half holds Qwen3-14B in FP8 with cache for about 22 conversations by our rule, or a 32B model in 4-bit for about nine. MIG needs Linux, driver 575.51.03 or later and the card switched to compute mode, which turns its display outputs off, and NVIDIA's MIG guide states that NCCL is not supported with MIG, so a split model needs whole cards.

The 72 GB card is on NVIDIA's vGPU list from vGPU 20.2 of August 2026. Of the hypervisor pages we read on 24 October 2026, only NVIDIA's validated platforms for Red Hat Enterprise Linux with KVM list it, with the graphics editions vWS and vPC; the VMware vSphere and Ubuntu support matrices of 29 September 2026 do not. For virtual machines on vSphere, our guide to MIG and vGPU: VMs per GPU points to the Server Edition cards.

Towers, lanes and power on 230 V

As of 24 October 2026 Dell's Precision 7960 configurator offers four RTX PRO 5000 72 GB, and Lenovo lists the card for the ThinkStation PX without stating a maximum. HP's Z8 Fury G6i page lists four Max-Q and no RTX PRO 5000. Two of the Precision 7960's four GPU slots run at PCIe 4.0 x16, which matters for a TP=4 layout more than for four copies. The table of towers and the components we build with are on our page to buy an RTX PRO 5000 72 GB workstation.

Four cards at x16 take 64 PCIe lanes; AMD lists 128 PCIe 5.0 lanes and eight memory channels for the Threadripper PRO 9975WX. NVIDIA's configuration guide for certified systems asks for system memory of at least twice the GPU memory, 576 GB here, which we plan as 768 GB in eight modules. Power by our estimate: 1,200 W for the cards, 350 W for the processor and 15 per cent for memory, drives, fans and supply losses come to about 1.8 to 2 kW at the wall, 7.8 to 8.7 A on 230 V. A 16 A circuit planned at 80 per cent carries 2.94 kW. The heat, about 6,800 BTU/h at 2 kW, belongs in a technical room or an air-conditioned office; our article on RTX PRO 6000 Max-Q power on 230 V covers cords and UPS sizing for the same 300 W class.

Four RTX PRO 5000 72 GB against four RTX PRO 6000 Max-Q

Both cards are 300 W blower designs in two slots, so the same tower, power supply and circuit carry either build. On every published figure that decides LLM work, the Max-Q build is ahead.

NVIDIA product pages and datasheets, MIG user guide (13 October 2026) and vGPU supported-GPU list (2 October 2026); conversations by our users-per-card rule.

A Max-Q holds Llama 3.3 70B in FP8 on one card, so four Max-Q can run four independent copies where the RTX PRO 5000 build needs pairs. Per card the Max-Q has a third more bandwidth and about 70 per cent more FP4 compute. Published measurements point the same way: Puget Systems found the 72 GB card 7 per cent behind the Max-Q in Topaz Video 1.6.1 in August 2026, as our benchmarks article reports. MIG is no argument for the RTX PRO 5000 either, since the Max-Q offers 48 GB halves as well as 24 GB quarters.

Two reasons remain to choose the RTX PRO 5000 72 GB. The first is vGPU on Red Hat KVM, for a deskside host that runs graphics virtual machines; the Max-Q is not on NVIDIA's vGPU list. The second is price: when the models fit 72 GB per copy or per pair, the difference between the two quotes decides, and we quote both configurations side by side so it is on paper before you choose. Our comparison of two RTX PRO 5000 72 GB or one RTX PRO 6000 Max-Q makes the same decision at a smaller size, and four RTX PRO 6000 Max-Q in one workstation covers the other build.

When another build fits better

• Models up to 32B per copy and many developers: four RTX PRO 5000 48 GB, with eight MIG instances of 24 GB that NVIDIA's MIG guide lists.
• One 70B model in FP8 per card, or 235B-class models with room for many users: four RTX PRO 6000 Max-Q.
• A service running continuously for several departments: a rack server with RTX PRO 6000 Server Edition cards, passive, with redundant supplies and vGPU on more hypervisors; for the RTX PRO 5000 in a rack, see our page on an RTX PRO 5000 rack server.
• 84 GB per card: the RTX PRO 5500, whose specifications NVIDIA still marks as preliminary; our RTX PRO 5500 vs RTX PRO 5000 72 GB comparison sets out what is published.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build four-card RTX PRO 5000 72 GB workstations to order on Threadripper PRO or Xeon W platforms, assembled and tested before delivery, with the operating system, driver, CUDA and a container runtime installed on request, and we supply the cards for a tower your company already runs. Name the models and the tower you prefer, and we confirm four cards, lanes and the circuit before quoting. Models, retrieval on your documents and the platform on top are our Private AI/ML service.

FAQ