8× RTX PRO 6000 Server Edition or 8× RTX PRO 4500 Server Edition: memory, users, power

Same eight slots, two different servers

A server with eight NVIDIA RTX PRO 6000 Blackwell Server Edition cards and one with eight RTX PRO 4500 Blackwell Server Edition cards share a card count and little else. The server with RTX PRO 6000 Server Edition cards holds three times the GPU memory, runs models of up to about 80 GiB of weights on one card and needs about 6.3 kW. The one with RTX PRO 4500 Server Edition cards holds models of up to about 20 GiB per card, suits many small models and virtual desktops, and runs on about 2.2 kW from a single 16 A feed.

SE = Server Edition. NVIDIA product pages for both cards and product briefs SP-12355-001_v02 (300 W minimum) and PB-12790-001_02 (100 W minimum); NVIDIA MIG user guide (13 October 2026) and GPUs supported by vGPU (2 October 2026), as cited in our card guides; usable memory and wall power are our estimates, with two AMD EPYC 9555 and two EPYC 9355 processors respectively.

Neither card has NVLink, so in both servers a model larger than one card is split over PCIe. Our earlier comparison of eight RTX PRO 4500 or four RTX PRO 6000 Server Edition holds the summed bandwidth roughly equal. This page keeps the slot count equal instead, which is the choice a buyer faces once a chassis with eight double-width slots is set.

Memory per card decides which models run

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card. The RTX PRO 6000 Server Edition reports 95.6 GiB, which leaves 83.0 GiB per card and 664.3 GiB for eight. NVIDIA publishes no driver-visible figure for the 32 GB card, so we scale the 96 GB figure to about 31.9 GiB, which leaves 25.7 GiB per card and 205.6 GiB for eight; check it with nvidia-smi -q on your own card. The table counts concurrent conversations at full context length.

Our estimates by the users-per-card rule, not measurements: FP8 KV cache at 8K (0.625, 1.0 and 1.25 GiB per conversation), 16-bit cache at 32K; weights 15.2, 32.0, 67.7, 60.8 and 219.8 GiB from the Qwen, NVIDIA and OpenAI checkpoints, as in our sizing guides.

Two rows show the arithmetic. On one RTX PRO 6000 Server Edition, Qwen3-32B in FP8 leaves 83.0 − 32.0 = 51 GiB, which is 51 conversations of 8K per card and 408 for eight copies. On the RTX PRO 4500 Server Edition the same model needs at least two cards per copy, and a TP=4 copy has 102.8 GiB, which leaves 70.8 GiB, or 70 conversations, 140 for two copies. Qwen3-235B-A22B-2507 in FP8 has 219.8 GiB of weights, more than the 205.6 GiB the smaller server holds in total.

Splitting costs time. Every split copy exchanges partial results between its cards at every layer over PCIe, and by default vLLM's custom all-reduce is disabled on more than two GPUs without NVLink: its source logs "Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs". vLLM 0.30.0 (release notes, 22 September 2026) added an opt-in FlashInfer PCIe IPC all-reduce for systems without NVLink (#53576). On eight RTX PRO 6000 Server Edition cards a 70B model runs as single-card copies or as TP=2 pairs, which keep that path; on eight RTX PRO 4500 Server Edition cards it needs TP=4 or TP=8, which by default fall back to NCCL. Our guide to RTX PRO 6000 without NVLink compares the layouts.

Users per model for 500 to 2,000 staff

The conversations above are requests in flight at the same moment. With the example values of our guide to a private ChatGPT server by company size, 500 staff produce about 20 requests in flight at the busiest moment, 1,000 staff about 40 and 2,000 staff about 80.

By those values, eight RTX PRO 4500 Server Edition cards running copies of a 14B model in FP8 cover 2,000 staff, and two TP=4 copies of Llama 3.3 70B in FP8 cover about 1,000. Eight RTX PRO 6000 Server Edition cards serve the 70B model for well over 2,000 staff, with headroom for longer contexts or a second model, and they are the only one of the two that holds models of 200 GiB and more. If the plan is a 70B-class assistant for the whole company plus room to grow, the larger cards are the safer choice. If it is a set of small models, embeddings and rerankers for one or two departments, the smaller server carries it at a third of the power.

We build both servers to order and size the split per model. Send us your models, context lengths and the number of staff, and our team replies within one business day with the card, the layout and the conversations each model gets.

Published measurements per card

StorageReview tested one RTX PRO 4500 Server Edition in a Dell PowerEdge R770 with vLLM 0.27.1 and published the results on 14 October 2026, next to an RTX PRO 6000 that ran on the same container and driver in a Dell Pro Max Tower T2. That card was a Workstation Edition, which StorageReview says "stands in for the server card" because both are rated at 600 W and profiled the same in its testing.

StorageReview, RTX PRO 4500 Blackwell Server Edition review, 14 October 2026, read on 24 October 2026: output tokens per second at concurrency 32; the RTX PRO 4500 flattened at about 180 tok/s from concurrency 8 at 8K; per rated watt at 512/512. WS = RTX PRO 6000 Workstation Edition, standing in for the Server Edition. Host platforms differ.

At short context the larger card produces about twice the output; at 8K it produces about three times as much, because the smaller card runs out of memory for the KV cache. StorageReview wrote that "The flattening is KV cache running out, and 32GB sets that limit". Per rated watt the RTX PRO 4500 does more work at short context. By our arithmetic, eight independent copies would reach about 8,856 output tokens per second with eight RTX PRO 4500 Server Edition and about 18,000 with eight RTX PRO 6000 Server Edition at 600 W at short context, upper bounds from one card if the host does not become the limit. Our RTX PRO 4500 benchmarks collect the other published figures.

For eight cards, only the larger configuration has verified results in MLPerf® Inference v6.1 of 16 September 2026; we found no verified entry with eight RTX PRO 4500 Server Edition.

MLPerf® Inference: Datacenter v6.1, closed division, available category, offline scenario, Llama 2 70B (llama2-70b-99.9) and gpt-oss-120b, entries 6.1-0070 and 6.1-0050, eight RTX PRO 6000 Blackwell Server Edition each, retrieved from mlcommons.org on 24 October 2026. Results verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Power: 4.8 kW against 1.32 kW of cards

Eight RTX PRO 6000 Server Edition at 600 W draw 4.8 kW; eight RTX PRO 4500 Server Edition at 165 W draw 1.32 kW. With the processors of our eight-card guides and 15 per cent for memory, drives, network cards, fans and conversion losses, the server comes to about 6.3 kW and about 2.2 kW. The smaller server with the same two EPYC 9555 would reach about 2.3 kW.

Our estimate: board power from NVIDIA's product briefs; two AMD EPYC 9555 at 360 W for the server with RTX PRO 6000 SE and two EPYC 9355 at 280 W for the server with RTX PRO 4500 SE, plus 15 per cent; feeds planned at 80 per cent: 2.94 kW at 16 A, 5.89 kW at 32 A, 8.87 kW three-phase 16 A. Dell's datasheet gives about 8 kW per eight-card RTX PRO 6000 Server Edition node as an indicative figure; plan the feed on the maker's figure for the exact model.

The feed is often the deciding item. A rack position with a 230 V 16 A feed per side carries the server with RTX PRO 4500 Server Edition cards, at 2.94 kW planned, and not the one with RTX PRO 6000 Server Edition cards, even at their 300 W minimum. At 600 W the larger server needs a three-phase feed per side, and with A and B feeds each side carries the whole server. Heat follows: about 6.3 kW needs roughly 1,000 CFM of air at an 11 °C rise and 2.2 kW about 350 CFM, by the 160 CFM per kW of our guide to GPU rack power and cooling. The capped variants are compared in our page on eight GPUs at 300 W.

MIG, vGPU desktops and support horizon

NVIDIA's MIG guide lists up to four 1g.24gb instances on the RTX PRO 6000 and two 1g.16gb halves on the RTX PRO 4500, so the servers offer up to 32 isolated GPUs of 24 GB or 16 of 16 GB. A 24 GB slice holds Qwen3-14B in FP8 with cache for about four conversations of 8K; a 16 GB half holds an 8B model in FP8, an embedding model or a reranker, and Qwen3-14B in FP8 does not fit it. NVIDIA's MIG guide states that NCCL is not supported with MIG, so split models run on whole cards.

For virtual desktops, NVIDIA's vPC sizing guide of 19 August 2026 gives 16 desktops per RTX PRO 4500 Server Edition on 2 GB profiles, 128 for eight cards at what it calls maximum supported density. For the RTX PRO 6000 Server Edition the same guide gives 32 per card on a 2 GB profile, and 48 with MIG-backed time-sliced vGPU, so 256 or 384 for eight cards by our arithmetic. NVIDIA calls the RTX PRO 4500 Server Edition "the primary recommended GPU for NVIDIA vPC", and a pilot with your own desktops sets the final profile. Our guide to VDI GPU sizing scales hosts to 1,000 desktops.

The support horizon differs. As of October 2026 the RTX PRO 6000 Server Edition is in NVIDIA's long-term R580 driver branch, supported to June 2028, with vGPU 19 to July 2028. The RTX PRO 4500 Server Edition is not in the R580 list and needs vGPU 20 on the R595 branch, whose end of life NVIDIA gives as March 2027, so a host bought now moves to a later branch within its first year. Profiles per card are in our guides to RTX PRO 6000 Server Edition vGPU profiles and MIG and vGPU seats per card.

Host, chassis and licences

NVIDIA's configuration guide for NVIDIA-Certified Systems, updated on 30 September 2026, asks for system memory of at least twice the total GPU memory and at least six physical cores per GPU. Both servers therefore need 48 cores, and the memory differs: 1,536 GB for eight RTX PRO 6000 Server Edition and 512 GB for eight RTX PRO 4500 Server Edition. NVIDIA's RTX PRO AI Factory reference architecture adds two cores per MIG instance, which counts more on the larger cards with 32 instances.

Several chassis take either card. Puget Systems configures its 4U eight-GPU server with one to eight RTX PRO 4500 or RTX PRO 6000 Server Edition, with 3+1 supplies of 3,200 W and four 200 to 240 V connections, and Supermicro lists eight of either card in its 5U SYS-522GA-NRT. For the RTX PRO 6000 Server Edition, Dell's XE7745, HPE's DL380a Gen12, Cisco's UCS C845A M8 and Lenovo's 3U SR675 V3 take eight. For the RTX PRO 4500 Server Edition, Supermicro also lists eight in its 4U SYS-422GL-NR, and its largest 2U listing is six. A chassis built for eight double-width cards has room the single-slot card does not use, and the maker's list decides the count for each model, as our list of Server Edition compatible servers shows for the larger card.

Licences do not separate them. NVIDIA's licensing guide states that NVIDIA AI Enterprise "is licensed on a per-GPU basis", so where it is used, eight cards need eight licences on either server, and neither card includes a subscription. NVIDIA sells vWS, vPC and vApps per concurrent user or per GPU. Serving with vLLM on bare metal needs no NVIDIA licence on either.

Which server for which workload

Our reading of the sections above; sources as in the tables.

For most companies that buy an eight-card server for LLM inference, we recommend the RTX PRO 6000 Server Edition, because the largest model decides and 96 GB per card avoids splitting 70B-class models over four cards. The server with RTX PRO 4500 Server Edition cards is the better fit where the work is many small models or desktops and the feed is 16 A. You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build both to order, assembled and tested before delivery, with manufacturer warranty on every component on one EU contract and invoice; the steps are on our pages to buy an 8× RTX PRO 6000 Server Edition server and an 8× RTX PRO 4500 Server Edition server. Describe your rack position and its feed, and we size both versions side by side.

FAQ