8× NVIDIA RTX PRO 4500 Blackwell Server Edition in one server: models, users and VDI

What eight single-slot cards add up to

A server with eight NVIDIA RTX PRO 4500 Blackwell Server Edition cards runs small and mid-size models, about 205.6 GiB of weights and cache in all, at about 2.2 kW for the whole server. That is enough for eight copies of a 14B model serving 128 conversations at once, two copies of Llama 3.3 70B in FP8, 128 virtual desktops at NVIDIA's maximum vPC density or 16 isolated MIG instances.

The card is NVIDIA's passive, single-slot data-centre version of the RTX PRO 4500. Its 32 GB sit on a 256-bit bus at 800 GB/s, it draws 165 W at most, and it connects over PCI Express 5.0 x16 with one 16-pin power connector. The eight cards do not pool their memory: the card has no NVLink, so a model larger than one card is split over PCIe. By our users-per-card rule, each card holds a model of up to about 20 GiB of weights with room for its cache.

NVIDIA RTX PRO 4500 Blackwell Server Edition product page, read on 24 October 2026; the 100 W minimum from NVIDIA's product brief PB-12790-001_02, as cited in our card guide. Usable memory is our estimate.

The card-level detail, from the board outline to the release history, is in our guide to the RTX PRO 4500 Blackwell Server Edition. This page works out what eight of them do together.

How much of 256 GB holds models and cache

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card for CUDA graphs, activations and the runtime. 90 per cent was vLLM's default gpu_memory_utilization before release 0.20.0, which raised it to 0.92. NVIDIA publishes no driver-visible figure for the 32 GB card, so we scale the 97,887 MiB that a 96 GB RTX PRO 6000 reports: about 31.9 GiB. That gives 0.9 × 31.9 − 3 = 25.7 GiB per card, and 205.6 GiB for eight. Check the figure with nvidia-smi -q on your own card. NVIDIA's vPC sizing guide states that Blackwell's GDDR7 has ECC built in that is "always enabled without reducing usable VRAM". The derivation of the rule is in our guide to how many users one RTX PRO 6000 can serve.

What is left after the weights is KV cache. With an FP8 cache, a conversation of 8,192 tokens takes 0.625 GiB on Qwen3-14B, 1.0 GiB on Qwen3-32B and 1.25 GiB on Llama 3.3 70B, by the layers, KV heads and head size in each model's configuration.

Models and layouts on eight 32 GB cards

Eight cards give four ways to place a model: one copy per card, TP=2 pairs, TP=4 copies or one copy split over all eight. More cards per copy means fewer copies of the weights and more room for conversations, and more traffic over PCIe on every layer.

Our estimates by the users-per-card rule, not measurements: concurrent conversations of 8,192 tokens at full length with an FP8 KV cache; gpt-oss-120b at 32,768 tokens with a 16-bit cache of 1.125 GiB. Weights are the Qwen, NVIDIA and OpenAI checkpoints as in our sizing guides.

Two rows show the arithmetic. A TP=4 copy of Llama 3.3 70B has 4 × 25.7 = 102.8 GiB; less 67.7 GiB of weights that leaves 35.1 GiB, and 35.1 ÷ 1.25 gives 28 conversations per copy, 56 for two. One copy of Qwen3-14B on one card leaves 25.7 − 15.2 = 10.5 GiB, which is 16 conversations.

Anything from Qwen3-32B in FP8 upwards needs at least two cards per copy. gpt-oss-120b does not fit a TP=2 pair at all, because two cards hold 51.4 GiB and its weights alone take 60.8 GiB. Models with several hundred GB of weights do not fit this server, by the sizes in our LLM hardware requirements by model.

Users per model for 500 to 2,000 staff

The conversations above are requests in flight at the same moment, not named users. With the example values of our guide to a private ChatGPT server by company size, 500 staff produce about 20 requests in flight at the busiest moment, 1,000 staff about 40 and 2,000 staff about 80.

By those values, eight copies of a 14B-class model in FP8 with 128 conversations cover 2,000 staff with headroom, and four copies on half the server cover 1,000. Two TP=4 copies of Llama 3.3 70B in FP8 hold 56, which covers 1,000 staff; with a 16-bit cache the same layout holds 28. For a company that wants one 70B model for many users, four RTX PRO 6000 Server Edition cards hold one copy per card and need no traffic between cards, as our comparison of eight RTX PRO 4500 or four RTX PRO 6000 Server Edition shows.

The eight-card server suits a company that runs several models at once. An assistant model can take four cards, a coding model two, and an embedding model and a reranker for RAG one card each, each engine with its own memory, restart and update cycle. A failed card then takes one eighth of the GPU memory out of service. Our guide to serving several models on one GPU server covers the engine settings for such a split. We build this server to order and size the split per model; send us your models and the peak requests in flight, and our team replies with a layout and the conversations each model gets.

Splitting a model over PCIe: TP=2, TP=4 or TP=8

Every split copy exchanges partial results between its cards at every layer, and here that runs over PCIe. By default, vLLM's custom all-reduce is disabled on more than two GPUs without NVLink: its source logs "Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs" and falls back to NCCL. A TP=2 pair keeps the faster path by default and a TP=4 or TP=8 copy does not. Release 0.30.0 of 22 September 2026 added an opt-in FlashInfer PCIe IPC all-reduce for systems without NVLink (pull request #53576), and we found no published gain for it on this card. vLLM's parallelism guide of 15 October 2026 advises GPUs without NVLink to "leverage pipeline parallelism instead of tensor parallelism for higher throughput and lower communication overhead".

No publisher has measured tensor-parallel scaling on several RTX PRO 4500 Server Edition cards in a way we can use: StorageReview tested one card, and Exxact's two- and four-card tables contradict their own text, as our RTX PRO 4500 benchmarks round-up shows. Plan TP=8 for capacity, and expect a single conversation on it to run slower than the bandwidth sum suggests. In a two-socket server, nvidia-smi topo -m shows which cards share a processor, and a pair under one socket avoids the link between the sockets. Our article on one model on several GPUs over PCIe or NVLink explains the options.

Published measurements for one card

StorageReview measured one Server Edition card with vLLM 0.27.1 in a Dell PowerEdge R770 and published the results on 14 October 2026. On Ministral 3 14B Instruct in FP8 with 512 input and 512 output tokens, the card produced 1,107 output tokens per second at 32 concurrent requests, against 995 for an L40S and 364 for an L4. With 8,192 input tokens the curve flattened at about 180 tokens per second from 8 concurrent requests, and StorageReview wrote that "The flattening is KV cache running out, and 32GB sets that limit", which our rule predicts for a 14B model on this card.

By our arithmetic, eight independent copies would reach about 8 × 1,107 = 8,856 output tokens per second at short context and 8 × 180 = 1,440 at 8K, if the host does not become the limit. That is an upper bound from one card, not a measured eight-card result. The round-up of all published figures, and what they imply, is in our RTX PRO 4500 benchmarks.

VDI with vGPU 20: desktops per server

NVIDIA's vPC sizing guide, updated on 19 August 2026, calls the card "the primary recommended GPU for NVIDIA vPC". It gives 16 desktops per card on 2 GB profiles and 10 on 3 GB profiles, and notes that 1 GB profiles are not supported on Blackwell. Its density table counts 128 desktops for eight cards on 2 GB profiles, a table that "reflects maximum supported density rather than a recommended deployment point". By the same figures, 3 GB profiles give 80 desktops per server. A pilot with your own desktops sets the final profile.

NVIDIA vPC sizing guide, updated 19 August 2026, read on 24 October 2026; eight-card totals are our arithmetic.

The card is on NVIDIA's list of GPUs supported by vGPU software from release 20.0, and it supports MIG-backed vGPU, so a half can back virtual machines. NVIDIA's support matrix for vSphere lists it for VCF 9.1, for VCF 9.0 from 9.0.1 and for ESXi 8.0 from Update 3g; Ubuntu and RHEL KVM are also listed. NVIDIA's vGPU documentation shows vGPU 20 on the R595 driver branch with an end-of-life date of March 2027, while the long-term vGPU 19 branch does not cover this card. A host bought now therefore moves to a later vGPU branch within its first year in service. NVIDIA licenses vPC, vWS and vApps per concurrent user, as our guide to MIG and vGPU seats per card explains, and VDI GPU sizing scales hosts to 1,000 desktops.

16 MIG halves on one server

NVIDIA's MIG guide lists two profile sizes for the 32 GB RTX PRO 4500: 1g.16gb, up to two per card with half the memory and half the SMs each, and 2g.32gb for the whole card. Eight cards therefore offer up to 16 isolated GPUs of 16 GB. MIG is set per card, so four cards can run a TP=4 copy while the other four serve small models in halves; NVIDIA's MIG guide states that NCCL is not supported with MIG, so a split copy runs on whole cards.

By our rule, a half that reports about 15.9 GiB holds 11.4 GiB of weights and cache. NVIDIA's FP8 checkpoint of Qwen3-8B, 8.8 GiB, fits with cache for about four conversations of 8K; Qwen3-14B in FP8 does not fit. A half suits embedding models, rerankers, classifiers and student or developer environments. With two plain halves, each gets one video decoder and one encoder, so 16 halves expose 16 decoders of the 24 on the cards. Our article on MIG and time-slicing in Kubernetes covers how a cluster hands the halves out.

Power at the wall and on a 16 A feed

At 165 W per card, eight cards draw 1,320 W. With two AMD EPYC 9355 processors at a default TDP of 280 W each, cards and processors are rated at 1,880 W. Memory, drives, network cards, fans and conversion losses come on top; we add 15 per cent for them, an assumption to be confirmed by a measurement on the delivered system. Processors with a higher TDP raise the total.

Our estimate: board power from NVIDIA's product page and brief, 280 W per processor from AMD's EPYC 9355 page, plus 15 per cent for the rest of the server.

A 230 V single-phase feed at 16 A is rated at 3.68 kW, and we plan 80 per cent of it, 2.94 kW. The eight-card server stays below that at full board power by our estimate, so one 16 A feed per side carries it, with redundant power supplies on A and B feeds each carrying the whole server. The number of power cords follows the chassis maker's supply layout: Puget Systems' 4U eight-GPU server, for example, uses 3+1 supplies of 3,200 W and asks for four 200-240 V connections on separate circuits, the same layout that serves its RTX PRO 6000 and H200 NVL options. About 2.2 kW of heat needs roughly 350 CFM of air at an 11 °C rise, by the 160 CFM per kW in our guide to GPU rack power and cooling.

Servers whose makers list eight cards

The card runs only in a chassis whose maker lists it with the matching risers, cables and fans. These are the eight-card listings we found.

Supermicro "RTX PRO Blackwell Server Edition" systems page and Puget Systems configurator, both read on 24 October 2026.

Supermicro's largest 2U listing for the card is six. NVIDIA's vPC sizing table assumes eight boards per 2U server without naming one, and Lenovo's guide for this card records that its 2U ThinkSystem SR650a V4 "now supports the GPU" without a card count we could read, so we do not count any 2U server as an eight-card option. AWS offers eight of these cards per instance in its g7.48xlarge, so the eight-card configuration is in production use. For any chassis, the maker's configurator decides. We check the chassis, feed and airflow before we quote; tell us the rack position and feed you have, and we name the chassis that fits.

When another configuration fits better

Four RTX PRO 6000 Server Edition cards suit one or two 70B-class models for many users, with one copy per card. Eight L40S cards give 48 GB per card at 350 W each and no MIG, a trade our sibling page on eight L40S or eight RTX PRO 4500 Server Edition weighs. Four of these cards in a 2U server suit a first VDI host or a department, as our page on four RTX PRO 4500 Server Edition in 2U explains. Training and fine-tuning of large models belong on H200 NVL with NVLink bridges. Where one server must not be a single point of failure, two four-card servers are the safer layout, as one 8-GPU server or two 4-GPU servers compares.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with eight RTX PRO 4500 Server Edition cards in chassis whose makers list them, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component, and install the operating system, drivers, CUDA and a container runtime on request. The configuration and ordering steps are on our page to buy an 8× RTX PRO 4500 Server Edition server. Models, RAG and the platform on top are our Private AI/ML service.

FAQ