NVIDIA RTX PRO 6000 Blackwell benchmarks: MLPerf®, LLMs, rendering and limits

Three editions, one chip

The RTX PRO 6000 Blackwell is three cards on one GB202 chip: 24,064 CUDA cores and 96 GB of GDDR7 with ECC on a 512-bit bus, of which nvidia-smi reports 97,887 MiB, or 95.6 GiB. The Workstation Edition runs at 600 W with a double flow-through cooler and 1,792 GB/s; the Max-Q keeps that memory and bandwidth at 300 W, with what Puget Systems describes as a blower-style cooler; the passive Server Edition draws up to 600 W, configurable, with its memory at 1,597 GB/s. Our edition comparison has the full table.

We are a supplier, not a lab: nothing here is our own measurement. Every measured figure is someone else’s published result, cited with source, date and, where stated, model, precision and software.

Single-user speed limit: bandwidth divided by bytes per token

Prefill is compute-bound; decode reads every active weight once per token, so for one user the limit is memory bandwidth divided by the bytes read per token. By our arithmetic, Llama 3.3 70B reads 70.6 GB per token in FP8 and 40.6 GB in NVFP4. That caps one stream at 25.4 and 44.1 tokens per second on the Workstation and Max-Q, and at 22.6 and 39.3 on the Server Edition. gpt-oss-120b reads about 5 GB per token in OpenAI’s MXFP4 checkpoint, which caps it at roughly 360 and 320. These are upper bounds for plain decoding, not measurements.

With many concurrent requests, each read of the weights serves them all, so total throughput climbs until compute or KV cache space runs out; that is where the FP4 Tensor Cores count.

NVIDIA’s own figures: throughput at full load

NVIDIA’s TensorRT-LLM performance overview has Server Edition tables for three models, each from an NVIDIA FP4 checkpoint. The metric is output tokens per second per GPU, first token included, measured with trtllm-bench and the PyTorch backend while a local client sends requests “at a high rate / no delay between messages”: a card at maximum load, with many options at their defaults, in figures NVIDIA says “should not be considered as the peak performance”.

NVIDIA TensorRT-LLM performance overview (documentation of TensorRT-LLM 1.4.0rc1), tables for the “RTX 6000 Pro Blackwell Server Edition”, last updated 19 October 2026 with unchanged values; synthetic prompts, input and output lengths in tokens. The four-card Qwen3-235B-A22B entry uses NVIDIA’s DEP2 and PP2 settings.

Prompt length moves the result most: the 70B model falls from 1,724 to 296 tokens per second per GPU as prompts grow from 1,000 to 8,192 tokens, and NVIDIA lists 32,768-token prompts for it only on two cards, in pipeline parallel. We found no per-user figure from NVIDIA. We supply the card in all three editions and build AI servers to order with the Server Edition. Describe your model and your users and our team will help.

MLPerf® Inference v6.0: verified results for two to ten cards

MLPerf® Inference v6.0 holds closed-division results from HPE, Dell, Cisco and other submitters for servers with two to ten Server Edition cards; the ten-card one is HPE’s ProLiant Compute DL380a Gen12 (entry 6.0-0045), although HPE’s QuickSpecs for that server, as read on 16 October 2026, limit the RTX PRO 6000 to eight cards. Offline, all queries are sent at once; in the server scenario they arrive at random intervals, and a run counts only if the time to the first token and the time per output token stay within limits, which the interactive scenario tightens. MLPerf® Inference v6.1, published on 16 September 2026, holds Server Edition results for four- and eight-card servers; the eight-card results follow in the next section.

MLPerf® Inference: Datacenter v6.0 (published 1 April 2026), closed division, available category; benchmarks Llama 2 70B (llama2-70b-99) and gpt-oss-120b, offline, server and interactive scenarios; retrieved from mlcommons.org on 24 September 2026; results verified by MLCommons Association, as is every MLPerf® result in this section. Values from MLCommons’ results table and raw logs; the RTX PRO 6000 systems ran FP4 weights on TensorRT 10.14 with CUDA 13.0 or 13.1. Per-card figures (our division of the system result by its number of GPUs) and tokens per second per stream (1,000 divided by the median time per output token in the log) are our arithmetic, not MLPerf® metrics. Limits for the first token and per output token: server 2 s and 200 ms for Llama 2 70B, 3 s and 80 ms for gpt-oss-120b; interactive 450 ms and 40 ms. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

By our division of each system result, not an MLPerf® metric, the three HPE systems in the table ran Llama 2 70B offline at 3,340 to 3,739 tokens per second per card. HPE’s eight-card server kept 96 per cent of its offline total in the server scenario, with a median of 175 ms per output token: about 5.7 tokens per second for the median stream, by our arithmetic. The interactive scenario is the one closest to chat. The eight-card servers of results 6.0-0004 and 6.0-0005 held the median stream at about 27 tokens per second (37 ms per token) and delivered 6,238.1 and 6,262.6 tokens per second, which is 22 and 23 per cent of their own offline results (27,812.9 and 27,730.1). On gpt-oss-120b the server median was 55 ms per token, about 18 tokens per second per stream. NVIDIA’s 1,724 per GPU and our per-card division of the MLPerf® results are different measurements: models, request lengths, software and tuning all differ.

HPE’s four-card DL385 Gen11 (entry 6.0-0054) capped its cards at 450 W, while the two-card one (entry 6.0-0052) ran them at 600 W. By the same per-card division, not an MLPerf® metric, the capped cards delivered 3 per cent less offline and 8 per cent less in the server scenario, where a result also depends on the load each run was set to. The two systems differ in card count and driver build (580.105.09 against 580.126.09), so this is an indication, not a controlled test.

Dell’s PowerEdge XE7740 with eight H200 NVL (entry 6.0-0021) delivered 18 per cent more on Llama 2 70B offline than the same chassis with eight RTX PRO 6000 (entry 6.0-0022): 32,004 against 27,034.8 tokens per second. The H200 NVL system ran FP8 weights on TensorRT 10.13 with NVLink named as the link between the cards; the RTX PRO 6000 system ran FP4 weights on TensorRT 10.14 over PCIe. In the interactive scenario the H200 NVL system reached 16,343.8 tokens per second, 2.6 times result 6.0-0004 from another maker’s server, at about 28 tokens per second for the median stream against 27. By our arithmetic, that is about two and a half times as many requests in flight within the same limits. Our H200 NVL comparison explains why.

MLPerf® Inference v6.1: eight Server Edition cards

MLPerf® Inference v6.1 holds results from five eight-card Server Edition systems of different makers. The table gives the system results as MLCommons publishes them; Llama 2 70B runs in this round at the 99.9 per cent accuracy target (llama2-70b-99.9), against 99 per cent in the v6.0 table above, so we do not compare the two rounds line by line.

MLPerf® Inference: Datacenter v6.1 (published 16 September 2026), closed division; benchmarks Llama 2 70B (llama2-70b-99.9), gpt-oss-120b, Llama 3.1 8B and Whisper; offline and server scenarios, tokens per second; entries 6.1-0070 (MiTAC G4520G6), 6.1-0013 (Cisco UCS C845A M8), 6.1-0036 (Dell PowerEdge XE7740), 6.1-0050 (HPE ProLiant Compute DL380a Gen12) and 6.1-0082 (Nebius RTX PRO 6000 n1), each with eight RTX PRO 6000 Blackwell Server Edition; retrieved from mlcommons.org on 24 October 2026; results verified by MLCommons Association. “Not cited”: a value we do not quote.

By our division of each system result by eight, not an MLPerf® metric, the four Llama 2 70B systems ran at 3,661 to 3,754 tokens per second per card offline and lie within 2.6 per cent of each other, so for this model the server maker matters less than the card count. Each kept 97 to 98 per cent of its offline throughput in the server scenario, against 96 per cent for HPE’s eight-card server in v6.0. The three gpt-oss-120b results lie within 3.2 per cent. HPE’s entry also has Llama 3.1 8B in the interactive scenario, at 39,986.3 tokens per second or 78 per cent of its offline result. NVIDIA’s DGX B300 with eight NVLink-connected GPUs reached 109,277 tokens per second on Llama 2 70B offline, about 3.6 times the best eight-card Server Edition result (MLPerf® Inference: Datacenter v6.1, closed division, llama2-70b-99.9, offline, entry 6.1-0071, retrieved from mlcommons.org on 24 October 2026, result verified by MLCommons Association). Our overview of MLPerf Inference v6.1 for PCIe GPU servers has every scenario, and our guide to eight Server Edition cards what such a server runs.

Third-party tests: single-user, multi-card and fine-tuning results

StorageReview’s RTX PRO 6000 Workstation review (7 October 2025) and HPE ProLiant DL380a Gen12 review (6 November 2025); Exxact’s LoRA fine-tuning benchmark (4 June 2026). StorageReview states neither the quantisation of its LM Studio models, nor the precision of its Llama 2 70B Chat run, nor its vLLM version; Exxact does not state its software versions.

StorageReview’s single-card figures come from one prompt that asks for a 500-word paper, on a Threadripper workstation under Windows; with the file format unstated, they cannot be set against the ceilings above. Llama 3.3 70B, at 67.7 GiB in NVIDIA's FP8 checkpoint, runs on one RTX PRO 6000 with an FP8 cache for about twelve 8K conversations by our users-per-card rule, and it does not fit the 32 GB of an RTX 5090, as our RTX 5090 comparison explains. Exxact, a system integrator, ran its rank-16 LoRA fine-tune on the query and value projections with batch 2 per GPU and 512-token sequences; two cards in distributed data parallel reached 9,541 tokens per second, which Exxact puts at 96.1 per cent scaling.

We build AI servers to order with RTX PRO 6000 Server Edition cards for inference and fine-tuning. Tell us the model and whether you will serve it or fine-tune it, and we reply with a configuration and a quote within one business day.

Rendering, image and video

Puget Systems compared the Workstation Edition with the RTX 6000 Ada in June 2025, updated in July, on a Ryzen 9 9950X with driver 576.52: 50 per cent faster in Blender, with a nearly identical gain in V-Ray’s RTX mode, 55 per cent in its CUDA mode, 34 per cent in Unreal Engine, and in DaVinci Resolve 43 per cent in LongGOP tests and 78 per cent in GPU effects. StorageReview measured the Workstation Edition at 7,870 samples per minute in Blender 4.4’s Monster scene, ahead of the RTX 5090’s 7,421.5, and 12,128 vpaths in V-Ray, where the 5090 led with 14,764; UL Procyon’s Stable Diffusion XL test in FP16 took 5.364 seconds per image.

For the Server Edition, NVIDIA’s inference performance page, as of September 2026, lists vision, image and video generation models only, among them Flux at 0.20 images per second in FP4 at batch 1, Stable Diffusion XL at 0.72 at batch 4 and Stable Video Diffusion at 2.82 videos per minute, with TensorRT and without resolution or step count. Phoronix’s Linux review of the family (21 May 2026, driver 595.58.03) shows its results as charts only.

Max-Q, power, heat and noise

Puget’s July 2025 test on a Threadripper PRO 7965WX put the 300 W Max-Q 5 to 13 per cent behind the Workstation Edition in Blender, Octane, Redshift and V-Ray, and 14 per cent behind in DaVinci Resolve’s GPU effects and in Unreal Engine; we found no published LLM test of both editions on one bench. Our four-card workstation guide covers towers with four Max-Q.

StorageReview’s whole workstation with the card drew 918.5 W on average and 1,036.3 W at peak during Stable Diffusion XL, and 152.3 W at idle. The Server Edition has no fan of its own and is cooled by the server’s fans: NVIDIA’s product brief gives minimum ducted airflow through its heat sink from 43 CFM at a 25 °C inlet to 125 CFM at 45 °C, as a qualification reference.

Limits: NVLink, MIG, vGPU, power and workloads for other cards

No NVLink. NVIDIA’s product brief for the Server Edition lists NVLink as not supported, and no edition has it: two cards are two 96 GB GPUs on PCIe 5.0 x16, not one 192 GB pool. Tensor parallelism works, as StorageReview’s four-card runs show, but every exchange crosses PCIe, and a note in vLLM’s documentation suggests pipeline parallelism instead on nodes without NVLink, for higher throughput.

MIG needs preparation on the workstation editions. All three split into up to four instances of 24 GB, two of 48 GB or one of 96 GB and, on bare metal or in passthrough, need Linux and driver 575.51.03 or later; the Workstation and Max-Q also need a minimum vBIOS and the display mode switched to compute, which turns the display outputs off, while the Server Edition ships in display-off mode. Our MIG runbook has the steps.

vGPU runs on the Server Edition only. NVIDIA’s list of GPUs supported by vGPU, updated 2 October 2026, has the Server Edition from vGPU 19.0 and its liquid-cooled version from vGPU 20.0, and neither workstation edition; the Server Edition’s brief says vGPU requires the display-off mode.

Power and software. The Server Edition’s 600 W depends on the cable: NVIDIA’s brief defines a 450 W mode for a 16-pin cable strapped for it, and asks for an R575 driver and CUDA 12.9 or later.

Workloads for other cards. The RTX PRO 6000 is the wrong choice for interactive serving of 70B-class models to many users at once, where the H200 NVL, with 4,800 GB/s of HBM3e against 1,597 GB/s of GDDR7, delivered 2.6 times the throughput above (MLPerf® Inference: Datacenter v6.0, closed division, Llama 2 70B (llama2-70b-99), interactive scenario, entries 6.0-0021 and 6.0-0004, FP8 against FP4 weights, retrieved from mlcommons.org on 24 September 2026, results verified by MLCommons Association); models that need more than 96 GB and fast tensor parallelism; double-precision work, since FP64 runs at 1/64 of the FP32 rate by NVIDIA’s whitepaper; virtual desktops on the workstation editions; and servers qualified only for 350 W cards, where the L40S may still fit better.

We also supply the H200 NVL and the L40S, so the card can follow the workload. Write to us with the workload and the server the card goes into, and we will say which card fits.

What we supply

We supply the RTX PRO 6000 Blackwell in all three editions, with manufacturer warranty on one EU contract and invoice, as cards or in AI servers built to order, sized for the workload, assembled and tested before delivery; two of our three reference configurations carry two and four Server Edition cards. If you also need the software, our engineering partner Vixen.UNO deploys private LLMs on vLLM, Ollama or NVIDIA AI Enterprise and RAG that respects each user’s access rights, under the same contract, starting with a pilot on one process with clear metrics (Private AI/ML).

FAQ