RTX PRO 5000 72 GB vs DGX Spark: faster tokens or larger models for local LLMs

RTX PRO 5000 72 GB vs DGX Spark: the short answer

The RTX PRO 5000 72 GB generates tokens several times faster than a DGX Spark on any model that fits in its 72 GB, and the DGX Spark holds models that do not fit on the card. Both differences come from memory, as the card has 72 GB of GDDR7 at 1,344 GB/s and the Spark 128 GB of LPDDR5x at 273 GB/s, shared by its CPU and GPU. Token generation is limited by memory bandwidth, and 1,344 GB/s is almost five times 273 GB/s.

Choose the card for models up to a 4-bit 70B, for interactive speed per user and for teams that share one model. Choose the Spark when the model is larger than 72 GB, such as gpt-oss-120b with room for context, a 70B model in FP8 or a 4-bit model of up to about 200 billion parameters, and when capacity matters more than speed per user. This comparison uses the 128 GB DGX Spark Founders Edition; our 48 GB or 72 GB comparison covers the smaller card.

RTX PRO 5000 72 GB and DGX Spark specifications

NVIDIA RTX PRO 5000 Blackwell datasheet (July 2026, the version linked from the product page) and product page; NVIDIA DGX Spark product page and hardware documentation; all read on 16 October 2026.

The RTX PRO 5000 is a card that goes into a workstation or server with its own processor, system memory and drives. The Spark is a complete computer on the GB10 Grace Blackwell chip, where 20 Arm cores and a Blackwell GPU draw on one pool of memory, so it has no separate VRAM figure.

Tokens per second: 1,344 GB/s against 273 GB/s

To generate one token for one user, the GPU reads every active weight once, so single-stream speed cannot exceed the memory bandwidth divided by the bytes read per token. Llama 3.3 70B in NVFP4 reads about 40.6 GB per token. On the card, 1,344 GB/s divided by 40.6 GB sets a ceiling of about 33 tokens per second; on the Spark, 273 GB/s divided by 40.6 GB gives about 6.7. Qwen3-32B in NVFP4 reads about 19.1 GB per token, which gives about 70 on the card and about 14 on the Spark. These ceilings come from the specifications, and engines stay below them, more so as the cache grows with the context; our RTX PRO 5000 benchmark article sets out the method and the bytes per token.

On the Spark, the measurements stay inside that ceiling. LMSYS measured Llama 3.1 70B in FP8 at 2.7 tokens per second for one user with SGLang, published on 13 October 2025; 273 GB/s divided by about 70 GB of weights allows just under 4. For the RTX PRO 5000 we found no published token rate that states the engine version, the precision, the concurrency and the test system together. The nearest comparison is the same LMSYS test, where gpt-oss-20b in Ollama generated 49.7 tokens per second on the Spark and 215 on an RTX PRO 6000 Blackwell, whose edition LMSYS does not name; the RTX PRO 5000 has three quarters of the 1,792 GB/s of the RTX PRO 6000 Workstation and Max-Q cards.

Mixture-of-experts models run much faster on the Spark than dense models of the same size, because each token reads only a small part of the weights. gpt-oss-120b has 117 billion parameters with 5.1 billion active, and NVIDIA measured it at 55.4 tokens per second on one Spark with llama.cpp, at a 2,048-token prompt and batch size 1 (NVIDIA technical blog, 24 October 2025). Our DGX Spark benchmarks collect the other published runs. On the 72 GB card the same model only just loads, as the next section shows.

Which LLMs fit in 72 GB and in 128 GB

On the card, our sizing rule takes 90 per cent of the memory the driver reports, less about 3 GiB for activations and the runtime. On the Spark, NVIDIA’s serving playbooks set memory fractions of 0.8 to 0.9, which our article on what fits on a DGX Spark turns into a working set of roughly 102 to 115 GB for weights and cache.

Weights and fit on the card from our 48 GB or 72 GB comparison (FP8 cache), Qwen3-32B from our RTX PRO 5000 benchmark article; fit on the Spark from our 128 GB article; measured rates: NVIDIA technical blog of 24 October 2025 (2,048-token prompt, batch size 1) and LMSYS of 13 October 2025; ceilings are our arithmetic, 1,344 or 273 GB/s divided by about 19.1 GB (Qwen3-32B) or 40.6 GB (Llama 3.3 70B) read per token.

Up to a 4-bit 70B model, both hold the model, and the card’s generation ceiling is about five times higher, as 1,344 GB/s is 4.9 times 273 GB/s. gpt-oss-120b marks the boundary: its 60.8 GiB of weights load on the 72 GB card with under 1 GiB left for context, while the Spark keeps about 45 GB available for cache. Beyond that, one Spark holds what one card cannot, a 70B model in FP8 or a 4-bit model of up to about 200 billion parameters. NVIDIA’s product page gives up to 400 billion parameters for two linked Sparks and up to 700 billion for four. Two 72 GB cards in one workstation hold 144 GB, but without NVLink a model split across them exchanges data over PCIe.

We supply both the DGX Spark Founders Edition and the RTX PRO 5000 in 48 and 72 GB. Tell us the models, their precision and the context length, and we check which of the two holds them.

Prompt processing and long contexts

Prompt processing, or prefill, is bound by compute rather than bandwidth, so the gap is smaller than in generation. On paper the card has 2,064 FP4 TOPS with sparsity against the Spark’s 1,000, about twice as much. In the llama.cpp maintainers’ results for gpt-oss-120b (February 2026 build), the Spark processes prompts at about 2,400 tokens per second with 4,096-token prompts, whether one request is waiting or 32. Thirty-two such prompts take 54.4 seconds before the last request sees its first token. For the dense Llama 3.1 70B in FP8, LMSYS measured 803 tokens per second of prefill.

We found no prefill figure for the RTX PRO 5000 that states its test conditions. In LMSYS’s gpt-oss-20b test with Ollama, the Spark processed 2,053 prompt tokens per second and the RTX PRO 6000 Blackwell 10,108. The RTX PRO 5000 has about half the FP4 peak of the 600 W RTX PRO 6000 Workstation Edition, 2,064 against 4,000 AI TOPS, so its lead over the Spark in prefill should be smaller than that of the 6000.

Several users at once

For the Spark, concurrency has been measured. The llama.cpp maintainers ran gpt-oss-120b with 4,096-token prompts and 32 generated tokens per request at 55.2 tokens per second for one request. With 8 requests the aggregate was 139.07, and with 16 it was 196.26, or about 17.4 and 12.3 per request. Our article on one DGX Spark for a team reads this as about 16 simultaneous requests before each answer drops below a comfortable reading pace of 10 tokens per second.

For the card, no comparable measurement is published. A dense model’s weights are read once per step for the whole batch on the card as well, so aggregate throughput rises with concurrency until compute or cache space runs out. With a 4-bit 70B model, the 72 GB card has cache for about seventeen 8K conversations. With a 32B model in FP8 it keeps about 30 GiB for cache, roughly thirty 8K conversations at 1 GiB each. The Spark holds the 4-bit 70B model with far more cache, and with a dense model added requests cost little speed per stream: LMSYS measured Llama 3.1 70B in FP8 at 2.7 tokens per second for one request and 20.2 in total for eight, about 2.5 each. With every stream below about 6.7 tokens per second, the Spark suits dense 70B models for batch jobs more than for a chat assistant.

Operating system, software and power

The Spark runs NVIDIA DGX OS, which NVIDIA bases on Ubuntu, on Arm cores. CUDA code runs on it once built for Arm, but every binary-only dependency needs an Arm Linux build, so check the toolchain before ordering. The card goes into an x86 workstation with the operating system and management tools your IT already uses. NVIDIA’s datasheet lists CUDA 12.8 for the card and MIG with up to two isolated 36 GB instances on the 72 GB version, so two services can each get half the card. MIG needs Linux and the display mode switched to compute, which turns off the card’s display outputs. NVIDIA’s list of GPUs supported by vGPU, updated on 2 October 2026, names the 72 GB Workstation Edition from release 20.2, so the card can also serve virtual machines.

NVIDIA’s hardware documentation gives the GB10 chip a TDP of 140 W, with 100 W available for the other components, and the Spark ships with a 240 W external supply. The card has a total board power of 300 W, and the host’s processor, memory, drives and fans come on top. A workstation with two cards needs a power supply sized for 600 W of cards plus the host, while a Spark plugs into a desk socket.

Worked example: a team of 10 to 30 developers and analysts

For a team of 10 to 30 developers and analysts who want a private assistant for code, data and documents, either option is sized by the peak of requests in flight in a pilot’s gateway logs, not by the headcount.

With two to four Sparks, each serves gpt-oss-120b as its own copy behind a load balancer. By the llama.cpp results, each keeps about 16 requests with 4,096-token prompts at 10 tokens per second or more, so two Sparks carry about 32 such requests and four about 64. Each Spark can also run a 70B model in FP8 for batch work or, as NVIDIA’s technical blog of 24 October 2025 reports, a QLoRA fine-tune of Llama 3.3 70B.

With two workstations, each with one or two RTX PRO 5000 72 GB cards, every card serves its own copy of a 4-bit 70B model or a 32B model. Four cards hold cache for about seventeen 8K conversations each, about 68 in all, at a single-stream ceiling of about 33 tokens per second; two cards hold half that. To serve gpt-oss-120b with room for context, a workstation needs both of its cards, with the model split over PCIe.

Choose the Sparks if the team needs gpt-oss-120b or models above 72 GB and sends chat-length prompts. Choose the workstations if the team settles on models up to a 4-bit 70B, sends long prompts from coding agents or retrieval, or wants faster answers per user.

We build workstations with one or two RTX PRO 5000 cards to order and supply the Spark on the same contract. Send us the team size, the models and the prompt lengths through the form below.

What we supply

We supply the NVIDIA DGX Spark Founders Edition (128 GB, 4 TB) and the RTX PRO 5000 Blackwell with 48 GB and 72 GB across the EU, with manufacturer warranty, on one EU contract and invoice. The card comes on its own or in a workstation built to order, assembled and burn-in tested; for rack servers we check the rack, power and airflow before we quote. Where a model needs more memory per card, we also supply the 96 GB RTX PRO 6000 in all three editions and the H200 NVL. See the DGX Spark page, the professional GPU catalogue and our AI servers built to order.

FAQ