DGX Spark, one RTX PRO 6000 or a server with H200 NVL: which to start a first AI project on

The four questions that decide it

Most first AI projects arrive at the same shortlist: an NVIDIA DGX Spark on a desk, one RTX PRO 6000 Blackwell in a workstation or a server, or a server with one or two H200 NVL cards. They are not three sizes of the same machine, and the choice falls out of four questions that have nothing to do with the product pages.

How big is the model, and at what precision? A parameter count settles nothing on its own. The same 70 billion parameters occupy 141.2 GB at BF16, 70.6 GB at FP8 and 35.3 GB at NVFP4, so the hardware answer changes by a factor of four on a decision nobody has made yet.

How many people use it at once? One developer iterating on prompts is a batch of one, which is also how NVIDIA benchmarks a Spark. Twenty people inside an assistant is twenty concurrent sequences, each holding its own key-value cache, and on a single card the cache runs out before the weights do.

Where will the box physically live? A DGX Spark is 150 mm square, 50.5 mm high and 1.2 kg, wants 5 to 30 °C and draws a measured maximum of 233.2 W from an ordinary socket. An H200 NVL is a passively cooled dual-slot card needing a named fan kit, a named air duct, a specific riser and the right 16-pin cable part number for that slot, in a server on the vendor’s supported list. Different projects, different people.

What happens after the pilot? Each starting point grows differently, and two of the three have a hard limit written into the hardware. That belongs in the decision now.

The three options side by side

Everything below comes from NVIDIA’s product pages, datasheets, user guides and product briefs, plus the OEM guides where an OEM overrides NVIDIA. Read the bandwidth row twice; it decides how the machine feels.

NVIDIA DGX Spark product page and user guide; RTX PRO 6000 datasheets and the Blackwell PRO architecture whitepaper; H200 NVL product brief PB-12128-001 and datasheet; Dell, HPE and Lenovo product guides. September 2026.

One row is misquoted more than any other. The RTX PRO 6000 figure that circulates, 1,792 GB/s, is the Workstation Edition and Max-Q number. The Server Edition, the only one passively cooled for a chassis and the only one on NVIDIA’s vGPU list, runs at 1,597 GB/s, about 11 per cent lower, so sizing a rack deployment on 1,792 GB/s overstates its token generation by roughly 12 per cent. Our edition comparison covers the rest.

The entitlement differs too. H200 NVL is sold with a five-year NVIDIA AI Enterprise subscription, licensed per GPU and activated against the GPU serial number, so record the serials before the cards are racked. DGX Spark carries no such entitlement: NVIDIA states it exists only if purchased, requested as an evaluation or granted by certificate, and the free path is a 90-day licence with community-driven support only.

Memory is the first gate, bandwidth is the second

Weights are parameters multiplied by bytes per parameter: two at BF16 or FP16, one at FP8, half a byte at NVFP4 or MXFP4. That arithmetic, and nothing else, decides whether a model loads at all.

Weights only, from nominal counts of 8.03B, 32.8B and 70.6B. Cache, activations and framework overhead come on top; the VRAM guide has the formula.

A 70B model at BF16 needs 141.2 GB: it fits an H200 NVL with 141 GB and nothing to spare, and fits neither the 96 GB RTX PRO 6000 nor the Spark’s 128 GB. NVIDIA’s NIM support matrix agrees in its own words: on the RTX PRO 6000 Server Edition a 70B model runs at FP8 and NVFP4 on one card, and BF16 needs four or eight cards.

The Spark’s 128 GB is misread in two ways. It is unified system memory shared with a 20-core Arm CPU, DGX OS and the desktop session, not a framebuffer: nvidia-smi prints “Memory-Usage: Not Supported” on this platform, and NVIDIA warns that cudaMemGetInfo under-reports what can be allocated. NVIDIA publishes no usable-capacity figure; its own playbooks default to 0.8 of memory in vLLM and 0.80 to 0.9 in TensorRT-LLM, which puts a working set at roughly 102 to 115 GB. That is our arithmetic from NVIDIA’s defaults, not a specification.

The second misreading is the headline. NVIDIA says one Spark can “test, validate, and inference with AI models up to 200 billion parameters”. True, and it is a 4-bit claim: 200 billion × 0.5 bytes is 100 GB of weights, leaving ten to twenty-eight gigabytes for cache, activations and the operating system. At FP8 the same box reaches roughly 110 billion parameters and at BF16 roughly 55 billion. Reading “200B on a desk” as a BF16 statement mis-sizes the project by four times.

Fitting is only the first gate. Token generation re-reads every weight from memory for every token it emits, so the per-user rate at low batch is bandwidth divided by bytes of weights, not FLOPS.

Our arithmetic: bandwidth divided by 70.6 GB. An upper bound real systems do not reach, not a measurement. At NVFP4 the same model is 35.3 GB and every ceiling roughly doubles.

Three point nine tokens per second is the sharpest number here. Silent reading of non-fiction averages 238 words per minute, which at roughly 0.75 words per token is about 5.3 tokens per second. A dense 70B model at FP8 on a Spark cannot reach reading speed even in theory, and the theory is generous: NVIDIA’s own Spark measurements put Llama 3.1 8B at NVFP4 at 38.65 tokens per second against a ceiling of 68, and Qwen3 14B at 22.71 against 39, both near 57 per cent of the ceiling, which puts the dense 70B nearer two than four. Every Spark benchmark NVIDIA publishes is 4-bit, at batch size one.

This is why the FLOPS headlines mislead, and both of the ones an engineer will meet are sparsity figures by the vendor’s own footnotes. NVIDIA’s Spark page prints “Up to 1 PFLOP FP4” under the footnote “Theoretical FP4 TOPS using the sparsity feature”, and publishes no dense figure for GB10 anywhere. The RTX PRO 6000 Server Edition page prints 4 PFLOPS FP4, 2 PFLOPS FP8 and 1 PFLOP FP16 with no basis stated, but the architecture whitepaper gives the Workstation Edition as dense and sparse pairs, 2,015.2 against 4,030.4 TFLOPS at FP4, footnoted “Effective TOPS / TFLOPS using the Sparsity Feature”, and those dense values scale to the published Server Edition figures. Halve them. Every H200 NVL datasheet tensor figure carries “With sparsity” too. Standard LLM serving does not use 2:4 structured sparsity, and none of these numbers predicts tokens per second at batch one. Bandwidth does.

Concurrency changes the answer

One developer and twenty employees are different machines, and the reason is the key-value cache. Every active conversation holds keys and values for every token it has seen, sized as 2 × layers × key-value heads × head dimension × bytes per element × sequence length. Grouped-query attention is what makes single-card serving possible: Llama 3.1 70B has 80 layers, 64 query heads and 8 key-value heads, so it costs 320 KiB per token at FP16 instead of the 2,560 KiB it would cost without GQA.

Run that against a 96 GB card, at vLLM’s default of 90 per cent of the board less about 3 GiB for activations and CUDA graphs.

Maximum concurrent sequences each holding a full-length context, with an FP16 cache / an FP8 cache. A floor rather than a ceiling, since PagedAttention allocates blocks on demand, but it is what decides whether a request is admitted or queued.

Read the 70B rows together. At FP8 the model leaves 17.6 GiB of cache, seven full 8K sessions or one at 32K; at NVFP4 it leaves 50.5 GiB and twenty. The quantisation decision buys more concurrency on one card than a second card would. Long context collapses concurrency faster than model size does, which is why the first question to ask is not how many users but how long the conversations are.

An honest gap belongs here. NVIDIA publishes no absolute tokens-per-second LLM figure for any RTX PRO 6000 edition, and there is no MLCommons-verified MLPerf submission for the card; its own inference performance page lists this GPU with vision and image-generation models only. Its published LLM claims are relative, against an L40S baseline with no precision or batch size stated. The measurement that does exist is on your own card, where vLLM prints GPU KV cache size and Maximum concurrency at startup.

For the shape of the answer, NVIDIA’s reference architecture sizes a single H100 NVL at 137 concurrent users on Llama 3.1 8B at FP8, delivering 2,977 tokens per second at a time to first token near 987 ms: different silicon with HBM3, so not a substitute figure, but it shows the order of magnitude. The industry ships interactive services against about 50 tokens per second per user and a time to first token at or below 250 ms, eight to ten times true reading speed. The concurrency article works it through.

If the users arrive as virtual machines rather than API clients, the edition question returns. vGPU runs on the RTX PRO 6000 Server Edition from vGPU 19.0; the Workstation and Max-Q editions are not on NVIDIA’s supported list at all. MIG splits any of the three into four instances of 24 GB, two of 48 GB or one of 96 GB, and for compute profiles the ceiling is 12 time-sliced virtual machines of 8 GB. Our MIG and vGPU guide has the tables.

Where each one goes wrong

Each of the three is right for some projects and wrong for others. These are the failure modes we actually meet, and what to do about each.

DGX Spark fails on speed, not on size. It will load a model that it then serves too slowly to put in front of a person, which is a worse outcome than refusing to load it: 273 GB/s is 6.6 times below an RTX PRO 6000 Workstation card and 17.6 times below an H200 NVL. NVIDIA positions it as a machine to prototype, fine-tune and validate on before migrating the work to a data centre. So buy it for the people who write the code, and decide the serving hardware once the prototype has settled the model, the precision and the context length. Note the envelope too: 5 to 30 °C is narrow for an office, and GPUDirect RDMA is not supported because of the unified memory model. Where the box is genuinely strong is fine-tuning, and NVIDIA’s documented ladder on one unit is a full fine-tune at 3B, LoRA at 8B and QLoRA at 70B, measured at 13,519.54, 6,969.59 and 759.79 peak tokens per second. The benchmark article has the table.

The RTX PRO 6000 fails on the edition and on context length. A Workstation Edition ordered for a rack is a 600 W card with double flow-through cooling that no server will accept, and no vGPU; a Server Edition ordered for a workstation is a passive card with no fan. Then the memory: 96 GB is comfortable for a 70B model at NVFP4 and tight at FP8, and it is long conversations rather than parameter counts that empty it. One card is also one failure domain, and neither MIG nor vGPU changes that, since both partition a single physical GPU. Fix the edition against where the card will live, quantise before buying a second card, and if the service matters put the second card in a second host.

The H200 NVL fails on the chassis, not on the card. Support is configuration-scoped rather than server-scoped, and the gates are all retrofit blockers. Dell lists the card only on 17th-generation systems, two maximum in an R770 or R7725, and not in the 15G or 16G tables, so a server you already own may not be on the list. Lenovo supports it in the SR650a V4 and states it is “not supported on SR650 V4”, two machine types of what reads like one 2U family. Beyond the model number there are named parts: Dell’s HPR Platinum fans, its riser kits and a different power-cable part number for each slot; Lenovo’s 600 W fan module, its front double-width air duct, slots 21 and 23 only, and an ambient cap of 30 °C. The drive backplane can veto the GPU outright, because it sets the air impedance in front of the card. And the failure that wastes a maintenance window: the 16-pin auxiliary cable must be strapped for the 451 to 600 W class, or the card does not boot at all rather than running throttled. Validate machine type, backplane, riser, fan kit, air duct and cable part number before ordering: neither NVIDIA nor the OEMs publish an airflow figure to check a chassis against, so the supported-configuration list is the thermal specification. The retrofit checklist is the long version.

What the upgrade path looks like from each

The three starting points scale in three different ways, and only one of them scales by adding another box identical to the one you already own.

Take the Spark path first, because one claim about it is out of date. DGX Spark does not stop at two units: NVIDIA’s clustering documentation supports up to three systems connected directly through cables and up to four through a switch. The two published model-size claims disagree on NVIDIA’s own properties, 405 billion across two units and 700 billion across four, and both hold only at 4-bit: 405 billion × 0.5 bytes is about 203 GB against two units’ 256 GB, and 700 billion × 0.5 is about 350 GB against four units’ 512 GB. NVIDIA’s validated-model table flags exactly one entry as multi-node, Qwen3-235B-A22B at NVFP4, measured at 11.73 tokens per second across two Sparks. Growth here buys capacity, not speed. The clustering article covers the cabling.

The RTX PRO 6000 path carries the constraint that surprises most people: there is no NVLink on any edition, workstation or server. Lenovo’s product guide states it flatly, NVIDIA’s specification tables omit it, and the reference architecture describes GPU placement purely in PCIe terms. The gap against a bridged card is roughly seven times, 128 GB/s against 900 GB/s. So the ladder is: quantise before scaling out, because FP8 or NVFP4 weights on one card beat BF16 split across two cards with no fast link; then add independent replicas, one model instance per card behind a load balancer, which creates no inter-GPU traffic and doubles availability; and only when the model genuinely will not fit, split it, preferring pipeline parallelism to tensor parallelism, which is vLLM’s own guidance for nodes without NVLink.

The H200 NVL path is the one designed for growth, with a caveat about geometry. Each card carries exactly one bridge, the biggest change from H100 NVL and its three, so an existing bridge set does not carry over. The cards must be physically adjacent and under the same CPU, which in a server whose risers split GPUs two and two across sockets makes the 4-way bridge uninstallable as intended. Plan the slot map before the order. Our head-to-head comparison takes those two cards further.

If the answer is genuinely unclear, there is a defensible sequence. Start on a Spark while the model, the precision and the context length are still moving. Move to one RTX PRO 6000 when a team uses the thing daily and bandwidth becomes the complaint. Move to H200 NVL when latency, long context or a model that must span GPUs makes the bridge and the 4.8 TB/s the point of the purchase.

What we supply

Eurokommerz supplies all three, in any quantity, on one European contract: DGX Spark units, RTX PRO 6000 Blackwell cards in all three editions with the workstation or server around them, and H200 NVL cards with the bridges, cables, fan kits and chassis the OEM documents require. The fastest way to the right answer is to describe three things: the model you intend to run and at what precision, how many people will use it at the same time, and where the machine will physically live. With those three we can return the memory arithmetic, the bandwidth ceiling and the parts list.

FAQ