NVIDIA DGX Spark: real benchmarks and limits

What is inside

The heart is the GB10 Grace Blackwell chip: twenty Arm cores (10× Cortex-X925 + 10× Cortex-A725) plus a Blackwell GPU with 6,144 CUDA cores and fifth-generation Tensor Cores. CPU and GPU share coherent memory over NVLink-C2C, so nothing is ever copied between system RAM and VRAM.

First surprise for new owners: the system shows about 119–122 GiB instead of 128 GB. Part of that is units (Linux counts in gibibytes), the rest is memory reserved by firmware and the GPU driver; independent tests measure about 121 GiB usable per unit. NVIDIA published a dedicated knowledge-base note because the question kept coming back.

The limit the slide decks skip

An LLM works in two phases: it reads your prompt (prefill), then emits the answer token by token (decode). Prefill is compute-bound; decode is memory-bound, because producing each token means reading every weight of the model once.

Spark has compute to spare and memory that moves at 273 GB/s, seventeen times slower than an H200 NVL, and that gap explains nearly every owner complaint. The spread on gpt-oss 120B looks like this:

A 3.8× spread on one and the same model: different engines, builds, quantisation formats and test dates. Reading a prompt at 1,725 tok/s is genuinely fast; generating at 55 tok/s is far faster than anyone reads. Fine for chat, thin for batch document pipelines.

The number to know before buying

Most buyers want 128 GB for 70B-class models that no ordinary GPU can hold. LMSYS measured Llama 3.1 70B in FP8 with one user: 2.7 tokens per second. A three-hundred-word answer takes about two and a half minutes to type itself out. The model fits and runs; the question is whether you are willing to watch it think. Acceptable overnight, painful interactively.

The arithmetic agrees: a dense 70B in FP8 weighs ~70 GB, every token requires reading all of it, and 70 GB at 273 GB/s caps the ceiling below four tokens per second. The measured 2.7 sits exactly inside that maths.

The part that changes the picture

All of the above assumes one user. Pile up requests and the weights are read once per batch; the machine becomes a different animal. Dendro Logic’s vLLM measurements:

tokens per second, aggregate

A 120× gain on Nemotron Super 49B is not a typo. Spark was built to serve a workload, not to chat with one person. A fifteen-developer team behind an in-house coding assistant: yes. One enthusiast with the biggest model they could download: the money went the wrong way.

Power, and the petaflop footnote

NVIDIA’s forum hosts a telling thread: “Only getting half the advertised performance and capping at 100W”. The unit was fine. 240 W is the external brick’s rating; the GPU section draws 120 W at most, with the rest budgeted for the CPU, ConnectX-7 and peripherals. The advertised petaflop also needs its footnote: FP4 with sparsity, delivered in short bursts. Idle draw is unspecified; the network chip turned out to be the main idle consumer, partly fixed by firmware.

Two nodes

Two Sparks link directly over a QSFP cable and share a model across 2 × 128 GB, sharded tensor- or pipeline-parallel over the link; NVIDIA’s launch figure was support for models up to 405B parameters this way (the product page now quotes four units for up to 700B). What pairing does not buy is single-user speed: in community llama.cpp tests prompt reading gains 6–25%, generation roughly nothing. You pair to fit a bigger model, not to get answers sooner; batch serving is the exception.

Who it suits, and who it does not

A good fit: CUDA development needing a production-identical environment without a monthly cloud bill; data that must not leave the premises; a team to serve; QLoRA fine-tuning up to 70B with patience to spare. A poor fit: snappy interactive chat on the largest model available; Windows-only toolchains (DGX OS is aarch64 Linux); gaming or classic rendering.

What we supply

Eurokommerz delivers NVIDIA DGX Spark (Founders Edition, 4 TB) across the EU: available to order, with NVIDIA AI Enterprise subscriptions; multi-unit clusters are set up with NVIDIA Sync. When a prototype outgrows the desk, we configure the next step: GPU servers or Nutanix NX nodes.

FAQ