What actually fits in the 128 GB of a DGX Spark, and what you can fine-tune on one

There is no VRAM figure on this machine

Every GPU datasheet an engineer has read for fifteen years opens with a VRAM number. DGX Spark does not have one, and that is not an omission. NVIDIA’s porting guide states that the system “uses a unified memory architecture (UMA), where the GPU shares system memory (DRAM) with the CPU and other compute engines”. There are 128 GB of LPDDR5x on a 256-bit interface, and the 20-core Arm processor, DGX OS, the desktop session and the GPU all draw from that one pool. Nothing is reserved for graphics, and nothing is guaranteed to the model.

Two consequences are documented by NVIDIA itself, and both are met on the first afternoon. The first: nvidia-smi cannot report GPU memory usage on this platform. The known-issues page says the tool “will display ‘Memory-Usage: Not Supported’ even though per-process GPU memory is listed”, because an integrated GPU has no dedicated framebuffer to count. The second: cudaMemGetInfo is pessimistic. The porting guide warns that “the memory size reported by cudaMemGetInfo may be smaller than the actual allocatable memory, since the CPU may be able to release additional DRAM pages”, and tells developers to consider memory reclaimable from the operating system rather than rely on that call alone.

Put plainly: the two instruments that normally answer “will this model fit” both give the wrong answer here, one by refusing and one by answering low. NVIDIA documents the failure in the other direction too. Its Unsloth playbook lists “memory pressure within capacity” as a known condition caused by the UMA buffer cache not being released, with a one-line workaround: sync, then write 3 into /proc/sys/vm/drop_caches. An out-of-memory error on this box is not always what it looks like.

How much of the 128 GB you actually get

NVIDIA publishes no usable-capacity figure. There is no “X GB available to the GPU” line on the product page, in the user guide, in the porting guide or in the release notes, so anyone quoting one is guessing. What does exist is what NVIDIA sets as the default in its own serving playbooks.

The 0.8 and 0.9 values are NVIDIA’s own defaults. The 102 to 115 GB range is this article’s arithmetic from them, and is not a published specification.

Use 102 to 115 GB as the budget for weights and KV cache together, and treat the remaining 13 to 26 GB as the tax paid to DGX OS, the desktop session, the page cache and whatever those twenty Arm cores are doing. Quote the number with its reasoning attached, because the day NVIDIA changes a playbook default it moves. Raising the fraction towards 0.95 is documented and sensible on a unit that serves models and nothing else; on a unit somebody also uses as a workstation, it is how you meet the out-of-memory condition above.

Bytes per parameter, and what 200 billion really means

Weight-only sizing is arithmetic anyone can repeat: parameters multiplied by bytes per parameter. BF16 and FP16 are two bytes, FP8 is one, and the 4-bit formats are about half a byte before block scales are counted. Run the working set above through that, taking 110 GB as a midpoint.

Weights only: no KV cache, no activations, no framework overhead. This is the article’s arithmetic from NVIDIA’s memory figure and standard bytes-per-parameter values, not an NVIDIA table.

Now reconcile that with the headline. NVIDIA’s launch announcement and its product page both say the machine runs inference on models with up to 200 billion parameters. Two hundred billion parameters at half a byte each is 100 GB of weights, which against a 102 to 115 GB working set leaves between 2 and 15 GB for the KV cache, activations and overhead. The claim is true, and it is an FP4 claim. At BF16 the same box tops out near 55 billion parameters, which is a different machine in the reader’s head.

NVIDIA’s own material confirms where the single-unit line sits, without anyone having to take our word for it. Its validated-model table for TensorRT-LLM lists Llama-3.3-70B-Instruct, Qwen3-32B and Nemotron-3-Super-120B as NVFP4 checkpoints for the supported hardware platform, and exactly one entry in that table is flagged multi-node: Qwen3-235B-A22B, a 235-billion-parameter mixture of experts. NVIDIA’s own shopping list says a 235B model needs more than one Spark. The same page carries the instruction “confirm that the selected model fits available memory before downloading it”.

The concrete full-precision example NVIDIA gives is GPT-OSS-120B, or FLUX 2 at 90 GB. Ninety of the 128 for one model is a useful calibration: comfortably inside the box, and very little left for anything else. The lever that makes any of this work is NVFP4, which NVIDIA’s quantisation playbook puts at about 3.5 times less memory than FP16 and about 1.8 times less than FP8, with typically under one per cent accuracy loss. There is no second lever.

One honesty note on the headline. NVIDIA has also published a lower single-unit figure, “100-billion-parameter models on DGX Spark”, in a January 2026 blog post. Both numbers are its own; the 200 billion figure is the one in the launch press release and on the product page.

The KV cache is what turns a model that fits into a model that does not

Weights are the half of the budget you can read off a model card. The other half is the KV cache, which grows linearly with context length and with the number of concurrent sequences, and which is why a deployment that loads happily at a 2,048-token prompt falls over at 32,000 tokens with eight users. Our VRAM guide has the full formula, layer count, key and value heads, head dimension, bytes per element, sequence length and batch size; what matters here is its conclusion, that the cache is neither a rounding error nor a fixed cost.

Three cases against a 110 GB working set make the point. A 70-billion-parameter model in NVFP4 is roughly 35 GB of weights and leaves about 75 GB, which is a lot of cache and a genuinely comfortable single-unit workload. A 120-billion-parameter model, which NVIDIA itself measures at 90 GB, leaves about 20 GB: enough for one conversation at a working context, not enough to serve a team. A 200-billion-parameter model leaves single digits of gigabytes, which is a demonstration rather than a service.

This is also the right place to read NVIDIA’s benchmark methodology rather than its benchmark numbers. Every published inference figure for DGX Spark is quoted at “ISL|OSL = 2048|128, BS=1”: a 2,048-token input, 128 generated tokens, one request at a time. That is a small context and a single user, and nothing in the published set says what happens at long context under concurrency, which is precisely where the cache rather than the weights decides.

What one unit can actually fine-tune

NVIDIA’s product page says “fine-tune AI models up to 70 billion parameters” and does not say by what method. Its own PyTorch fine-tuning playbook does, and the published script names settle it: Llama3_3B_full_finetuning.py, Llama3_8B_LoRA_finetuning.py and Llama3_70B_qLoRA_finetuning.py. The ladder is bound to the method, not to the parameter count.

Peak tokens per second from NVIDIA’s performance blog, calculated by NVIDIA as (batch_size × steps × sequence_length) divided by total training time, at sequence length 2,048, batch size 8 (4 for the 8B LoRA run), one epoch and 64 steps.

Why the ladder has that shape is arithmetic NVIDIA does not publish for this machine, so here is ours, openly, and it should be labelled as ours wherever it is quoted. A full fine-tune in BF16 with the Adam optimiser carries, per parameter, two bytes of weights, two bytes of gradients, eight bytes of fp32 optimiser moments and four bytes of fp32 master weights: about 16 bytes per parameter before a single activation is stored. A 3-billion-parameter model is therefore about 48 GB, which fits with room for activations. A 7-billion-parameter model is about 112 GB, which does not fit a 102 to 115 GB working set once activations are added. That one multiplication explains why NVIDIA’s full-fine-tune example stops at 3B, why LoRA reaches 8B, and why 70B arrives only through QLoRA, where the base sits frozen in 4 bits at roughly 35 GB.

Read the throughput column as wall-clock time rather than as a score. At 759.79 tokens per second, a million training tokens is about 22 minutes on the 70B QLoRA run, and a hundred million tokens is a multi-day job on one unit. That constrains how many experiments a team runs in a week, and it matches how NVIDIA positions the machine: prototype and fine-tune locally, then evaluate the work for migration to DGX cloud or other accelerated infrastructure.

The reproduction stack is published too, which matters if you intend to repeat the numbers rather than admire them: NVIDIA’s PyTorch container 25.12-py3 with transformers, peft, datasets, trl 0.26.2 and bitsandbytes 0.49.1, plus one caveat printed in the guide, that bitsandbytes must be forced to its CUDA 13.0 binary because CUDA 13.1 is not yet supported. NVIDIA treats out-of-memory as an expected first-run outcome and prescribes reducing batch size or sequence length and preferring LoRA or QLoRA for a first attempt.

Capacity is not throughput

Everything above is a capacity argument, and capacity is the half marketing sells. The other half sits on the same spec line: 273 GB/s. That is what predicts how fast tokens come out.

Single-stream decoding reads every weight once per generated token, so the ceiling is memory bandwidth divided by the size of the resident weights, and the arithmetic is one division. A dense 70-billion-parameter model in FP8 is about 70 GB of weights; 273 divided by 70 is under four tokens per second for one user, before overhead. The same model in NVFP4 at about 35 GB doubles that ceiling. Nothing in the compute specification changes either figure, and NVIDIA publishes no measured single-unit number for a dense 70B model, so the division is the honest estimate to work with.

NVIDIA technical blog, measured at ISL|OSL 2048|128 with batch size 1. The two mixture-of-experts entries beat what their total parameter count would suggest, because only a fraction of the weights is read per token.

Against the parts a buyer would actually put beside it, all from NVIDIA’s own pages, the trade is explicit rather than subtle.

NVIDIA product pages, September 2026. The ratios are this article’s arithmetic: 1,792 divided by 273, and 4,800 divided by 273.

Spark holds more memory than an RTX PRO 6000 and moves it 6.6 times more slowly. That is a capacity purchase set against a speed purchase, and which is right depends on whether the problem is “the model does not fit” or “the answers arrive too slowly”. Our benchmark article covers the measured side.

One correction belongs here, because it is repeated everywhere. “Up to 1 PFLOP FP4” is not a dense figure. NVIDIA’s own footnote on the product page reads “theoretical FP4 TOPS using the sparsity feature”, and the user guide spells it out as up to 1 petaFLOP at FP4 precision with sparsity. NVIDIA publishes no dense FP4 number for GB10, so anyone printing “1 petaflop” without the word sparse is reprinting a marketing figure as a specification. It would not predict token generation in any case: 273 GB/s does.

When one unit is not enough, and what a second one buys

The other widely repeated error is that DGX Spark maxes out at two units. It does not. NVIDIA’s clustering documentation states that it “supports up to three DGX Spark systems connected directly through cables, and up to four systems when using a switch”, and NVIDIA ships separate playbooks for two units, for three in a ring topology and for four through a switch, plus an NCCL guide explicitly covering two, three or four Sparks. Two is the common case, not the ceiling.

The link is a direct 200 GbE QSFP connection between the ConnectX-7 ports, with approved cables named by part number and a RoCE device alongside each Ethernet interface. Quote the measured figure rather than the nameplate: NVIDIA’s own RDMA write benchmark reports 92.57 and 97.28 Gb/s across the two logical links, 189.85 Gb/s in total, about 95 per cent of the 200 Gb/s rating. One consequence of the unified-memory design travels with the cluster: GPUDirect RDMA is not supported, because pinned device allocations cannot be coherently accessed by the CPU complex or by PCI Express devices, so zero-copy paths from the network card into GPU memory are not available. Our two-node article covers the wiring.

What the second unit buys is another 128 GB, not more speed. NVIDIA’s measured two-unit figure is Qwen3 235B in NVFP4 at 11.73 tokens per second, which is the shape of the whole trade: the model becomes loadable, it does not become fast.

On the multi-unit size claim, NVIDIA contradicts itself, and a buyer should hear it from us rather than discover it later. The live DGX Spark product page says ConnectX networking enables the connection of up to four systems to work with models of up to 700 billion parameters. NVIDIA’s own Markdown mirror of that same product page says two systems and up to 405 billion parameters. Both were published and reachable in September 2026. The four-unit wording is the one consistent with the clustering documentation and with the four-unit NCCL playbook, so treat 700 billion on four units as the current claim and 405 billion on two as also published. The arithmetic ties both to 4-bit weights and to nothing else: 700 billion at half a byte is 350 GB against 512 GB in four units, and 405 billion is about 203 GB against 256 GB in two.

Eurokommerz supplies DGX Spark units across the EU, and we would rather size the memory before the order than explain it after. Send the model, the precision and the context length, and we will run the same arithmetic on your numbers, including the case where one unit is not the right machine.

FAQ