One DGX Spark for a team: how many people it serves, and how to share it

What NVIDIA says, and what it does not

NVIDIA positions the DGX Spark for “AI developer, researcher, and data scientist workloads” and publishes no figure for how many people can share one. The nearest statement is about software: “With support from frameworks that handle concurrency well (such as NVIDIA TensorRT LLM, vLLM, and SGLang), multiagent workloads run smoothly on NVIDIA DGX Spark.” The warranty is more direct about intended use: a “small-scale server for consumer and enterprise end user and multi-users purposes”. Sharing one unit is within what NVIDIA intends. The question is how far it stretches.

The unit to size by is simultaneous requests, not people. Ten developers with a coding assistant rarely send ten requests in the same second, and a document pipeline sends all of its requests at once. Our article on users per RTX PRO 6000 explains how to read your own logs for the number that matters; here we apply the published measurements to the Spark.

Speed per request, from 1 to 32 at once

The most complete public data comes from the llama.cpp maintainers, who benchmark the DGX Spark with several requests in flight. The table gives the generation speed each request sees.

Tokens per second per request. llama.cpp maintainers’ DGX Spark results (llama-batched-bench, build of February 2026, 4,096-token prompt and 32 generated tokens per request); per-request speed is the published aggregate divided by the number of requests.

The aggregate keeps rising, from 55 tokens per second for one request on gpt-oss-120b to 262 for 32, while each request slows down. LMSYS measured the same shape at launch in October 2025, with SGLang and 2,048-token prompts and answers: gpt-oss-120b fell from 50.5 tokens per second alone to 14.6 per request at 8, 9.2 at 16, 6.3 at 32 and 4.6 at 64, for a total of about 290 tokens per second. NVIDIA’s own concurrency test, published in March 2026, ran Qwen3 Coder Next in FP8 on vLLM with 32K-token prompts and 1K-token answers: one, two and four tasks at once took 35, 54 and 91 seconds, with the first token arriving after a median 9, 12 and 15 seconds.

Our 70B sizing example works with 10 tokens per second as a comfortable reading pace. Measured against that, gpt-oss-120b on one Spark stays above it up to about 16 simultaneous requests with 4,096-token prompts and falls below it at 32, while gpt-oss-20b stays above it at 32.

Why long prompts form the queue

Reading a prompt is compute work, and on the Spark it proceeds at a fairly fixed rate: about 2,400 tokens per second for gpt-oss-120b in the llama.cpp results, whether one request is waiting or thirty-two. Prompts therefore queue behind each other. Eight requests with 4,096-token prompts take 13.6 seconds to read in total and thirty-two take 54.4 seconds; with 8,192-token prompts, thirty-two take 111 seconds. Longer contexts also slow the reading itself: at a context depth of 32K tokens the same model processes prompts at about 1,570 tokens per second.

That is the difference between a chat team and an agent team. People who type questions send short prompts, and one unit carries a lot of them. Retrieval pipelines and coding agents send documents and whole files with every call, and NVIDIA’s own table stops at four such tasks at once. Answers count too: in that test, generating the 1K-token answers took longer than reading the 32K-token prompts. Before sizing, look at the prompt and answer lengths in your logs, not only at the number of users.

Which model to serve

Mixture-of-experts models are what make the Spark usable for a team, because each token reads only the active parameters. gpt-oss-20b has 21 billion parameters with 3.6 billion active and 13.8 GB of weights; gpt-oss-120b has 117 billion with 5.1 billion active and 65.3 GB; Qwen3-Coder-30B-A3B has 30.5 billion with 3.3 billion active. Dense models are the opposite case. LMSYS measured a dense Llama 3.1 70B in FP8 at 2.7 tokens per second for one request and 20.2 in total for eight, 2.5 each: a model for a batch job overnight, not for a team.

Memory decides how many of them fit at once. Our 128 GB article derives a practical working set of about 102 to 115 GB for weights and cache from NVIDIA’s own playbook defaults. gpt-oss-120b and gpt-oss-20b together take about 79 GB of weights, which leaves roughly 23 to 36 GB of cache for both: a fast model for everyday questions and a larger one for the hard ones, on the same unit.

How to share one unit

One serving engine. vLLM, SGLang, TensorRT-LLM, the llama.cpp server and NVIDIA NIM all expose an OpenAI-compatible API, and NVIDIA publishes DGX Spark playbooks for each. Check the concurrency defaults before blaming the hardware: Ollama processes one request per model at a time unless OLLAMA_NUM_PARALLEL is raised, and queues the rest; the llama.cpp server is set up for four concurrent requests by default; vLLM takes its limits from --max-num-seqs and --gpu-memory-utilization. NVIDIA’s vLLM playbook sets the second to 0.8, with the note to raise it towards 0.95 on a dedicated GPU.

A front end with accounts. Open WebUI connects to OpenAI-compatible endpoints such as vLLM and the llama.cpp server. The first account created becomes the administrator, new sign-ups wait in a pending state until someone approves them, permissions decide who sees which models and knowledge bases, and sign-in through OIDC or LDAP is documented. NVIDIA’s Open WebUI playbook sets it up with a local administrator account on the unit.

Developer accounts. DGX OS is Ubuntu, and every developer should have a Linux account of their own. The DGX Dashboard gives each account its own JupyterLab on its own port and working directory, NVIDIA Sync can add the same Spark more than once under different user accounts since version 0.117, and Docker access is granted per user. For access from outside the office, NVIDIA Sync has Tailscale built in.

One workload at a time for training. A fine-tuning run and a serving engine share the same 128 GB and the same GPU. Schedule training outside working hours, or give it a second unit.

Where one unit stops

For chat on gpt-oss-120b with 4,096-token prompts, the measurements put one unit at about 16 simultaneous requests before each one drops below reading pace, and at about 8 if you want every answer to stream at 17 tokens per second or more; gpt-oss-20b roughly doubles that headroom. For coding agents and document pipelines with long prompts, the limit arrives much earlier, at a handful of parallel tasks.

A second Spark adds capacity rather than speed per request. Two linked units hold 256 GB, and StorageReview measured 555 tokens per second in total for gpt-oss-120b at 128 requests with pipeline parallelism, as our two-node article describes, but each request still runs at Spark bandwidth. When a team needs many fast streams from one large model, the next step is a card with more bandwidth: an RTX PRO 6000 Server Edition moves 1,597 GB/s and the Workstation card 1,792 GB/s, against the Spark’s 273. Our first AI project comparison walks through that decision.

What we supply

Eurokommerz supplies the NVIDIA DGX Spark Founders Edition across the EU, with manufacturer warranty, singly or as a linked pair. When a team outgrows one unit, we size the next step from its own numbers, whether that is a second Spark or a server with RTX PRO 6000 cards.

FAQ