One DGX Spark for a team: how many people it serves, and how to share it

How many people one DGX Spark serves

By the llama.cpp maintainers’ published measurements, one DGX Spark (128 GB) serving gpt-oss-120b with 4,096-token prompts keeps each answer at 10 tokens per second or more, a comfortable reading pace, up to about 16 simultaneous requests. How many people that covers depends on how often they send requests. NVIDIA positions the DGX Spark for “AI developer, researcher, and data scientist workloads”, and we found no NVIDIA figure for how many people can share one. The nearest statement is about software: “With support from frameworks that handle concurrency well (such as NVIDIA TensorRT LLM, vLLM, and SGLang), multiagent workloads run smoothly on NVIDIA DGX Spark.” The warranty is more direct about intended use: a “small-scale server for consumer and enterprise end user and multi-users purposes”, as opposed to “large-scale datacenter use or GPU cluster commercial deployments”, which it calls “Enterprise Use” and excludes. A Spark shared by a team in an office fits the first description. The German warranty text renders the exclusion as use „für Unternehmenszwecke“, for business purposes, so ask the seller to confirm the warranty for your use; our guide to buying a DGX Spark in the EU quotes the English and the German wording.

The right sizing measure is the number of simultaneous requests rather than the number of people. Ten developers with a coding assistant rarely send ten requests in the same second, and a document pipeline sends all of its requests at once. Our article on users per RTX PRO 6000 explains how to read your own logs for the number that matters; here we apply the published measurements to the Spark.

Speed per request, from 1 to 32 at once

The most complete public data comes from the llama.cpp maintainers, who benchmark the DGX Spark with several requests in flight. The table gives the generation speed each request sees.

Tokens per second per request. llama.cpp maintainers’ DGX Spark results (llama-batched-bench, build of February 2026, 4,096-token prompt and 32 generated tokens per request); per-request speed is the published aggregate divided by the number of requests.

Aggregate throughput keeps rising, from 55 tokens per second for one request on gpt-oss-120b to 262 for 32, while each request slows down. LMSYS measured the same pattern at launch in October 2025, with SGLang and 2,048-token prompts and answers. In its test, gpt-oss-120b fell from 50.5 tokens per second alone to 14.6 per request at 8, 9.2 at 16, 6.3 at 32 and 4.6 at 64, for a total of about 290 tokens per second. NVIDIA’s own concurrency test, published in March 2026, ran Qwen3 Coder Next in FP8 on vLLM with 32K-token prompts and 1K-token answers. One, two and four tasks at once took 35, 54 and 91 seconds, and the first token arrived after a median 9, 12 and 15 seconds.

Our 70B sizing example works with 10 tokens per second as a comfortable reading pace. Measured against that, gpt-oss-120b on one Spark stays above it up to about 16 simultaneous requests with 4,096-token prompts and falls below it at 32, while gpt-oss-20b stays above it at 32. We supply the DGX Spark Founders Edition and size the next step from your own numbers. Tell us your model and how many requests arrive at once, and we tell you whether one Spark carries them.

Long prompts and time to first token

Prompt processing (prefill) is compute-bound, and on the Spark it runs at a fairly fixed rate: about 2,400 tokens per second for gpt-oss-120b in the llama.cpp results, whether one request is waiting or thirty-two. Prompts therefore queue behind each other. Eight requests with 4,096-token prompts take 13.6 seconds of prompt processing in total, and thirty-two take 54.4 seconds; with 8,192-token prompts, thirty-two take 111 seconds. Longer contexts also slow prompt processing: at a context depth of 32K tokens the same model processes prompts at about 1,570 tokens per second.

This is why chat and agent workloads are sized differently. People who type questions send short prompts, and one Spark can handle many of them. Retrieval pipelines and coding agents send documents and whole files with every call, and NVIDIA’s own table stops at four such tasks at once. Answer length matters as well; in that test, generating the 1K-token answers took longer than processing the 32K-token prompts. Before sizing, look at the prompt and answer lengths in your logs, not only at the number of users.

Which model to serve

Mixture-of-experts models are what make the Spark usable for a team, because each token reads only the active parameters. gpt-oss-20b has 21 billion parameters with 3.6 billion active and 13.8 GB of weights; gpt-oss-120b has 117 billion with 5.1 billion active and 65.3 GB; Qwen3-Coder-30B-A3B has 30.5 billion with 3.3 billion active. Dense models behave differently. LMSYS measured a dense Llama 3.1 70B in FP8 at 2.7 tokens per second for one request and 20.2 in total for eight, 2.5 each, which suits an overnight batch job rather than a team.

Memory decides how many of them fit at once. Our 128 GB article derives a practical working set of about 102 to 115 GB for weights and cache on a 128 GB Spark from NVIDIA’s 0.8 and 0.9 playbook settings, and about 90 GB from the 0.7 of NVIDIA’s single-Spark vLLM launch script. On its own, gpt-oss-120b leaves room in 102 GB for about 30 conversations of 32K tokens each by our count, and in 90 GB for about 20, so for chat the speed per request in the table above sets the limit before memory does. gpt-oss-120b and gpt-oss-20b together take about 79 GB of weights, which leaves roughly 23 to 36 GB of cache for both, or about 10 GB at 0.7. That allows a fast model for everyday questions and a larger one for harder questions on the same Spark, as long as the two engines’ memory fractions add up to that budget rather than each taking its default. On the 64 GB configuration, available from OEM partners since 23 October 2026, the two models’ 79 GB of weights do not fit.

How to share one DGX Spark

One serving engine. vLLM, SGLang, TensorRT-LLM, the llama.cpp server and NVIDIA NIM all expose an OpenAI-compatible API, and NVIDIA publishes DGX Spark playbooks for each. Check the concurrency defaults first: Ollama processes one request per model at a time unless OLLAMA_NUM_PARALLEL is raised, and queues the rest; the llama.cpp server is set up for four concurrent requests by default; vLLM takes its limits from --max-num-seqs and --gpu-memory-utilization. NVIDIA’s vLLM playbook, as read on 10 October 2026, sets the second to 0.7 in the single-Spark launch script linked from build.nvidia.com, to 0.8 or 0.4 in its other single-Spark examples and to 0.8 or 0.9 on two, and its examples cap --max-num-seqs at between 1 and 10, so further requests wait in the queue.

A front end with accounts. Open WebUI connects to OpenAI-compatible endpoints such as vLLM and the llama.cpp server. The first account created becomes the administrator, new sign-ups wait in a pending state until someone approves them, permissions decide who sees which models and knowledge bases, and sign-in through OIDC or LDAP is documented. NVIDIA’s Open WebUI playbook sets it up with a local administrator account on the unit. Open WebUI fixed two High-rated advisories in version 0.11.4 (September 2026) and a further one in 0.12.2 (18 October 2026), so run 0.12.2 or later on a Spark that several people sign in to. If Ollama serves the models, run 0.35.0 or later, which fixes CVE-2026-103663 (8 October 2026), rated critical (CVSS 9.4) where the Ollama service can write to /usr/lib/ollama, otherwise 6.9, and keep port 11434 off the office network.

Developer accounts. DGX OS is Ubuntu, and every developer should have a Linux account of their own. The DGX Dashboard gives each account its own JupyterLab on its own port and working directory, NVIDIA Sync can add the same Spark more than once under different user accounts since version 0.117, and Docker access is granted per user. For access from outside the office, NVIDIA Sync has Tailscale built in. NVIDIA’s Brev playbook, updated 21 September 2026, registers a Spark with NVIDIA Brev as “A shared GPU workspace your team can access from anywhere”; it needs a Brev account and network access to Brev services.

One workload at a time for training. A fine-tuning run and a serving engine share the same 128 GB and the same GPU. Schedule training outside working hours, or give it a second Spark.

You can order the hardware alone, the setup of the serving engine, front end and accounts alone, or both together: the Spark delivered with everything installed and configured; write to us with the number of users and the requests that arrive at once.

When a team needs more than one Spark

For chat on gpt-oss-120b with 4,096-token prompts, the measurements put one Spark at about 16 simultaneous requests before each one drops below reading pace. If every answer should stream at 17 tokens per second or more, the figure is about 8. gpt-oss-20b roughly doubles both figures. For coding agents and document pipelines with long prompts, the limit is reached much earlier, at a handful of parallel tasks.

A second Spark adds capacity more than speed per request. Two linked 128 GB Spark systems hold 256 GB, and StorageReview measured 555 tokens per second in total for gpt-oss-120b at 128 requests with pipeline parallelism, as our two-node article describes. With pipeline parallelism, each request still runs at the bandwidth of one Spark. When a team needs many fast streams from one large model, the next step is a card with more bandwidth: an RTX PRO 6000 Server Edition in a server moves 1,597 GB/s and the RTX PRO 6000 Workstation Edition in a workstation 1,792 GB/s, against the Spark’s 273. Our first AI project comparison walks through that decision.

For a whole company rather than one team, sizing a private ChatGPT server by company size works through configurations for 100, 500 and 2,000 employees.

What we supply

We supply the NVIDIA DGX Spark Founders Edition (128 GB, 4 TB) across the EU on one contract and invoice, with manufacturer warranty; quantities and lead time are confirmed in writing with the quote. When a team outgrows its Spark, we size the next step from its own numbers: two or four linked Spark systems, or a GPU server built to order with RTX PRO 6000 Server Edition cards.

FAQ