gpt-oss-puzzle-88B hardware requirements: VRAM, users and one NVIDIA RTX PRO 5000 72 GB

How much GPU memory gpt-oss-puzzle-88B needs

gpt-oss-puzzle-88B needs 46.6 GiB of GPU memory for its weights, plus about 0.75 GiB of 16-bit KV cache for every conversation of 32,768 tokens in flight. One NVIDIA RTX PRO 5000 Blackwell 72 GB therefore runs it for about 19 concurrent conversations at that length, or 39 with an FP8 cache, by our sizing rule. The parent model, gpt-oss-120b, loads on the same card with 0.7 GiB to spare, which holds no conversation at all.

NVIDIA published the model on Hugging Face on 26 March 2026 as version 1.0 and describes it as "a deployment-optimized large language model developed by NVIDIA, derived from OpenAI's gpt-oss-120b". The Files page lists eleven safetensors files that add up to 50.00 GB, which is 46.57 GiB; the page shows the folder as 50 GB. The card gives the size as about 88B parameters and explains that the 91B on the Hub "includes additional MXFP4 quantization scale tensors". Our gpt-oss-120b hardware requirements list the model as one row; this page sizes it card by card.

What NVIDIA pruned out of gpt-oss-120b

The config.json keeps the shape of gpt-oss-120b: 36 layers, a hidden size of 2,880, 64 attention heads, 8 KV heads of dimension 64, and 4 experts per token. What changed is the number of experts and the attention pattern. The first 15 layers and three more keep all 128 experts; the other 18 layers have 64. That makes 3,456 experts against 4,608 in the parent, and the expert weights stay in MXFP4 while attention, router, embeddings and output head stay in BF16, as the quantisation block of the config lists.

Hugging Face model cards, file lists and config.json of openai/gpt-oss-120b and nvidia/gpt-oss-puzzle-88B, read on 24 October 2026; checkpoint is the sum of the top-level safetensors files.

Each token still passes through 4 experts of the same size in every layer. By our reading of the config, the weights read per generated token are therefore close to those of the parent; the savings sit in the weights held in memory and in the cache. NVIDIA does not state an active-parameter count.

NVIDIA reports on the card that the model "matches or slightly exceeds parent accuracy across reasoning efforts", on MMLU-Pro, GPQA-Diamond, HLE, AIME25, IFBench, SciCode, AA-LCR and RULER 128K. The figures sit in charts on the card, so we do not repeat them. The three reasoning levels of gpt-oss remain, set with reasoning_effort.

KV cache per conversation from block_configs

The config has a block_configs entry per layer. Ten layers keep the full context, eight keep a window of 8,192 tokens and eighteen a window of 128 tokens. Each layer stores keys and values for 8 heads of 64 values, 2 KiB per token in 16-bit or 1 KiB in FP8. A full layer grows with the conversation; a windowed layer stops growing at its window.

Our arithmetic from config.json: layers × 2 (keys and values) × 8 KV heads × 64 × bytes per value × tokens held, counting the 8,192-token and 128-token windows at their window size. gpt-oss-120b from our gpt-oss guide.

At 32K that is 10 × 2 KiB × 32,768 = 640 MiB for the full layers, 8 × 2 KiB × 8,192 = 128 MiB for the 8K-window layers and 4.5 MiB for the short windows, 772.5 MiB in all. At 8K the puzzle model saves nothing, because an 8,192-token window holds the whole conversation; at 131K it needs 42 per cent less cache than gpt-oss-120b, in line with the "~40%" NVIDIA states for long contexts. The card's FP8 cache with calibrated scales halves the figures again, and NVIDIA gives it "~2× KV-cache token capacity".

The table assumes the engine reserves only the window for windowed layers. If an engine allocates full-length cache for the 8K-window layers, a 32K conversation takes 1.13 GiB in 16-bit by our arithmetic, as much as on gpt-oss-120b. The cache size and concurrency vLLM reports at start-up are the figures to plan with, as our guide to vLLM KV cache size and maximum concurrency explains.

One RTX PRO 5000 72 GB: weights, cache and conversations

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card for activations and the runtime. For the 72 GB card that is 0.9 × 71.7 − 3 = 61.5 GiB, as on our four-card RTX PRO 5000 72 GB workstation page. After 46.6 GiB of weights, 14.9 GiB is left for cache.

Our estimates by the users-per-card rule: concurrent conversations each holding a full 32,768-token context, 0.75 GiB in 16-bit or 0.38 GiB in FP8; the two 48 GB cards run one copy with TP=2. Usable memory as on our RTX PRO 5000, RTX PRO 6000 and H200 NVL pages.

For comparison, gpt-oss-120b holds 19 conversations of 32K on one RTX PRO 6000 Max-Q and 55 on one H200 NVL by the same rule. The puzzle model on a 72 GB card matches what its parent does on a 96 GB card. At the full 131,072 tokens the 72 GB card holds 5 conversations with a 16-bit cache and 11 with FP8.

These are requests in flight at the same moment, not staff. A department of 200 with the assistant open produces far fewer generations at once, and the peak at your gateway is the figure to size against, as our article on how many users one RTX PRO 6000 serves shows. A higher reasoning level makes every request longer and keeps it in flight longer.

The 48 GB card cannot hold the weights alone: 40.0 GiB is less than 46.6. Two of them split one copy with tensor parallelism over PCIe; the 8 KV heads divide evenly over two cards, so the cache of both adds up. Our comparison of the RTX PRO 5000 48 GB and 72 GB covers the difference in price class and MIG.

We supply the RTX PRO 5000 72 GB as a card and in workstations built to order; send us the model, the context and the peak requests in flight, and our team replies with the card count and a configuration.

Software: the card's vLLM build and the hardware it names

The model card names vLLM as the runtime engine and lists "NVIDIA B200" and "NVIDIA H100-80GB" as supported hardware, with Linux as the operating system. Its instructions start from the container image vllm/vllm-openai:v0.17.1, install FlashInfer 0.6.6 and a vLLM build pinned to one git commit, and serve the model with --trust-remote-code. The repository brings its own modelling code, and vLLM's supported-models list, read on 24 October 2026, has a row for gpt-oss but none for the puzzle architecture.

The card's example command sets --kv-cache-dtype fp8, --gpu-memory-utilization 0.95, --max-num-seqs 8 and --max-model-len 131072. Two of these matter for sizing. At 0.95 the 72 GB card gives about 3.6 GiB more cache than our 0.9 rule; and --max-num-seqs 8 caps the server at eight requests at once, so raise it for a shared service. On the B200 the card adds VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8=1.

The RTX PRO 5000, like every RTX PRO Blackwell card, is compute capability 12.0, which the engines call SM120. vLLM 0.32.0 of 18 October lists "OAI Triton MXFP4 MoE on SM12x" and a fallback for "MXFP4 W4A4 checkpoints" to W4A16 "when no W4A4 kernel exists", and TensorRT-LLM's quantisation matrix lists MXFP4 for sm120. The card does not name either for this model, so a run of the card's pinned build on one RTX PRO 5000 comes before an order for many. Our guide to Blackwell RTX software support on SM120 tracks the kernels.

Generation speed on 1,344 GB/s

Each new token reads the active weights once, so memory bandwidth sets a ceiling for one user. gpt-oss-120b reads about 5 GB per token, and the puzzle model, with the same 4 experts per layer, reads about the same. On the 1,344 GB/s of an RTX PRO 5000 that gives a ceiling of roughly 270 tokens per second for one request, and about 360 on the 1,792 GB/s of an RTX PRO 6000 Max-Q, by our estimate an upper bound, since batched tokens hit different experts and attention adds its own reads.

NVIDIA's own throughput claims compare the model with gpt-oss-120b: 1.63 times at 64K input and output on an 8 × H100 node, 1.22 times at 4K and up to 2.82 times on a single H100. By our reading, they follow from the smaller cache, which leaves room for larger batches. We found no published measurement of the puzzle model on an RTX PRO card.

Workstation or server for the puzzle model

One RTX PRO 5000 72 GB draws 300 W. With a 350 W workstation processor and 15 per cent for the rest, a one-card workstation comes to about 0.75 kW at the wall by our estimate, and two cards to about 1.1 kW, both well within an office circuit. Four cards in a tower run four independent copies, 76 conversations of 32K with a 16-bit cache or 156 with FP8, at about 1.8 to 2 kW, as our page to buy an RTX PRO 5000 72 GB workstation sets out.

MIG does not help with this model: the datasheet splits the 72 GB card into two halves of 36 GB, smaller than the weights. For virtual machines, the 72 GB card is on NVIDIA's vGPU list from release 20.2, with Red Hat Enterprise Linux and KVM as the only hypervisor listed. A service for several departments in continuous operation belongs in a rack, with RTX PRO 6000 Server Edition cards or with the RTX PRO 5000 in the chassis covered by our RTX PRO 5000 rack server page.

The choice between one RTX PRO 6000 Max-Q and one RTX PRO 5000 72 GB comes down to cache: 36.4 GiB against 14.9 GiB after the same weights, and a third more bandwidth on the Max-Q. Our comparison of two RTX PRO 5000 72 GB or one RTX PRO 6000 Max-Q works through that decision for other models.

Licence: NVIDIA Open Model License

The card states that use "is governed by the NVIDIA Open Model License" and that the model "is ready for commercial use"; it gives the deployment geography as "Global". The licence page, version of 24 October 2025, says "Models are commercially usable" and grants "a perpetual, worldwide, non-exclusive, no-charge, royalty-free, revocable" licence. Anyone who redistributes the model includes a Notice file reading "Licensed by NVIDIA Corporation under the NVIDIA Open Model License".

Two clauses end the rights. They terminate automatically if you bypass or reduce the efficacy of a safety guardrail without a substantially similar one, and they end on the date you file a copyright or patent suit over the model. NVIDIA "claims no ownership rights in outputs", the licence asks for compliance with export and sanctions laws, and it is governed by the laws of the United States and the State of Delaware. gpt-oss-120b itself is under Apache 2.0. Whether the terms fit your use is for your legal counsel; our overview of open LLM licences for company use compares the common ones.

When gpt-oss-120b or another card fits better

Choose gpt-oss-120b when you want OpenAI's release unchanged, for instance to compare results with published work, or when you plan to fine-tune from it, as our guide to fine-tuning gpt-oss-120b does. On a 96 GB or 141 GB card the parent runs with room for cache, so the memory argument for the puzzle model weakens there. It stays strongest on 72 GB, and on a pair of 48 GB cards, where the parent leaves 19.2 GiB for cache against 33.4 GiB for the puzzle model.

For the L40S and other Ada cards, we found no engine documentation of MXFP4 support for this model, so we do not size them here. On Hopper, the H100-80GB on the card's list stores MXFP4 weights and computes at higher precision; the H200 NVL with 141 GB of HBM3e at 4.8 TB/s holds the model with room for about 100 conversations of 32K by our rule.

What we supply

You can buy this configuration from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the RTX PRO 5000 Blackwell 72 GB and 48 GB as cards for a workstation you already run, and build workstations with one to four cards to order, assembled and tested before delivery, with the operating system, driver, CUDA and a container runtime installed on request, on one EU contract and invoice with manufacturer warranty. All cards are on our professional NVIDIA GPU page. Write to us with the workstation or server you plan, and we confirm slots, power supply and circuit before we quote.

FAQ