Nemotron hardware requirements: Nemotron 3 Nano, Super and Ultra on-premise, with NIM

Nemotron hardware requirements in brief

Nemotron 3 Super, NVIDIA’s 120B open model of March 2026 with 12B active parameters, runs for one user on one RTX PRO 6000 or one DGX Spark in its 80.3 GB NVFP4 checkpoint, and on one H200 NVL or two RTX PRO 6000 in its 128.4 GB FP8 version. Nemotron 3 Nano 30B-A3B, Nemotron 3.5 Lightning and Nano 4B fit one card or one Spark with room for 100 conversations. Nemotron 3 Ultra, with 550B parameters, ships as a 352.3 GB NVFP4 checkpoint and needs four H200 NVL or eight RTX PRO 6000.

These are our memory estimates for conversations of 32,768 tokens, from NVIDIA’s Hugging Face files as read on 9 October 2026. Most Nemotron 3 layers keep no KV cache, so the weights usually decide the GPU count. Our LLM hardware requirements by model applies the same method to other model families.

Nemotron models on Hugging Face in October 2026

Model cards, file lists and config.json files on huggingface.co/nvidia, read on 9 October 2026; sizes are the sums of the safetensors files. Release dates on the cards: Nano 30B-A3B 15 December 2025, Super 11 March 2026, Nano 4B 16 March 2026, Ultra 4 June 2026, 3.5 Lightning 11 August 2026, Super 49B v1.5 25 July 2025.

Nemotron 3 Nano, Super and Ultra interleave Mamba-2 layers with mixture-of-experts (MoE) layers and add a few attention layers. Super’s config.json lists 88 layers: 40 Mamba-2, 40 MoE and 8 attention, with 512 experts of which 22 are active per token. Its card calls the design “LatentMoE” and adds a multi-token prediction (MTP) layer. Ultra has 106 layers, 12 of them attention, and Nano 30B-A3B and 3.5 Lightning have 52 layers, 6 of them attention, with 6 of 128 experts active. Nano 4B has 42 layers, four with attention, and no experts.

Llama-3.3-Nemotron-Super-49B-v1.5 is derived from Llama 3.3 70B Instruct by Neural Architecture Search (NAS), which skips attention in some blocks; by our count of its config.json and weight index, 49 of its 80 layers keep attention.

The RTX PRO 6000 and the DGX Spark compute FP4 directly, while the H200 NVL computes FP8 but not FP4. We found no FP8 checkpoint of Ultra on NVIDIA’s Hugging Face pages, and its NIM documentation states that FP8 profiles were not released “because NVFP4 profiles work on all tested Hopper-architecture GPUs (for example, H100 and H200)”.

KV cache and Mamba state per Nemotron conversation

Only attention layers keep a KV cache that grows with the context: per token, attention layers × KV heads × head dimension × 2 (key and value) × bytes. For Super that is 8 × 2 × 128 × 2 × 1 byte in FP8, 4 KiB per token, against 160 KiB for Llama 3.3 70B with an FP8 cache. Each Mamba-2 layer keeps a fixed state per conversation instead: heads × head dimension × state size, 128 × 64 × 128 values for Super, or 4 MiB in the float32 that vLLM’s recipe sets with --mamba-ssm-cache-dtype float32, plus a small convolution buffer. The general method is in our guide to how much VRAM an LLM needs.

Our arithmetic from the config.json files, read on 9 October 2026. KV cache in FP8, as NVIDIA’s vLLM commands set with --kv-cache-dtype fp8; Mamba state in float32 as an upper bound, Ultra’s in float16 as its card sets. Super 49B in 16-bit, vLLM’s default, as its card’s command sets no cache type.

Our budget follows the VRAM guide: 90 per cent of the memory the driver reports, less 3 GiB per card, which leaves 83.0 GiB per RTX PRO 6000 and 123.4 GiB per H200 NVL. For one DGX Spark (128 GB) we take 102 GB. With tensor parallelism we count the whole cache and state of a Nemotron 3 model on every card, an upper bound. Super 49B’s cache splits over 2, 4 or 8 cards by its eight KV heads, since “TP splits the KV cache by those heads first”, as vLLM’s blog of 7 August 2026 states.

GPUs for 1, 20 and 100 users of each Nemotron model

Our estimates, not measurements: cards for 1, 20 and 100 concurrent conversations of 32,768 tokens in 1, 2, 4 or 8 cards with tensor parallelism, cache as in the table above. DGX Spark shows a memory fit only. W4A16 is NIM’s 4-bit weight profile for 3.5 Lightning, listed for Ampere or newer.

Nano 30B-A3B in FP8 leaves room for 377 conversations on one RTX PRO 6000, so the card is chosen for speed, not memory. Super 49B v1.5 behaves like a dense transformer: at 6.1 GiB per 32K conversation, one RTX PRO 6000 holds five beside the FP8 weights, so 20 users need four cards, or two H200 NVL, and eight hold 100 with no margin. An FP8 cache, set with --kv-cache-dtype fp8, halves the cache per conversation.

We build inference servers with 2 to 8 GPUs per node, sized by model size and concurrent users. Send us the Nemotron variant, its format, your context length and peak requests through the form below.

Nemotron 3 Super on one RTX PRO 6000, a DGX Spark or an H200 NVL

vLLM’s recipe for Nemotron 3 Super, updated on 8 October 2026, states: “The NVFP4 variant runs on a single RTX Pro 6000 at TP=1.” It requires vLLM 0.24.0 or later. Its command sets --gpu-memory-utilization 0.92 “to avoid an OOM at startup”, together with --kv-cache-dtype fp8, --mamba-ssm-cache-dtype float32, --max-num-seqs 8 and MTP speculative decoding with three tokens. By our rule, 8.2 GiB remain beside the 74.8 GiB of weights, room for 29 conversations of 32K; for 20 users, --max-num-seqs has to be raised above 8. We count one Mamba state per conversation, though the recipe’s prefix caching may keep more.

NVIDIA’s NVFP4 card names “1× B200 OR 1× DGX Spark” as the minimum GPU requirement, and its Spark command sets --max-model-len 1000000 with --max-num-seqs 4. By our estimate one Spark holds four conversations of 1M tokens at 4.2 GiB each. Its memory bandwidth is 273 GB/s, against 1,597 GB/s on the RTX PRO 6000 Server Edition, so speed limits a team before memory does.

The FP8 checkpoint’s card gives “2× H100-80GB” as the minimum GPU requirement. One H200 NVL holds the 119.5 GiB of FP8 weights with 3.8 GiB to spare, room for 13 conversations of 32K, so 20 users need a second card, joined by an NVLink bridge.

The RTX PRO 6000 has no NVLink, so the FP8 checkpoint on two cards splits every layer over PCIe, as our guide to one model on several GPUs explains. The NVFP4 checkpoint needs no split, and four copies on four cards hold 116 conversations, while two cards with tensor parallelism hold 160 by our estimate.

Nemotron 3 Ultra on four H200 NVL or eight RTX PRO 6000

Ultra’s card lists “4xGB200, 4xB200, 4x GB300, 4x B300, 8xH100” as the minimum GPU requirement. Its vLLM command for four B200 adds expert parallelism to --tensor-parallel-size 4 and sets --mamba-ssm-cache-dtype float16.

By our estimate four H200 NVL hold the 328.1 GiB of NVFP4 weights with 41.3 GiB per card to spare, room for 109 conversations of 32K. A four-way NVLink bridge joins them into one domain, as our guide to H200 NVL card counts for large models describes, and the Hopper card runs the 4-bit weights weight-only. Four RTX PRO 6000 hold the weights but leave only 1.0 GiB per card for the cache, which is not a usable configuration even for one user. Eight are the minimum and leave 42.0 GiB per card for 111 conversations, split over PCIe. Ultra’s NIM entry lists neither card among its verified GPUs, and its card gives eight H100 (640 GB) as the Hopper minimum, against 564 GB on four H200 NVL, so that layout rests on our memory estimate alone.

We build servers with four or eight H200 NVL and their NVLink bridges, and we check the rack, power and airflow before we quote. Describe the Nemotron model, your rack position and its power feed in the form below.

Nemotron NIM: support matrix and NVIDIA AI Enterprise

The current support matrix of NIM for LLMs, last updated on 6 October 2026 and showing no release number, lists NIMs for Nemotron 3 Nano, Super, Ultra, 3.5 Lightning and Super 49B v1.5. Super has BF16, FP8 and NVFP4 profiles at tensor parallelism of 1, 2, 4 or 8, and “some verified GPUs support only TP4 or TP8 profiles”. The verified GPUs for Nano and Super 49B include the H200 NVL, the RTX PRO 6000 Blackwell Server Edition and the GB10, the chip in DGX Spark; Super’s list includes the first two but not the GB10. For 3.5 Lightning it gives a minimum of 30 GB per GPU for NVFP4 at TP1 and 66 GB for BF16, without “additional headroom for large KV caches”.

Super’s card states that “The NIM container is governed by the NVIDIA Software License Agreement”, and production use of NIM needs an NVIDIA AI Enterprise licence per GPU, as our AI Enterprise licensing guide explains. NVIDIA states that the “H200 NVL comes with a five-year NVIDIA AI Enterprise subscription”, activated by serial number, while the RTX PRO 6000 Server Edition includes none and DGX Spark has its own AI Enterprise product. Without NIM, vLLM serves the same checkpoints under its Apache 2.0 licence.

Nemotron licences: Nemotron Open Model License and OpenMDW

Nano 4B, Nano 30B-A3B and Super are published under the NVIDIA Nemotron Open Model License, last modified on 15 December 2025, which states: “Works are commercially usable.” The licence ends for a Work if you start patent or copyright litigation claiming that it infringes. Ultra and 3.5 Lightning come under the OpenMDW License Agreement, version 1.1, which requires a copy of the agreement and the copyright notices on distribution and also ends on such litigation, unless it answers a suit first brought against you. Ultra’s card states: “This model is ready for commercial and non-commercial use.” Super 49B v1.5 names the NVIDIA Open Model License and the Llama 3.3 Community License Agreement, and its card carries “Built with Llama”. Whether a clause applies to your company is a legal assessment for your legal department.

What we supply

We supply the DGX Spark Founders Edition, the RTX PRO 6000 in its Workstation, Max-Q and Server editions, and the H200 NVL with its NVLink bridges, as cards or in AI servers built to order. NVIDIA AI Enterprise licences for NIM, listed on our software and licensing page, come on one EU contract and invoice with the hardware and its manufacturer warranty. The models, RAG and MLOps on top are our Private AI/ML service, with engineering by our partner Vixen.UNO.

FAQ