Nemotron 3.5 Super VL hardware: RTX PRO 6000 Blackwell Server Edition or H200 NVL

What Nemotron 3.5 Super VL needs from a GPU

Nemotron 3.5 Super VL runs on-premise on one NVIDIA H200 NVL or on two NVIDIA RTX PRO 6000 Blackwell Server Edition cards with room for many documents and videos at once; one Server Edition card holds the model with 13.2 GiB to spare by our users-per-card rule, enough for a pilot or a small department. Its NVFP4 checkpoint is 75.0 GB, and the memory each request adds is small, because only a few of its layers keep a KV cache.

NVIDIA published the model on Hugging Face on 15 October 2026 as a "Mamba-2–Transformer hybrid Mixture of Experts with a C-RADIO vision encoder". It has 120B parameters, of which 12B are active per token, 512 routed experts per MoE layer, and takes text, images and video. Our Nemotron hardware guide covers the text models of the family and leaves this one out of its tables; this page sizes it. The figures below are our estimates from NVIDIA's files, read on 24 October 2026, with the arithmetic shown.

The checkpoints and what is in them

Hugging Face model cards and file lists of nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 and -BF16, read on 24 October 2026; sizes are our sums of the safetensors files.

The NVFP4 version comes from quantisation-aware distillation of the BF16 model. Its card states that "The vision encoder, connector, Mamba convolutions, and embeddings remain in BF16" and that "the MoE router remains in FP32", so the file is larger than a flat 4 bits per parameter would give. On the benchmarks NVIDIA lists, NVFP4 stays close to BF16 on most rows, for example OCRBench v2 (English) 68.34 against 68.12 and SWE-bench Verified 71.07 against 71.10, but drops on a few agentic ones, such as WebVoyager 66.31 against 77.72.

The card also states: "Inference uses W4A4 on Blackwell and W4A16 on non-Blackwell platforms." On an RTX PRO 6000 the 4-bit weights are computed in FP4 on the Tensor Cores; on an H200 NVL they are unpacked to 16 bits for each operation, which saves memory but not arithmetic. The difference between the formats is explained in our guide to FP4 models on H200 NVL and Ada GPUs.

Memory per request: images, pages and video

Each request needs a KV cache in the attention layers and a fixed state in the Mamba-2 layers. The language model's config.json gives 2 KV heads of dimension 128 per attention layer, and eight layers use attention, as in Nemotron 3 Super: our count of the layer list in config.json, which matches the eight attention-sized weight files among the 88 in the NVFP4 repository. That makes 8 × 2 × 128 × 2 (key and value) × 2 bytes = 8 KiB per token with a 16-bit cache, which vLLM uses unless --kv-cache-dtype says otherwise, or 4 KiB with an FP8 cache. Each Mamba-2 layer holds 128 heads × 64 × 128 values per request, 2 MiB in the float16 that the card's vLLM command sets; for about 40 such layers that is roughly 80 MiB per request, whatever its length.

Images become tokens before they reach the cache. The config sets a patch size of 16 pixels, a downsample ratio of 0.5, which in this InternVL-style design merges 2 × 2 patches into one token, and 1,024 to 13,312 patches per image. By our arithmetic one image therefore costs 256 to 3,328 tokens. An A4 page scanned at 150 dpi, 1,240 × 1,754 pixels, comes to about 8,500 patches and 2,100 tokens.

For video the card states MP4 "up to 2 minutes", sampled by default at 2 frames per second up to 256 frames, with a video pruning rate of 0.5 in its serve commands; for 1080p it recommends 1 frame per second and up to 128 frames. The card gives no token count per frame, so measure the prompt_tokens your clips produce in the usage field of the API response. The upper limit is the context: the card states that "The prompt, visual inputs, conversation history, tool results, and generated response share this context window", up to 1M tokens, and 1M tokens of 16-bit cache is 8 GiB by the figure above.

Our estimate: 8 KiB of 16-bit KV cache per token plus about 80 MiB of Mamba-2 state per request; an FP8 cache halves the first part. The examples are illustrations; measure your own inputs.

This is the main difference from a dense vision-language model. Qwen3.8-27B needs about 0.36 GiB for the same scanned page, as our guide to vision-language models for document AI works out, three times as much from a model a quarter of the size. For Nemotron 3.5 Super VL the weights decide the card count, and the cache decides how many requests run at once.

One Server Edition, two, or one H200 NVL

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card for CUDA graphs and activations: 83.0 GiB on an RTX PRO 6000 Server Edition, which reports 95.6 GiB, and 123.4 GiB on an H200 NVL, which reports 140.4 GiB. The rule is derived in our guide to how many users one RTX PRO 6000 can serve. What remains after the 69.9 GiB of NVFP4 weights holds the cache.

Our estimate by the users-per-card rule: requests held in memory at the same moment, each at the full size shown, 16-bit KV cache and float16 Mamba state as above. An FP8 cache roughly doubles the 128K and 1M columns.

These are memory limits. The card's vLLM commands cap the scheduler at --max-num-seqs 128 on H100 and H200, and the vision encoder's activations come out of the same free memory, so keep headroom above the table when many images arrive at once. In the card's own words, "Individual deployment recipes may use a smaller context window to accommodate the available GPU memory and concurrency."

With a second Server Edition card the layout matters more than the extra memory. Two copies, one per card, keep 13.2 GiB each and need no traffic between the cards. One copy split over both with tensor parallelism stores the weights once and leaves 96.2 GiB for requests, seven times as much, at the price of an all-reduce over PCIe for every layer, since the card has no NVLink. Our comparison of TP=2, TP=4 and replicas on the RTX PRO 6000 covers that trade. We supply both cards and build the servers around them; send us your document volume, video lengths and peak requests, and our team returns a configuration with the figures for each layout.

Software and supported hardware as of October 2026

The card lists the NVIDIA Blackwell and Hopper architectures and gives four deployment targets: "1× DGX Spark, 1× H200, 2× H100, or 1× GB200". Its deployment table shows text, image and video with a 1M-token context and multi-token prediction for H200 and H100, and text and image for DGX Spark. The vLLM commands use the image vllm/vllm-openai:v0.32.0 or a nightly build, --trust-remote-code, the reasoning parser nemotron_v3 and the tool parser qwen3_coder. For one H200 the card says to run the two-H100 command with --tensor-parallel-size 1; that command sets --gpu-memory-utilization 0.89, a Mamba cache in float16 with stochastic rounding, three MTP draft tokens and video at 2 frames per second up to 256 frames. SGLang and TensorRT-LLM 1.3.0rc28 configurations are on the card as well.

The RTX PRO 6000 Server Edition is not among the targets on the card, nor among the seven SKUs of NVIDIA's Dynamo recipe for the model (H100, H200, B200, GB200, B300, GB300 and VR200), all of which use "NVFP4 weights, FP8 KV". It is a Blackwell card with FP4 Tensor Cores, and vLLM's recipe runs the text model Nemotron 3 Super in NVFP4 on one RTX PRO 6000, as our Nemotron guide quotes, but we found no published run of the VL model on it as of 24 October 2026. Plan a pilot on the delivered system before you size more than one server. NVIDIA's NIM support matrix, updated on 19 October 2026, lists a Nemotron 3.5 Super VL NIM with NVFP4 profiles at TP1 and TP2 and BF16 profiles at TP2, TP4 and TP8; production use of a NIM needs an NVIDIA AI Enterprise licence per GPU, as our guide to NVIDIA AI Enterprise licences explains.

Throughput: what is published and what to measure

We found no published throughput for this model on an RTX PRO 6000 or an H200 NVL as of 24 October 2026. NVIDIA's technical report of 21 October 2026 gives figures for GB200 only, a different class of system. The rate a document or video service needs is therefore best found in a pilot: send a representative set of pages and clips, read the prompt and output tokens per request from the API's usage field and the requests per second from vLLM's metrics, and scale from there.

Three settings decide the result. The render resolution of document pages sets the image tokens, from 256 to 3,328 per image. The frame rate and frame cap set the tokens per clip, and the card's advice of 1 frame per second for 1080p halves the frames. Reasoning adds output tokens to every request; the card recommends temperature 0.6 and top_p 0.95 for vision tasks. The H200 NVL reads memory at 4.8 TB/s against 1,597 GB/s on the Server Edition, while the Server Edition computes the 4-bit weights natively, so the two cards do not rank the same way on every workload. Our RTX PRO 6000 vs H200 NVL comparison sets out where each one leads.

Which configuration to choose

One RTX PRO 6000 Server Edition suits a pilot or a department that sends a few dozen pages or short clips at a time. It holds the model with 13.2 GiB of cache, 40 requests of 32K by our estimate, and runs the 4-bit weights natively. It fits an existing server whose maker lists the card for that slot, as our list of RTX PRO 6000 Server Edition compatible servers explains.

Two Server Edition cards as one tensor-parallel copy hold 293 requests of 32K or 11 at the full 1M context. This is our entry server: two RTX PRO 6000 Server Edition, two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives, about 1.8 kW at the wall by our estimate, within one 230 V 16 A feed. Our page on the two-card Server Edition server sets out what else it runs.

One H200 NVL holds 163 requests of 32K beside the NVFP4 weights; NVIDIA's card names one H200, and the H200 NVL has the same 141 GB of HBM3e. Two H200 NVL also take the BF16 checkpoint, 231.8 GiB, with 14.9 GiB to spare, which suits a team that wants to compare the two formats; our two-H200 NVL server draws about 2.0 kW by our estimate. Four Server Edition cards hold the BF16 weights too, with 100.3 GiB to spare.

For pure OCR at volume, a small document model on a 24 GB card costs far less per page, as our document AI guide shows; Nemotron 3.5 Super VL is the better choice where pages need reasoning, tables and charts must be interpreted, or video has to be described. For camera streams that run all day, our guide to a GPU server for video analytics covers the decoders and stream counts. We build servers with one to eight RTX PRO 6000 Server Edition or H200 NVL cards; describe your documents, clips and response times, and we size the card count and the layout.

Licence: OpenMDW-1.1

The model card states: "This model is ready for commercial or non-commercial use." The OpenMDW License Agreement 1.1 grants permission "to deal in the Model Materials without restriction" and contains no field-of-use limit, revenue threshold or territorial restriction, so it does not exclude companies in the EU. On distribution it requires "a copy of this agreement" together with the copyright notices and other notices of origin that come with the materials. Rights end if you start litigation claiming that the materials infringe a patent or copyright, unless it answers a suit first brought against you. The materials are provided "AS IS". Whether a clause applies to your company is for your legal counsel to confirm, and our overview of open LLM licences for company use compares the other families.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the RTX PRO 6000 Server Edition and the H200 NVL as cards or in AI servers built to order, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component; quantities and lead time are confirmed in writing with the quote. Deploying the model with retrieval and the document or video pipeline on top is our Private AI/ML service, delivered with our engineering partner Vixen.UNO.

FAQ