Local LLM on 16 GB: which models fit the NVIDIA RTX PRO 2000 Blackwell and RTX 2000 Ada

A local LLM on a 16 GB card

A compact workstation with one 16 GB card can hold a local language model for a single user or a small team. On the NVIDIA RTX PRO 2000 Blackwell and the NVIDIA RTX 2000 Ada Generation our sizing rule leaves 11.3 GiB for model weights plus KV cache. Model files up to about 10 GiB fit, for example Ministral 3 8B Instruct in FP8, Gemma 4 12B-it in Google's 4-bit QAT files or phi-4 as a Q4_K GGUF. gpt-oss-20b is above that budget, and we do not plan on it; 14B models in FP8 need 24 GB.

The page works with file sizes and arithmetic. It prints no speeds, because the sources we accept hold no measurement in tokens per second for either card; a later section says what is known instead. Servers with the L4 and counts of users per model are covered in our guide to small language models on the L4 and RTX PRO 2000.

The 11.3 GiB budget and the three fit classes

The rule is our own: a card has 0.9 times the memory the driver reports, less 3 GiB, for weights plus KV cache. NVIDIA publishes no driver-reported memory for these cards, so we scale it from the 95.6 GiB the driver shows on the 96 GB card. That gives 15.9 GiB reported, 14.3 GiB at 0.9 and 11.3 GiB after the 3 GiB. The scaling is an assumption; nvidia-smi on the delivered card shows the real figure, and that figure decides.

Three classes follow from it.

• Fits. The weights are within the budget, and what remains of the 11.3 GiB is room for context.
• Tight. The weights are above the budget and within 0.9 times the reported memory, here between 11.3 and 14.3 GiB. Such a file is above our budget, and we do not plan on it.
• Does not fit. The weights exceed 0.9 times the reported memory.

The rule was written for vLLM serving several users. For one user with a GGUF file in llama.cpp or Ollama it is conservative, and no published measurement says by how much on these cards. We therefore state the rule and the file size and keep the class "tight" in the tables as a mark of its own. Our guide to how much VRAM an LLM needs explains the rule itself.

Model files that fit a 16 GB card

All sizes in the table are in GiB and come from the publishers' own Hugging Face repositories as we read them on 10 November 2026. The third column is 11.3 GiB less the weights.

file sizes and licence tags from the publishers' Hugging Face repositories, read 10 November 2026; budget by our sizing rule. The rows for Gemma 4 12B-it Q4_0 and Ministral 3 14B Q4_K_M are GGUF files and include the publisher's mmproj file; the Ministral 3 14B row is the Instruct model.

Three notes help with reading the table. First, the remainder is shared: the KV cache of every open conversation comes out of it, and it grows with the length of the context and with the number of parallel users. Second, the name of a model says less than its file. The Gemma 4 E2B checkpoint holds about 5.1 billion parameters, which is why its BF16 file takes 9.54 GiB, and NVIDIA's repository named Nemotron 3.5 Bolt E4B-4GB holds a Q4_K_M file of 5.76 GiB. That file fits with 5.5 GiB left; its licence is OpenMDW-1.1, whose terms we have not read. Third, Qwen3.5-4B in BF16 (8.68 GiB) fits as well, with 2.6 GiB left, but we did not re-read its licence for this page, so check the model card before commercial use.

The licence column gives the tag on the repository. Whether a clause matters for your company is a question for your legal department; our overview of LLM licences for commercial use describes the common ones.

You can order the cards or a workstation alone, the setup of vLLM or Ollama with a local model alone, or both together: the workstation delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

gpt-oss-20b and 14B models in FP8 on a 16 GB card

file sizes from the publishers' Hugging Face repositories, read 10 November 2026; classes by our sizing rule with budgets of 11.3 GiB for 16 GB and 18.5 GiB for 24 GB. The Gemma 4 26B-A4B row includes the publisher's mmproj file. The Ministral 3 14B rows are the Instruct model. All files are tagged Apache 2.0 except phi-4, which is MIT; the gpt-oss-20b repository also holds a usage policy file.

The three files classed as tight lie between 11.3 and 14.3 GiB. Take gpt-oss-20b: OpenAI publishes it only as MXFP4, a file of 12.82 GiB, so no smaller original exists. On a 16 GB card it is above our budget, and we do not plan on it. The last row shows the other edge: an 8B model fits in FP8 and does not fit in BF16.

RTX PRO 2000 Blackwell or RTX 2000 Ada for AI, FP4 against FP8

Both cards have 16 GB, so the budget is the same. Both draw 70 W without a power cable and are half-height (2.7 in), the format usually called low profile, and two slots wide. They differ in generation.

NVIDIA datasheets 5322351 (June 2026) and 2999417 (January 2024); NVIDIA's "CUDA GPU Compute Capability" page (16 October 2026); budget by our sizing rule.

NVIDIA states that the fifth-generation Tensor Cores of the Blackwell cards add support for FP4 precision. For the Ada generation it documents FP8: its NIM support matrix for visual generative AI of 20 October 2026 requires compute capability 8.9 for FP8, which the RTX 2000 Ada has, and Ada cards have no hardware acceleration for FP4. In practice the question is which checkpoint you want to run. NVFP4 files compute natively only on Blackwell cards, so they belong on the RTX PRO 2000; on the RTX 2000 Ada the choices are FP8, BF16 and GGUF files. None of the files in the first table is NVFP4, although publishers offer such versions, for instance IBM for Granite 4.2, which we did not size. Our article on FP8, NVFP4, MXFP4, INT4 and GGUF explains which card runs which format.

The two AI figures in the table cannot be set against each other. 545 AI TOPS is a theoretical FP4 rate with sparsity, 191.9 TFLOPS an FP8 rate with sparsity, and neither is a measured speed. The ECC of the Blackwell card's GDDR7 is always on and costs no usable memory. The cards themselves are described in the RTX PRO 2000 Blackwell specs and the RTX 2000 Ada specs, and our comparison of the RTX PRO 2000 and the RTX 2000 Ada covers CAD and rendering. GeForce cards with 16 GB are set beside the Blackwell card in RTX PRO 2000 Blackwell vs RTX 5060 Ti and 5070.

LLM performance of the RTX PRO 2000 and RTX 2000 Ada in published figures

No acceptable measurement in tokens per second exists for the RTX PRO 2000 Blackwell or the RTX 2000 Ada: none from Puget Systems or StorageReview with absolute figures, and no result at MLCommons. We print no number of our own, because we are a supplier and run no test bench.

Three published statements come close. Puget Systems' professional GPU roundup of 18 December 2025, on a Ryzen 9 9950X3D, gives the RTX PRO 2000 a 25 per cent higher token rate than the RTX 2000 Ada in MLPerf Client v1.0; this was Puget's own run and is no result published by MLCommons. NVIDIA's blog of 11 August 2025 states a 2.3 times gain in text generation for the RTX PRO 2000 over the previous generation, without naming the baseline card and without publishing test conditions. And the datasheets give 288 against 224 GB/s of memory bandwidth, 29 per cent more by our arithmetic. That is a ratio of two specifications and no measured rate.

One user or a small team

The remainder in the first table is the planning figure. A 3B model in FP8 leaves about 7 GiB for the KV cache, which all open conversations share. Ministral 3 8B Instruct in FP8 leaves 1.6 GiB; by our reading that is a card for one user with a short context. Our small-model guide linked above counts conversations per model from each model's configuration; it uses the nominal 16 GiB and so a budget of 11.4 GiB.

A seat that answers from company documents needs an embedding model beside the chat model. NVIDIA's Nemotron 3.5 Embed 1B in BF16 is a file of 2.90 GiB; together with Granite 4.2 8b as a Q4_K_M GGUF that is 7.88 GiB and leaves 3.4 GiB, by our arithmetic. We did not read the licence of the embedding model, so check it before use. Sizing for retrieval is the subject of our article on embedding and reranker models for RAG; image models follow their own arithmetic, which ComfyUI on the RTX PRO 2000, 4000 and 4500 sets out.

On the system side, NVIDIA's quick start guide for the Blackwell cards asks for system memory of at least the GPU memory and recommends twice as much, which is 16 GB and 32 GB. CUDA 13.x applications need a driver of at least 580, and the first Linux driver that names the RTX PRO 2000 is 580.76.05, by our reading of NVIDIA's product lists.

The RTX 2000E Ada with 16 GB in a single slot

The NVIDIA RTX 2000E Ada Generation carries the same 16 GB of GDDR6 with ECC in a single slot at 50 W, by NVIDIA's datasheet of October 2025. The budget by our rule is the same 11.3 GiB. NVIDIA's compute capability page does not list the card. Its datasheet gives 8.9 TFLOPS in single precision, against 12.0 for the RTX 2000 Ada and 17 for the RTX PRO 2000, so it has the lowest published rate of the three, and no measurement exists for it either. It is the card for a system with one free slot; see RTX 2000E Ada for edge AI, its specs and the page to buy the RTX 2000E Ada.

When 24 GB is the right card

If the model you need is in the second table, a 16 GB card is the wrong order. The NVIDIA RTX PRO 4000 Blackwell has 24 GB and a budget of 18.5 GiB by the same rule: gpt-oss-20b then leaves 5.7 GiB for context, and Ministral 3 14B Instruct in FP8 fits. The card has a total board power of 145 W, needs a 16-pin cable and takes a full-height single slot. Where the bay is half-height, the RTX PRO 4000 Blackwell SFF Edition brings the same 24 GB at 70 W. The model list is in our article on the RTX PRO 4000 Blackwell for local LLMs, the cards are compared in RTX PRO 2000 vs RTX PRO 4000 Blackwell, and the larger workstation cards are in our overview of the best workstation GPU for AI in 2026.

In the other direction, 8 GB leaves 4.2 GiB by the same rule, which our article on the RTX A1000 and RTX A400 for AI works through.

The pages to buy the RTX PRO 2000 Blackwell, to buy the RTX 2000 Ada and to buy the RTX PRO 4000 Blackwell, with its specs, describe how an order runs. If the model is chosen and the card is not, send us the model, the context length and the number of users, and we name the card that holds them.

FAQ