NVIDIA RTX 5000 Ada and RTX 4500 Ada for AI: which models fit in 32 GB and 24 GB

What the RTX 5000 Ada and RTX 4500 Ada offer for AI work

For AI work the NVIDIA RTX 5000 Ada Generation offers 32 GB and the NVIDIA RTX 4500 Ada Generation 24 GB of GDDR6 with ECC. By our sizing rule that leaves 25.7 GiB and 18.5 GiB for a language model and its context. Both cards compute in FP8 and BF16 and have no FP4 acceleration in hardware. In 24 GB that covers, for example, Ministral 3 14B Instruct in FP8; 32 GB adds Gemma 4 12B-it in BF16 and Devstral Small 2 24B Instruct in FP8.

Two kinds of company ask this question. One already has these cards in its CAD or rendering workstations and wants to know whether they are enough for a local assistant, document search, image generation or speech to text. The other is deciding in late 2026 whether an Ada card is still the right order for a new AI seat. The answers below rest on NVIDIA's documents, on file sizes read from Hugging Face on 10 November 2026 and on the few published measurements that exist.

The two cards in the figures that count for AI

NVIDIA datasheets 2898211 (August 2023) and 2895216 (October 2023); NVIDIA's CUDA GPU Compute Capability page of 16 October 2026; NVIDIA's vGPU supported-GPU list of 2 October 2026. The budget row is our arithmetic.

We plan with the budget. Our rule takes 0.9 times the memory the driver reports and subtracts 3 GiB; what remains holds the weights and the KV cache, which is the memory a model needs for its context. We scale the reported memory from the 95.6 GiB the driver shows on the 96 GB card. That scaling is an assumption, and the driver on the delivered card decides. The rule was written for a vLLM server with several users; for one user with a GGUF file it is conservative.

The Tensor row answers the search for an AI TOPS figure. NVIDIA's datasheets state 1044.4 and 634.0 TFLOPS as effective FP8 rates with sparsity, and NVIDIA's line card of March 2025 prints the same values, 1,044 and 634, as AI TOPS. They are peak figures on paper. They cannot be set against the AI TOPS of a Blackwell card, which NVIDIA states as FP4 figures with sparsity.

FP8 without FP4 and which file of a model to take

Many models are published in several number formats, and a card accelerates only some of them. NVIDIA's CUDA page lists the RTX 5000 Ada and the RTX 4500 Ada at compute capability 8.9. In NVIDIA's NIM for Visual Generative AI support matrix of 20 October 2026, FP8 requires compute capability 8.9 and NVFP4 requires 10.0, and the Ada-generation GPUs it lists receive FP8 engines only. On an Ada card the file to take is therefore the FP8 or the BF16 one.

Files in NVFP4 compute natively only on the Blackwell cards. That matters where a publisher offers nothing else: gpt-oss-20b exists only as an MXFP4 file. It fits both Ada cards by size, but the cards do not accelerate FP4 arithmetic in hardware. What a serving engine does with such a file on a GPU without FP4 Tensor Cores is the subject of our article on FP4 models on H200 NVL, L40S and Ada cards.

Four-bit GGUF files are a different format from NVFP4 and MXFP4, and the publishers' own files of this kind are small. Ministral 3 14B Instruct as Q4_K_M with its projector file is 7.67 GiB, and Gemma 4 26B-A4B-it as QAT Q4_0 with its projector file is 13.45 GiB, so a model with about 26 billion parameters fits the budget of the 24 GB card.

Which language models fit in 24 GB and in 32 GB

sizes of the publishers' own files on Hugging Face, read 10 November 2026, in GiB; budgets and remainders are our arithmetic and no measurement.

The remainder is the room for the KV cache, that is for the length of the context and for requests that run in parallel. A file that leaves 0.5 GiB, as Qwen3.5-9B in BF16 does on the 24 GB card, is inside the budget and has next to no room for context. For a team that shares one model, choose a file that leaves several GiB.

"Tight" marks files that are above our budget and within 0.9 times the memory the driver reports, which is 28.7 GiB on the 32 GB card by our scaling. Ministral 3 14B in BF16 (25.97 GiB) and Granite 4.2 30b in FP8 (28.03 GiB) are in that band: above our budget, we do not plan on them. Qwen3.8-27B in FP8 (28.75 GiB) is above our budget too, and we do not plan on it for the 32 GB card.

Read the size from the file and not from the name. The checkpoint of Gemma 4 E4B-it holds about 8.0 billion parameters and is 14.89 GiB in BF16. On licences, the repositories read on 10 November 2026 carry Apache 2.0 for gpt-oss-20b, Ministral 3, Gemma 4, Granite 4.2 and Qwen3.8-27B in FP8. We did not read the licences of Qwen3.5-9B, Devstral Small 2 and Qwen3.6-35B-A3B, so check them before commercial use.

You can order the cards alone, the setup of the inference software and the models alone, or both together: a workstation with the card, delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

Image generation and speech to text on 24 and 32 GB

file sizes on Hugging Face, read 10 November 2026, set against 0.9 times the card memory; our arithmetic, nothing measured, no speeds.

"All" means that the diffusion model, the text encoders and the VAE fit in card memory together. "Model" means that the model file fits and the text encoder has to be unloaded or kept in system memory. "Offload" means that the model file alone exceeds 0.9 times the card memory and runs only with partial loading from system memory, which is slower. The NVFP4 files of the FLUX models compute natively only on Blackwell cards; on an Ada card take the FP8 or BF16 file.

NVIDIA's own guidance, in the NIM support matrix of 20 October 2026, is 16 GB minimum and 32 GB recommended for FLUX.1-dev and for SD 3.5 Large, 48 GB for FLUX.2 klein and 80 GB for Qwen-Image. Neither Ada card is in its list of GPUs with pre-built engines. How such models are run in ComfyUI is described, for the Blackwell cards, in ComfyUI on RTX PRO 2000, 4000 and 4500.

Speech to text needs little memory. OpenAI states about 10 GB of VRAM for Whisper large-v3 and about 6 GB for the turbo model in the README of its reference implementation, under the MIT licence, so either card holds either model.

Published measurements of the RTX 5000 Ada and RTX 4500 Ada for AI

StorageReview's review of the Lenovo ThinkStation P3 Tiny Gen 2, dated 20 November 2025 and modified on 14 August 2026, includes a ThinkStation P3 Tower Gen 2 with a Core Ultra 9 285 processor and an RTX 5000 Ada. In UL Procyon's AI text generation test that system reached 160.757 tokens per second with Phi, 117.784 with Mistral, 96.565 with Llama3 and 57.623 with Llama2. In the image generation test it needed 1.674 seconds per image with Stable Diffusion 1.5 in FP16 and 11.386 seconds with SDXL in FP16. These are results of a whole system under Windows 11 Pro, and the review does not state the driver.

Puget Systems tested both cards in its article on LLM inference with professional cards of 22 August 2024, updated on 4 October 2024, with llama.cpp and a 4-bit Phi-3-mini model. Its token rates are in chart images. The text says that the RTX 5000 Ada reached about 90 per cent of the token generation rate of the RTX A6000, a 48 GB card of the Ampere generation with 768 GB/s. For the RTX 4500 Ada we can quote no token rate from any source we use.

MLCommons has published no result for either card. The SPEC and Blender results that exist for them concern CAD and rendering and are in our RTX 5000 Ada specs and RTX 4500 Ada specs.

More than one card, vGPU and MIG

NVIDIA's datasheets state "No" for NVLink on both cards, and NVIDIA's MIG guide of 4 November 2026 lists no Ada card. Our reading: no bridge between two of these cards and no isolated instances on one.

For sharing through virtual machines NVIDIA lists one of the two. NVIDIA's vGPU list of 2 October 2026 includes the RTX 5000 Ada from release 16.1, with the display ports turned off when vGPU software is used, and does not include the RTX 4500 Ada. In a fleet that also holds Blackwell cards one current driver runs both generations, as our article on mixing Ada and Blackwell GPUs explains.

When the Blackwell card with the same memory is the better order

NVIDIA datasheets 2895216 (October 2023), 5323450 (June 2026), 2898211 (August 2023) and 5108623 (April 2026).

Memory size sets the budget, so each Blackwell card loads the same files as the Ada card beside it. What changes is how the files are read and computed. Both Blackwell cards have 56 per cent more memory bandwidth by our arithmetic, at a lower board power. Their Tensor Cores add FP4, so NVFP4 files compute natively. For a new AI seat these points speak for the NVIDIA RTX PRO 4000 Blackwell at 24 GB and for the NVIDIA RTX PRO 4500 Blackwell Workstation Edition at 32 GB.

One published figure supports this, with a caveat. In Puget Systems' own runs of MLPerf Client v1.0 for its roundup of 18 December 2025, which are no results published by MLCommons, the Blackwell cards were on average about 50 per cent ahead of the Ada card of the same number in token rate, and 25 per cent for the 2000 pair. Those pairs go by number, the RTX PRO 4500 Workstation Edition against the RTX 4500 Ada for instance, and not by memory as in the table. No token rate that we can quote exists for the RTX PRO 4000 or for the RTX PRO 4500 Workstation Edition.

The Ada card remains the right order in three cases: the cards are already in the workstations and the models fit; a fleet has a validated image and an FP8 software stack that should not change; or the 32 GB card has to serve virtual machines through vGPU, and the RTX PRO 4500 Workstation Edition is not on NVIDIA's vGPU list.

The orders are described on our pages to buy the RTX 5000 Ada, buy the RTX 4500 Ada, buy the RTX PRO 4000 Blackwell (specs) and buy the RTX PRO 4500 Blackwell (32 GB spec page). The pairs are compared in RTX 4500 Ada vs RTX PRO 4500 and 4000 Blackwell and RTX 5000 Ada vs RTX PRO 4500 Blackwell, and the 24 GB Blackwell card has its own guide, local LLMs on the RTX PRO 4000 Blackwell. Models above 32 GB lead to the card in RTX PRO 5000 Blackwell vs RTX 5000 Ada, and smaller ones to local LLMs on 16 GB cards.

If you already know the models, send us their names and the number of people who will use them at once, and we answer with the card and the file format we would plan with.

FAQ