NVIDIA RTX A1000 and RTX A400 for AI: which local models fit in 8 GB and in 4 GB

What the RTX A1000 and the RTX A400 run for AI

The NVIDIA RTX A1000 with 8 GB has room for speech recognition up to Whisper turbo, for a small embedding model and for language-model files of up to about 4 GiB; by our sizing rule it has 4.2 GiB for model weights plus KV cache. The NVIDIA RTX A400 with 4 GB has 0.6 GiB by the same rule. Its 4 GB hold Whisper small by OpenAI's figure and, by our rule, leave no room for a language model with context; NVIDIA itself describes the card's use as "light AI workloads". By the same rule a local LLM for a team starts at 16 GB.

The question comes up where compact workstations already carry one of the two cards, or where such machines are about to be ordered and a seat should also transcribe meetings, search documents or run a small assistant without sending data to a cloud service. Both cards are single-slot boards of 2.7 in height with 50 W of total board power, and both have third-generation Tensor Cores, 72 on the RTX A1000 and 24 on the RTX A400. On both, memory sets the limit first.

The memory budget of 4.2 GiB on 8 GB and 0.6 GiB on 4 GB

Our sizing rule for language models sets memory aside before it counts the model: 0.9 times the memory the driver reports, less 3 GiB. What remains has to hold the weights and the KV cache, the memory a model keeps for every token of context.

memory and Tensor figures from NVIDIA's datasheets for the RTX A1000 (April 2024) and the RTX A400 (3279761, May 2024), the Tensor figures being effective FP16 TFLOPS with sparsity; budgets are our arithmetic.

Two things have to be said about these numbers. The memory the driver reports is scaled here from the 95.6 GiB it shows on the 96 GB card; that is an assumption, NVIDIA publishes no such figure for either card, and nvidia-smi on the delivered card decides. A file inside the budget fits, and the remainder is its room for context. A file above the budget and below 0.9 times the memory is what we call tight: we do not plan on it. A file above that limit does not fit.

The rule was written for a serving engine such as vLLM with several users. For one user with a GGUF file in llama.cpp or Ollama it is conservative, and no published measurement of the real overhead on these two cards exists, so this page states the rule and the file size and nothing beyond them. Speech and image models are counted differently, as their sections explain.

RTX A1000 local LLM and the model files that fit

file sizes from the publishers' own repositories on Hugging Face, read on 10 November 2026; fit classes by our sizing rule, which is arithmetic and no measurement.

Two language-model files lie inside the budget of the RTX A1000. Granite 4.2 3b in FP8 leaves 0.3 GiB for the KV cache; how many tokens of context that holds depends on the model's cache per token, which we have not sized. The FP8 format has a caveat of its own on this card, described further down. The files in the tight rows use part of the 3 GiB the rule keeps back.

The name of a repository does not give the size of its file. NVIDIA's repository "Nemotron 3.5 Bolt E4B-4GB" holds a Q4_K_M file of 6.18 GB, which is 5.76 GiB: tight on the 8 GB card and too large for a 4 GB card. The Gemma 4 checkpoints E2B and E4B hold about 5.1 and 8.0 billion parameters, and their BF16 files of 9.54 and 14.89 GiB fit neither card. OpenAI publishes gpt-oss-20b in MXFP4 only, at 12.82 GiB.

On licences: Granite 4.2, Ministral 3, Gemma 4 and gpt-oss-20b are tagged Apache 2.0 on Hugging Face and phi-4 is tagged MIT, as read on 10 November 2026. We did not read the licence of the Qwen3.5 files on that date, and Nemotron 3.5 Bolt comes under OpenMDW-1.1, whose terms we have not read; check both before commercial use. Our guide to small language models on the L4 and the RTX PRO 2000 sizes the same model families for servers.

RTX A400 local LLM with a budget of 0.6 GiB

No language-model file we sized lies inside 0.6 GiB. Two files are tight on the RTX A400, Qwen3.5-0.8B in BF16 and the embedding model Nemotron 3.5 Embed 1B at 2.90 GiB: both are above our budget and stay below 0.9 times the memory, about 3.6 GiB, and we do not plan on them.

NVIDIA's own wording for the card is "light AI workloads". Its blog of 16 April 2024 speaks of chatbots and copilots on the desktop and gives no model size and no speed. The documented uses are AI functions inside applications: NVIDIA's product page names AI denoising and DLSS, and the AI chart in the datasheet uses Super Resolution in Adobe Photoshop, measured against the T400 4GB. No review that we could use covers the RTX A400, so no tokens per second can be quoted for it.

Whisper on 8 GB and on 4 GB

README of OpenAI's Whisper repository, table "Available models and languages", read on 10 November 2026.

OpenAI's figures are total VRAM for its reference implementation, so they are compared with the card's memory directly and our language-model rule does not apply. The README states neither precision nor batch size, and other runtimes use other amounts for which we have no primary figure. Whisper's code and weights are under the MIT licence. The turbo model is an optimised version of large-v3 that was not trained for translation. OpenAI measured its relative speeds on an A100, which says nothing about these two cards.

Whisper turbo leaves about 2 GB of an RTX A1000 free, so we do not plan transcription and a language model on the same card. NVIDIA's own speech models Parakeet TDT 0.6B v3 and Canary 1B v2, both under CC-BY-4.0 and both for 25 European languages, cannot be sized on paper: their model cards ask for at least 2 GB and 6 GB of system RAM to load and give no GPU memory figure. Our guide to Whisper GPU requirements covers the server side.

You can order the card or a complete workstation alone, the setup of Whisper and a small language model alone, or both together: the workstation delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

Embedding models and document search on 8 GB

An embedding model turns text into vectors for a search index. Nemotron 3.5 Embed 1B in BF16 is a 2.90 GiB file: it fits the RTX A1000 with about 1.3 GiB left and is tight on the RTX A400. We did not read its licence on 10 November 2026. The 8B embedding and reranking models of the same family are 15.61 GiB each in BF16 and by our rule need a 24 GB card.

Search with generated answers needs two models at once. The embedding model and the smallest language file together come to 4.53 GiB, above the 4.2 GiB of the RTX A1000 by our arithmetic. An index can be built on the card; answering from it with a language model on the same card is outside the budget. Our article on embedding and reranker GPU requirements goes further.

Stable Diffusion and other image models on the RTX A1000

For image models we compare file sizes with 0.9 times the card memory, about 7.2 GiB on the RTX A1000. This is arithmetic as well, and it says nothing about speed.

• SDXL base 1.0, fp16 parts. The model is 4.78 GiB and the whole pipeline 6.46 GiB, so model, text encoders and VAE fit together. The OpenRAIL++ licence permits commercial use within its use restrictions.
• Stable Diffusion 3.5 Medium. The model file of 4.76 GiB fits; the pipeline with the T5 text encoder in FP8 is 10.84 GiB, so the text encoder has to be unloaded or kept in system RAM. The Stability AI Community License sets conditions that we have not read.
• FLUX.2 klein 4B, Apache 2.0. The model file of 7.22 GiB alone exceeds the limit, so it runs only with partial loading from system RAM, which is slower.

We computed no row for 4 GB. The README of ComfyUI, read on 10 November 2026, names 4 GB of VRAM with 8 GB of system RAM as the low end it handles through model offloading; nothing measured exists for the RTX A400. NVIDIA's support matrix for NIM for Visual Generative AI of 20 October 2026 sets 16 GB as the minimum for FLUX.1-dev and Stable Diffusion 3.5 Large and lists neither card among its pre-built engines. The larger cards are the subject of our article on ComfyUI on the RTX PRO 2000, 4000 and 4500.

FP16, FP8 and FP4 on these two cards

NVIDIA publishes Tensor performance for these cards in FP16 and INT8: 53.8 TFLOPS and 107.8 TOPS for the RTX A1000, 21.7 TFLOPS and 43.3 TOPS for the RTX A400, all with sparsity. In FP16 the RTX A1000 has 2.5 times the figure of the RTX A400. NVIDIA mentions neither FP8 nor FP4 for either card, and its "CUDA GPU Compute Capability" page of 16 October 2026 lists neither.

That matters for the format of a model file. The same NIM support matrix ties bf16 to compute capability 8.0 and fp8 to 8.9, the level of the Ada generation. An FP8 file such as Granite 4.2 3b fits the RTX A1000 by size; whether a given runtime computes it on this card is a question for the runtime's documentation before the order. Files in NVFP4 compute natively only on Blackwell cards.

RTX A1000 AI performance in published figures

NVIDIA's product page states up to 3.6 times the generative AI performance of the previous generation. The test note names the T1000 8GB as that generation, image generation at 512 × 512 with Stable Diffusion webUI v1.7.0 and TensorRT, a Core i9-12900K and a pre-release build. NVIDIA's datasheet for the T1000 8GB lists no Tensor Cores, so the multiple compares a card with Tensor Cores against one without.

Independent figures exist for one system. StorageReview's review of the Lenovo ThinkStation P3 Tiny Gen 2 (20 November 2025, modified 14 August 2026; Core Ultra 9 285) reports for the RTX A1000 in UL Procyon's AI text generation test 39.379 tokens per second with the Phi model, 27.734 with Mistral and 24.323 with Llama3, and no result with Llama2. In the image test it reports 11.196 s per image with Stable Diffusion 1.5 in FP16 and no SDXL result. These are the rates of one benchmark with its own models in one system; they do not say how many people a card serves.

Where the step to 16 GB starts

Four cases lead past 8 GB: a language model above about 4 GiB with room for context, Whisper large-v3, transcription and a language model at the same time, and document search with generated answers. A 16 GB card has a budget of 11.3 GiB by the same rule. Qwen3.5-4B in BF16 (8.68 GiB) and Ministral 3 8B Instruct in FP8 (9.70 GiB) lie inside it; gpt-oss-20b at 12.82 GiB is above our budget even there, and we do not plan on it.

Three 16 GB cards have the same height of 2.7 in. The NVIDIA RTX 2000E Ada Generation is the single-slot card at 50 W; at 6.6 in it is 0.2 in longer than the two small cards, which is the one dimension to check in the chassis: RTX 2000E Ada specs, buy the RTX 2000E Ada, and our article on the RTX 2000E Ada for edge AI. The NVIDIA RTX 2000 Ada Generation and the NVIDIA RTX PRO 2000 Blackwell take two slots and 70 W; NVIDIA lists compute capability 8.9 for the first, the level its matrix asks for FP8, and 12.0 for the second, whose Tensor Cores add FP4. See the RTX 2000 Ada specs and the page to buy the RTX 2000 Ada, the RTX PRO 2000 Blackwell specs and the page to buy the RTX PRO 2000 Blackwell. Which models fit is the subject of local LLMs on 16 GB cards, and the hardware step is compared in RTX A1000 vs RTX 2000 Ada vs RTX PRO 2000.

For the two small cards themselves, see the RTX A1000 specs and the page to buy the RTX A1000, the RTX A400 specs and the page to buy the RTX A400, and the comparison RTX A1000 vs RTX A400. If you are unsure whether 8 GB is enough, send us the model files or the tasks you have in mind and we answer with the card that holds them.

FAQ