How much RAM a GPU server needs: system memory for 2, 4 and 8 GPUs

The published rules for host RAM

A GPU server needs system memory of at least twice its total GPU memory by NVIDIA's guide for NVIDIA-Certified Systems: 384 GB for two NVIDIA RTX PRO 6000 Blackwell Server Edition cards, 768 GB for four and 1,536 GB for eight, and 2,256 GB for eight NVIDIA H200 NVL. NVIDIA's reference architectures set a lower floor of 128 GB per GPU, and Cisco asks for at least three times the HBM in its H200 NVL servers. The memory should be spread evenly over every socket and memory channel, which in practice decides the module size.

These sources contradict each other less than they seem to. The certified-systems guide states a ratio to GPU memory; the reference architectures state a minimum per GPU that applies whatever the card's memory size; Cisco states a recommended minimum per configuration for its own server. HPE repeats the 2× value as a recommendation. All of them describe inference and training servers in general, not one workload.

NVIDIA-Certified Systems Configuration Guide, last updated 30 September 2026; NVIDIA Enterprise Reference Architecture for RTX PRO AI Factory, components, last updated 18 May 2026; NVIDIA AI Enterprise reference architecture, compute node hardware (undated); Cisco UCS C845A M8 spec sheet Rev. A.11 (11 May 2026); HPE ProLiant Compute DL380a Gen12 QuickSpecs. All read on 24 October 2026.

The certified-systems guide adds: "Using all memory slots can increase the bandwidth." Its other host rules apply to the same servers: at least two sockets, a 2.0 GHz base clock and six physical cores per GPU. The reference architectures ask for seven cores per GPU, two more per GPU for the operating system or virtualisation and two per MIG instance. Our guide to CPU for a GPU server covers the cores; this page covers memory.

System memory for 2, 4 and 8 cards

Applied to the cards we supply, the rules give these values. The H200 NVL has 141 GB of HBM3e, the RTX PRO 6000 Server Edition 96 GB of GDDR7 and the RTX PRO 4500 Server Edition 32 GB.

Our arithmetic from the rules above; Cisco values from table 8 of the C845A M8 spec sheet, which lists the RTX PRO 4500 without a memory value.

Two patterns stand out. For the H200 NVL with its 141 GB per card, twice the GPU memory is well above 128 GB per GPU, and Cisco's three times is above both. For the RTX PRO 4500 Server Edition with 32 GB per card, it is the other way round: 128 GB per GPU is four times the GPU memory, so the reference architecture's floor is the higher value. For the RTX PRO 6000 Server Edition, Cisco's table matches the 2× rule up to four cards and asks for 2,048 GB with eight.

In a quote, the column to start from is the 2× rule, because it is the one NVIDIA uses for certified systems and HPE repeats in its QuickSpecs. The 128 GB floor matters for cards with little memory, and Cisco's higher values for the H200 NVL are its recommendation for its own server. We size the memory of each server we build against these values and the workload; tell us the cards and the workloads you plan, and our team replies with the memory size and population for your configuration.

What the system memory holds

GPU memory holds what the cards compute with: weights, KV cache and activations. System memory holds the rest of the service. On an LLM server that is the operating system, the serving engine's processes, the model files on their way to the cards and the Linux page cache in which they stay after loading, so a restart or a second copy on another card reads them from RAM rather than from disk. Our guide to CPU, RAM and storage for a 4-GPU server explains how vLLM uses that page cache and which settings place weights or KV cache in system memory on purpose.

Three workloads push memory above the 2× rule:

• KV cache offloading. Conversations that return after a pause find their cache in RAM instead of recomputing it. Our guide to KV cache offloading to CPU memory and NVMe sizes the buffer per user; keeping 500 conversations of an assistant warm takes about 500 GiB on top of the base, which brings a four-card server to about 1 TB.
• Fine-tuning with optimiser offload. DeepSpeed's ZeRO-Offload moves optimiser states and gradients into system memory, 14 bytes per parameter in its paper, about 990 GB for a full fine-tune of Llama 3.3 70B by the arithmetic in our CPU and RAM guide. A LoRA run needs a fraction of that, as GPU memory for fine-tuning shows.
• RAG and virtual machines. A vector database, the document parser and the gateway run on the host and use its memory, and every virtual machine with a vGPU or MIG instance has its own RAM allocation.

Filling the memory channels: EPYC 9005 and Xeon 6

The memory has to be spread evenly over every channel, so the capacity comes in steps set by the processor's channel count and the module sizes the server maker lists. More channels in use also means more memory bandwidth, which is why NVIDIA's RTX PRO reference architecture asks for at least one DIMM in every channel. By our arithmetic, one DDR5 channel at 6400 MT/s moves 51.2 GB/s, 6,400 million transfers of 8 bytes, so twelve channels give about 614 GB/s per socket and six channels half that, about 307 GB/s. A PCIe Gen5 x16 link carries about 64 GB/s in each direction, and eight cards loading weights or exchanging offloaded KV cache at the same time can ask for about 512 GB/s from host memory. An empty channel is bandwidth the cards may need at exactly those moments.

AMD EPYC 9135, 9355 and 9555 product pages (12 channels per socket, DDR5 up to 6400 MT/s); Dell PowerEdge XE7745 Technical Guide Rev. A09 of September 2026 (24 slots, 1DPC, RDIMMs of 32 to 256 GB at 6400 MT/s, up to 6 TB) and XE7740 Technical Guide Rev. A04 (Xeon 6, 8 channels and 2 DIMMs per channel per processor, 6400 MT/s at 1DPC and 5200 MT/s at 2DPC, up to 4 TB); HPE DL380a Gen12 QuickSpecs (same speeds, 32 slots); Intel's specifications for the Xeon 6960P (12 channels) and Kingston's Xeon 6 platform guide (November 2025) for one DIMM per channel on the Xeon 6900P. Capacities are our arithmetic with 32, 64, 96 and 128 GB modules.

The table explains several common configurations. Two EPYC 9005 with 24 modules of 32 GB give 768 GB, exactly twice the memory of four RTX PRO 6000 Server Edition. With 96 GB modules they give 2,304 GB, just above twice the 1,128 GB of eight H200 NVL. Two Xeon 6700P at one module per channel take 16 modules, so the step after 512 GB is 1,024 GB; StorageReview's test system with four RTX PRO 6000 Server Edition in an HPE ProLiant Compute DL380a Gen12 had exactly that, 16 modules of 64 GB. For eight H200 NVL on Xeon 6700P, 16 modules of 128 GB give 2,048 GB, below the 2× value, so the choice is a second module per channel at 5200 MT/s or a different platform.

The board design matters as much as the processor. Cisco's C845A M8 with AMD EPYC 9005 processors lists eight channels per processor with up to two DIMMs each, 32 slots, at up to 5200 MT/s with one DIMM per channel and 4400 MT/s with two. Check the slot count and the speed table in the maker's guide for the exact server, and ask for the population, not only the total, in a quote.

What servers and clouds ship with

Published builds show where the industry settles in practice. The ratio column is our division of host memory by GPU memory.

Google Cloud blog, 20 October 2025 (g4-standard-96, -192, -384); CoreWeave instance documentation; StorageReview review of the HPE system, 6 November 2025, modified 14 October 2026; Helix engineering blog, 14 August 2026; OVHcloud guide to its 8 × H200 NVL instance, updated 1 October 2026; Lambda Scalar datasheet, March 2025. Read for our research notes on 24 October 2026.

Most builds sit between 1.7 and 2.7 times the GPU memory. Google's G4 shapes scale memory with the card count at 180 GB per GPU, and Lambda's H200 NVL server matches the 2× rule. Helix ran DeepSeek-V4-Flash on eight Server Edition cards with 256 GB of host memory, which shows that serving one model works far below the rule. It also leaves no room for a page cache of several models, a KV offload tier or a RAG index next to the engine, which is what the extra memory in the other builds is for.

Our reference configurations

Our reference configurations on AI servers follow NVIDIA's certified-systems guide: at least twice the GPU memory, at least six cores per GPU and two sockets.

Our AI servers page; ratio by our arithmetic. All three with two AMD EPYC processors; 16 GB DDR5 RDIMMs as listed in Puget Systems' E282-4U configurator.

On two EPYC 9005 processors these capacities fill every channel with one module: 24 modules of 16, 32 and 96 GB. The training node at 2,304 GB sits between the 2,256 GB of the 2× rule and Cisco's 4,096 GB for the same eight cards; where fine-tuning with optimiser offload is planned, we size it upwards. For servers with two or four H200 NVL we quote 768 GB and 1,152 GB, the first even populations of 24 channels above twice their GPU memory (24 modules of 32 GB, or of 48 GB where the maker lists that size). We supply these servers with more memory on request; send us your models, number of users and any fine-tuning plans, and our team replies with the memory population and a configuration.

When to go above or below the rule

More memory than twice the GPU memory pays off when the host does more than feed the cards. A four-card inference server with KV offloading for many returning users goes to about 1 TB. A training server for full fine-tunes with ZeRO-Offload needs about 1 TB of optimiser and gradient state for a 70B model in addition to the base. A server that runs virtual machines with vGPU or MIG instances needs each VM's memory on top. In each case, the next step comes from the channel table: on two EPYC 9005, 1,536 GB after 768 GB with 64 GB modules, or 1,152 GB with 48 GB modules if the maker lists them.

Less than twice the GPU memory works for a server that runs one model with one engine on bare metal, as the Helix build shows, and the reference architectures' 128 GB per GPU covers such a server. The cost is headroom: a second model, a larger page cache or an offload tier then needs more memory. On a board with one slot per channel, such as Dell's XE7745, that means replacing every module; on boards with two slots per channel it means a second module in each channel at the lower speed. We size memory for the planned workload and the growth step the server is meant to take, as our guide to adding GPUs to a server later explains for cards. For a four-card build the full breakdown of cores, memory and NVMe is in our CPU, RAM and storage guide, and NVMe storage for an LLM server covers the drives.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with two, four or eight RTX PRO 6000 Server Edition or H200 NVL cards, or RTX PRO 4500 Server Edition, on AMD EPYC or Intel Xeon processors with DDR5 ECC memory sized to the GPU pool and populated evenly across the channels. Each server is assembled and tested before delivery, and every component carries the manufacturer warranty on one EU contract and invoice.

FAQ