How many GPUs fit in one server: PCIe lanes, power and airflow decide

PCIe lanes, power and airflow: three limits in order

A 2U server holds up to four double-width GPUs, usually two if each card draws 600 W; 3U and 4U servers hold eight, and some 4U models ten, most of them behind PCIe switches. Which count a given server reaches is set by three limits, always in the same order: the PCIe lanes the processors provide, the power the chassis can deliver and dissipate, and the air the rack can move. Vendors design around all three, and their specification sheets state each limit.

Limit one: PCIe lanes

A data-centre GPU is a PCIe device designed for a full x16 link. On Gen5 that link carries about 64 GB/s in each direction; Lenovo quotes 63 GB/s for its x16 slots. NVIDIA’s certified-systems configuration guide (the 2023 PDF revision) is explicit: one Gen5 or Gen4 x16 link per GPU is strongly recommended, x8 is supported but performance may suffer, and at most two GPUs should share one upstream x16 link. The current web version (30 September 2026) asks for Gen5 x16 or better for the RTX PRO 6000 and H200 NVL and Gen4 x16 or better for the L40S.

Sources: AMD EPYC 9005 datasheet and 9004 tuning guide, Intel ARK and Xeon 6 product brief.

In a two-socket EPYC system, each processor uses 64 of its 128 lanes for the socket-to-socket Infinity Fabric links, which leaves 64 for devices; with three fabric links instead of four, each socket keeps 80, hence “up to 160”. The same lanes also serve the NVMe drives, the network cards and the management controller, so fewer are left for GPUs than the table shows.

Once the lanes run out, the vendor adds a PCIe switch. The Dell PowerEdge XE7745 uses four Broadcom Atlas II switches to feed eight double-width slots from two CPUs; the Supermicro 4U line runs a dual-root switch fabric behind thirteen x16 slots; one 2U design fits sixteen single-slot cards behind two switches. NVIDIA’s 2023 guide allows one or two switch layers, and its current guide recommends placing network cards and NVMe on the same switch or root complex as the GPUs they feed. For inference a switch costs little, because each card reads from its own memory rather than over the bus. The switch’s uplink, which all cards behind it share, matters when cards without NVLink exchange tensors across switches or with the processors. Cards under one switch exchange peer-to-peer traffic through it at full x16, as long as ACS on the switch or IOMMU translation does not redirect that traffic to the CPU root complex.

GPUs per chassis: 1U, 2U, 3U and 4U servers compared

DW = double-width card, SW = single-width. Sources: vendor specification sheets and product guides, September 2026. Configurations change with firmware and CPU generation; the vendor’s current guide takes precedence.

In 2U, up to four double-width cards connect directly to the CPUs; six to ten cards need 3U or 4U and switches. The same 2U accepts four 350 W cards but only two 600 W ones: Lenovo allows four RTX PRO 6000 in the SR650a V4 only in a version capped at 450 W. For four cards at the full 600 W, check that the maker lists 600 W per slot, or plan a 3U or 4U chassis. The card’s power rating, rather than its length, is what most often decides how many cards fit.

We build AI servers to order and check the platform, power and cooling of a server you already own. Tell us its model and the cards you want in it; we say in advance if something will not work.

Limit two: power

Eight cards at 600 W draw 4.8 kW before the two processors, at 350 to 500 W each, the fans, the memory and the network cards. Dell’s own budget for the XE7745 lists 2,160 W for the GPU-zone fans alone. By our arithmetic from that budget, the cards, the GPU fans, eight network cards and the baseboard come to 8.2 kW before the processors, so a fully loaded eight-GPU PCIe server can approach ten kilowatts at the wall. That is why every vendor in the table fits four to eight power supplies of 2,000 to 3,200 W, which deliver their full rating only on 200 to 240 V. Dell’s 3,200 W unit derates to 2,900 W at 200 to 220 V. A low-line mode, where one exists, does not help with GPUs: Dell’s 2,400 W unit falls to 1,400 W on 100 to 120 V, and with eight 600 W cards Dell supports even the full-rated 2,400 W unit only with all eight bays filled and without A/B grid redundancy.

At the rack, the power supplies take IEC C20 inlets rated at 16 A, fed from C19 outlets; a 16 A single-phase PDU delivers 3.7 kW in total, however many C19 outlets it has. One eight-GPU server exceeds a 16 A feed on its own. This is the item that most often stops a delivered server from being switched on: the rack position has a 16 A feed, and the electrical work has to be scheduled after delivery. For eight cards, plan three-phase feeds, about 11 kW at 16 A and 22 kW at 32 A on 400 V, with each of the A and B feeds able to carry the whole server, and plan them before the order is placed.

The card itself needs the right cable. Current 600 W cards use the 12V-2×6 connector from the PCIe CEM 5.1 specification, with sense pins that tell the card whether it may draw 150, 300, 450 or 600 W. Dell documents a specific cable per riser for L40S and H100-class cards and limits some 2U configurations to two double-width cards with power cables. An RTX PRO 6000 Server Edition on a 450 W cable runs at 450 W and reports that limit, with a quarter less power budget; an H200 NVL, which supports only the 600 W level, does not boot on such a cable at all.

Limit three: air

The usual planning figure for air is about 160 CFM per kilowatt at an 11 °C temperature rise across the server, or roughly 90 CFM per kilowatt if the exhaust may run 20 °C above the inlet. A 7 kW server therefore moves in the region of 600 to 1,100 CFM, which is why the GPU-zone fans have their own 2 kW budget.

Inlet temperature limits differ by vendor and configuration. ASHRAE class A2, which Dell specifies for the XE7745, allows 10 to 35 °C, but Dell’s thermal matrix limits that server to 30 °C with the H200 NVL or RTX PRO 6000 (35 °C with the L40S or L4). Supermicro quotes 10 to 35 °C for its 4U systems. Lenovo tells SR675 V3 owners to keep ambient at 30 °C or below with L40S or H100 PCIe cards, and at 25 °C with one high-frequency EPYC option. By our arithmetic, a passive 600 W card needs roughly 70 per cent more airflow than a 350 W L40S at the same inlet temperature and temperature rise, and every degree of inlet temperature costs fan power. In a warm hall the server still boots, but its fans run faster and draw more power, and the GPUs lower their clocks.

NVLink bridges and models split across several GPUs

Among the PCIe cards in this class, the Hopper cards connect to their neighbours through NVLink bridges: the H100 PCIe and H100 NVL two-way at 600 GB/s, the H200 NVL two- or four-way at 900 GB/s per GPU. The RTX PRO 6000 Server Edition and the L40S have no NVLink; every byte between cards crosses PCIe, at about 64 GB/s per direction on the RTX PRO 6000’s Gen5 x16 link and about 32 GB/s on the L40S, a Gen4 card. For inference of one model on one card this does not matter. For a model split across four cards it is the main limit, which is why the four-card H200 NVL configuration exists as a product. Four RTX PRO 6000 cards run either one model copy per card or a larger model split over PCIe, where vLLM’s documentation suggests pipeline parallelism for GPUs without NVLink; our article on splitting one model over PCIe or NVLink covers the traffic and the peer-to-peer settings.

How we size a multi-GPU server

Our engineering partner Vixen.UNO starts from the model, the precision and the number of concurrent users, which give the GPU count, and then chooses the chassis: direct-attached for up to four cards, with PCIe switches for more. Next come the power at the rack, in amps and phases rather than kilowatts, and the inlet temperature the hall can hold. We ask about these two first, because they depend on the building rather than on the order. We then build the server to order, assembled and tested before delivery, with manufacturer warranty on one EU contract and invoice.

FAQ