Liquid cooling for 50 to 120 kW GPU racks: what changes, what it demands, when there is no way round it

Where air stops

The published limits agree more than the marketing suggests. Uptime Institute puts perimeter air cooling at 20 to 25 kW per rack and close-coupled air, meaning in-row units with containment, at up to 50 kW. Conventional design guidance puts 25 to 30 kW as the ceiling for ordinary air; Vertiv and Schneider both draw the practical line around 20 kW. ASHRAE’s own paper gives the physical reason: a good raised floor delivers about 1,900 CFM per tile, and a 40 to 50 kW rack can need 5,000.

GPU racks reach that boundary in one purchase. Four eight-GPU air-cooled servers sit just under 50 kW; the 64-GPU racks Uptime describes run 80 to 90 kW, with fans consuming up to a tenth of the load. Above 50 kW, Uptime expects liquid, and at 150 kW total liquid cooling. The 2025 survey shows how new this is: the typical rack is still around 9 kW, 18 % of operators run racks above 30 kW, 9 % above 50 kW, and racks above 100 kW remain rare.

Three routes, three depths of change

Sources: vendor datasheets (Vertiv, Motivair, Legrand ColdLogik, GRC), Schneider Electric WP133, Uptime Institute, DCD.

The rear door is the route that changes least. The servers stay air-cooled; the door takes the exhaust through a coil and returns cooled air to the room. Passive doors rely on the server fans, active ones add their own: Vertiv’s active DCD50 moves 9,000 m³/h. Capacity depends on water temperature, which is why one product quotes 93 kW at 14 °C and 74 kW at 21 °C. NVIDIA’s DGX B300 guidance is blunt: passive doors are not recommended, and four systems per rack need an active door sized for 58 kW average and 76 kW peak at 6,200 CFM.

Cold plates take the heat where it is made. Vertiv puts the captured share at 70 to 75 %, Schneider at 60 to 90 %, and Supermicro claims 98 % for its latest generation. Whatever the figure, the remainder is still air: Uptime’s figures for a 132 kW-rated GB200 NVL72 are about 100 kW to the plates and more than 25 kW into the hall. A liquid-cooled rack still needs air cooling, only less of it.

Immersion removes the fans and most of the airflow problem, at the price of a different server, a different service procedure and, historically, a different warranty conversation. There is still no accepted standard for the fire behaviour of single-phase fluids, and interest in two-phase cooling has faded since 3M left the market.

The water side: temperatures, flow, distribution

ASHRAE classifies facility water by its maximum supply temperature: W17, W27, W32, W40, W45 and above, all with a 2 °C floor, with compliance meaning full unthrottled operation across the class. The industry is converging on 32 °C supply (class W32), with 40 and even 45 °C as the targets. The hotter the water the equipment accepts, the less refrigeration the site needs: NVIDIA’s stated point for its next generation is 45 °C water with no chillers at all.

Flow follows a simple rule: about 1.5 litres per minute per kilowatt, which is what water at a 10 K rise carries: a 9 kW server needs about 13 L/min. The coolant in the secondary loop is usually PG25, three parts water to one part propylene glycol, filtered to 50 µm, and always held above the room’s dew point so that nothing condenses on a cold plate.

Sources: Schneider Electric WP133, Vertiv CoolChip datasheets.

The liquid-to-air units are the tempting shortcut: no plumbing to the building, the heat goes back into the hall. They work for a rack or two and then hand the problem to the air conditioning, which is where it came from. Anything from a row upwards needs a facility loop, a heat-rejection path outside the building and, in a modern design, dry coolers rather than chillers.

What 120 kW looks like

The GB200 NVL72 is the reference because it settled the argument. NVIDIA rates the rack at approximately 120 kW; Supermicro’s version lists 132 kW. The eighteen compute trays and the nine NVLink switch trays are liquid-cooled; only the power shelves and the management networking stay on air. Eight power shelves deliver 33 kW each in an N+N arrangement, and leak detection is built in. At GTC NVIDIA described the loop as 25 °C in, 45 °C out, about 2 litres per second; integrators quote inlet limits around 45 °C, return up to 65 °C and up to 130 L/min per rack. There is no air-cooled version. The same silicon in air-cooled form is the DGX B200 at about 14.3 kW for ten rack units, and Supermicro fits 32 air-cooled B200 GPUs in a rack against 96 with cold plates.

The retrofit nobody budgets for

A liquid rack in an existing hall is a building project before it is an IT project. The floor first: TIA-942 rates a class 3 data-hall floor at 12 kPa, about 1,220 kg per square metre, and NVIDIA added more than 100 lb of steel to the NVL72 rack frame to withstand the mating force of the trays against the backplane. Then the loop: supply and return piping to every rack position, drip trays with leak detection and piped drains under anything that runs above live equipment, which ASHRAE lists as a requirement rather than a suggestion. Then heat rejection outside, commissioning of the CDUs with redundant pumps, and a service procedure for a rack that cannot simply be unplugged and rolled out.

Vertiv publishes a six-step roadmap for exactly this: workload, site audit, thermal modelling, site impacts, sustainability review, design; commissioning comes on top. No vendor publishes a timeline, and we will not invent one; the honest answer to “what is the fastest way to get liquid cooling deployed” is that the rear door is the only route that does not touch the building, and it stops somewhere between 50 and 90 kW depending on how cold your water is.

Deciding

Under 30 kW per rack, air with containment. Between 30 and 75 kW, an active rear door and cold water are usually the least disruptive answer. From 75 kW upwards, or wherever the servers themselves ship with cold plates, direct-to-chip with a facility loop. At 120 kW there is no decision, only the plumbing.

Our engineering partner Vixen.UNO treats the cooling method as an input to the server order, not an afterthought: the same GPUs come in air-cooled and liquid-cooled variants, and the wrong choice is expensive to reverse. The efficiency argument is real but secondary: liquid-cooled facilities report PUE of 1.05 to 1.15 against an industry average of 1.54, and Vertiv is right that PUE understates what direct-to-chip saves in fan power inside the server.

FAQ