Why AI racks operate at 60-100 kW (and what changes in your cooling)

Today’s AI racks operate at power levels that make traditional room cooling obsolete. When a rack moves from 30 kW to 60-100 kW, the cooling dynamic changes completely. The room stops breathing, air stops being enough, and the decision moves to liquid.

Here is why we got here, what changed in the hardware, and what it means for your data center cooling.

Where that power density comes from

AI racks are built on GPUs (Graphics Processing Units) that consume between 400 W and 1,200 W per unit. A typical GPU platform — such as systems based on hyperscaler reference architectures — hosts multiple GPUs in a single chassis. Multiply that by the number of nodes in a rack for training large models and you reach densities of 50, 80, or more than 100 kW per rack.

Why air no longer scales

ASHRAE TC 9.9 defines thermal classes for IT equipment from A1 to A4. Class A1 covers operation up to 25°C (77°F) inlet with limited margin. As rack density rises, the temperature difference between inlet and outlet air grows. A 100 kW rack cannot be cooled by air through natural convection.

1. Air has limited heat capacity. Moving 100 kW of heat with air requires enormous flow rates, high-power fans, and small temperature differentials.
2. Hot air rises, but not enough. Stratification in a closed room is hard to control at this density.
3. Floor diffusers and hot/cold aisles become irrelevant. You need cooling at the chip.

What options operators have

The architectures that AI rack operators are adopting include:

1. Direct liquid cooling to the chip (DLC). Cold plates connected to manifolds deliver coolant to the silicon. It is the most efficient option but requires compatible racks and CDUs (Coolant Distribution Units) in the row.
2. Liquid immersion. Servers are submerged in a dielectric fluid. It eliminates air completely from the rack. High efficiency, but a deep change in operation and maintenance.
3. Rear-door heat exchangers. For racks that cannot migrate to full DLC, the rear-door exchanger removes heat from the outlet air with water. It is an intermediate step.
4. Air with hot-aisle containment plus higher flow rate. Only viable up to moderate per-rack ranges. For 60-100 kW it stops being enough.

If your data center is going to host AI workloads, the question is not “can I cool it with air?” but “what liquid architecture do I implement?”. Air is history at this density. Liquid is the present.

Sources

[1] ASHRAE — TC 9.9 Datacom Equipment Power Trends (thermal guidelines): https://www.ashrae.org/technical-resources/bookstore/datacom-series

[2] Nvidia — HGX/MGX Platform specifications (vendor product documentation): https://www.nvidia.com/en-us/data-center/hgx/

[3] Vertiv — Liquid Cooling Solutions for AI (vendor reference): https://www.vertiv.com/en-us/solutions/liquid-cooling/

[4] Schneider Electric — High-Density Cooling for AI Workloads (vendor reference): https://www.se.com/ww/en/about-us/newsroom/blog/data-center-cooling/

[5] Wikipedia — Data Center Cooling (background reference): https://en.wikipedia.org/wiki/Data_center