In data centers, where a single AI accelerator can consume more than a kilowatt of power itself and a full compute rack can generate more than 100 kW of heat, air cooling is hitting its limits.

As AI power demands move toward a megawatt per rack, hyperscalers and other tech giants are installing liquid-cooling infrastructure throughout their data centers. A single NVIDIA GB200 NVL72 rack contains more than 100 fluid connections that circulate coolant within inches of some of the most advanced silicon on the market. Multiply those across thousands of racks, and the result is a sprawling network of pumps, pipes, manifolds, and other cooling infrastructure.

Consequently, millions of gallons of cooling fluid are now being piped directly into and around delicate electronic assemblies. In direct-to-chip cooling, often called cold-plate cooling, coolant circulates through metal plates mounted on top of CPUs and GPUs, removing heat more efficiently than cooling fans forcing air over them. In immersion cooling, the solution is more radical: processors and other hardware are fully submerged into large tanks of fluid. 

Looking further out, Corintis and other startups are investing in microfluidic cooling, which pushes coolant through microscopic channels integrated directly into the chip package or the silicon itself — and potentially even the power electronics feeding them. This brings cooling closer to the hottest parts of the system.

As compute racks become increasingly power-hungry, so do the liquid-cooling systems required to keep them running. Given that, improving the efficiency of liquid-cooling itself has become a top priority.

New innovations in motors and pumps (and the controllers driving them) are emerging to keep coolant flowing smoothly while reducing power, and increase system reliability. These include sensorless, brushless DC (BLDC) motor drivers designed to more efficiently circulate coolant through data centers.

While liquid cooling can help tame the thermal conditions surrounding AI, it also creates new failure risks for AI compute racks. Water or other coolants flowing through servers and rack-level plumbing pose a potential threat to densely packed circuit-board assemblies and all of the electronics mounted on them. A microscopic leak, hairline crack, or even a drip of condensation can trigger a short circuit across ultra-dense components or the destruction of board traces. Whatever the result, it could lead to costly system downtime.

Emerging to solve the problem are advanced sensors that can identify even trace amounts of coolant leaking into the system, while material innovations like conformal gel coatings could help protect sensitive electronics from fluid intrusion, moisture, and corrosion.

For more coverage on liquid cooling and the technologies that keep it running efficiently and reliably, check out the latest digital issue of Electronic Design here.