AI’s next ROI jump is not more AC, its less!

The most expensive mistake in AI infrastructure may be treating heat as only a building-services AC problem. It is not!

By the time heat has reached the computer room, the facility operator is already paying to manage a problem that began much earlier: at the semiconductor junction, inside the package, across the thermal interface, through the substrate and into the server architecture.

Bigger chillers, better airflow and more sophisticated air-conditioning can help, though they are treading the results not the cause, they are downstream remedies.

That distinction matters because AI factories are not conventional datacentres with more expensive servers. One way to see them is as industrial energy conversion systems. Electricity goes in. Computation comes out. Heat is the unavoidable by-product. It’s the result of losses and inefficiencies. The commercial result performance is capped due to the systems being throttled back and billions being spend to handle the heat.

For investors, its essential to their IRR. Everyone knows ROCE (Return On Capital Employed) the smart ones are now benchmarking ROEC or Return On Energy Concumed.

AI factories are changing the cooling equation

The energy figures are large enough to attract headlines, but the more important issue is density.

The International Energy Agency estimates that datacentres consumed around 415 TWh of electricity in 2024, about 1.5% of global electricity demand, and projects that this could more than double to around 945 TWh by 2030. AI is the main driver of that growth, but the IEA also notes that the local impact of large AI-focused datacentres can be much more pronounced than the global percentage suggests.

This is why the phrase “AI factory” is useful. It reminds boards that these facilities are closer to industrial plants than office IT rooms.

A modern AI rack concentrates extraordinary power into a small physical footprint. NVIDIA’s GB200 NVL72, for example, connects 36 Grace CPUs and 72 Blackwell GPUs in a rack-scale, liquid-cooled architecture. Supermicro’s rack-level specification for a GB200 NVL72 system lists total rack power at 132kW.

At this level, cooling ceases to be a matter of comfort cooling. Schneider Electric argues that air cooling becomes insufficient as AI rack densities exceed roughly 80–100kW, making liquid-based cooling the preferred architecture for stable high-performance compute.

That does not mean air disappears. Most facilities will remain hybrid for years. But it does mean the economic centre of gravity is moving. The question is no longer how to make a cold room colder. It is how to move heat away from the chip with the least energy, least complexity and least loss of compute performance.

The real bottleneck is thermal resistance

In the case of Datacentres its useful to think about AI cooling in terms of thermal resistance not just “temperature”.

The heat produced must travel from the semiconductor junction, through layers of material, into a heat spreader, across a thermal interface, into a cold plate or air stream, through a rack-level system and finally out of the building.

Each layer resists the movement of heat. Each resistance adds additional temperature increases. Each additional degree narrows the operating profit margin.

At the business level, that means lower sustained performance, higher fan power, more expensive cooling infrastructure, shorter component life or reduced upgrade flexibility. 

In some cases, it means the most expensive equipment in the building cannot be run at its full economic potential.

This is why “cooling efficiency” is too narrow a phrase. The strategic issue is the thermal pathway from junction to ambient.

A better thermal pathway gives the operator optionality. It can run the same processor cooler. It can run the same processor harder. It can reduce mechanical cooling overhead. It can increase rack density. It can extend equipment life. It can make sites viable that would otherwise require expensive compressor-based air-conditioning.

That is why upstream materials innovation matters.

Better AC is not enough

The datacentre industry has understandably focused on facility-level solutions: chillers, computer-room air handlers, airflow containment, rear-door heat exchangers, immersion systems, coolant distribution units and direct-to-chip liquid loops. These are important.

But they are still only part of the answer.

If the package and substrate around the chip trap heat, the facility can only compensate by spending more energy downstream. That is like improving a road network after deliberately creating a traffic jam at the factory gate. 

The next layer of advantage is closer to the heat source: advanced substrates, thermal interface materials, heat spreaders, vapour chambers, microfluidic structures, high-conductivity ceramics, copper-diamond composites, synthetic diamond and other materials designed to reduce the thermal penalty before it becomes a room-level cooling load.

This is not theoretical. DARPA’s Near Junction Thermal Transport programme focused on the region within 100 microns of the electronic junction, describing many high-power components as being limited by the thermal resistance of that near-junction region. Its approaches included high-thermal-conductivity diamond substrates, removal of low-conductivity transition layers and liquid cooling in the near-junction region.

The same DARPA work on GaN-on-diamond reported substantially lower junction temperatures than comparable commercial devices, with the steepest temperature rise occurring in the first few microns below the junction. The lesson is not that every AI server becomes GaN-on-diamond. The lesson is that solving heat nearer the device can change the performance envelope of the entire system.

For AI infrastructure, this is the direction of travel: not just colder rooms, but better materials and architectures between the chip and the cooling loop.

Room-temperature AI

No its not fictitious, though delivering it may deliver a Unicorn or two. 

“No AC” does not mean no cooling. Heat still has to leave the system. What changes is the grade and methods used to achieve the cooling required. If the server, package and cooling pathway can keep junction temperatures within specification while operating with warmer inlet air or warmer liquid supply temperatures, the facility can move away from compressor-driven air-conditioning toward ambient heat rejection, dry coolers, economisers or simpler liquid loops.

That is the commercial opening and the opportunity for materials technology advances and associated companies to become unicorns, who knows maybe even “Kilocorns”.

If conventional GPU servers can be adapted so they operate reliably in ordinary technical-room conditions, or with warmer liquid loops, the datacentre design changes. The operator may reduce mechanical cooling load. The site may need less dedicated HVAC equipment. Deployment may become faster. Water consumption may fall. More locations become viable. Retrofit economics become more attractive.

In many countries the climate makes this possible.

Retrofit may matter as much as new-build

Much of the AI infrastructure debate focuses on the newest rack-scale systems. That is understandable. New architectures are cleaner to design around.

But the larger near-term opportunity may be in the installed and soon-to-be-installed base of AI servers using standard NVIDIA and AMD accelerator platforms. These are the assets investors are already financing, operators are already deploying and enterprises are already trying to monetise.

If advanced thermal materials, improved heat spreaders, better cold-plate interfaces or server-level adaptations can materially reduce cooling overhead on existing platforms, the return profile changes quickly.

A 50–70% reduction in cooling overhead, does not mean the GPU itself uses 50–70% less power. The chip still consumes substantial energy. But the system may waste less energy moving heat, require less mechanical cooling, preserve performance under sustained load and make infrastructure easier to deploy in less specialised environments.

What investors should ask

Boards and investment committees should push AI infrastructure companies beyond generic claims about “efficient cooling”.

The better questions are more specific.

• Where is heat being removed: room, rack, cold plate, package, substrate or junction?
• What is the junction-to-ambient thermal resistance under sustained AI workloads?
• How much of the energy saving comes from reduced compressor load, reduced fan power, reduced throttling or higher utilisation?
• Can the system operate at room temperature across normal markets without mechanical AC, and where does that break down?
• What ambient temperature assumptions are embedded in the business model?
• Can the solution retrofit existing servers, or does it require a new-build architecture?
• What happens to performance, warranty and maintenance when the system is operated at warmer temperatures?
• How much useful AI output is produced per megawatt over the life of the asset?

These questions connect physics to finance. They also separate serious infrastructure advantage from marketing language.

The new ROI metric is useful compute per watt

The first phase of AI infrastructure rewarded access to chips. The second phase will reward the ability to operate those chips economically.

The highest return improvements will come from moving the intervention upstream: from the room to the rack, from the rack to the server, from the server to the package, and from the package to the semiconductor junction itself.

The AI factory will not be made efficient by surrounding inefficient thermal architectures with more air-conditioning.

It will be made efficient by reducing the heat penalty at source.

As in my previous article, “In orbit, ROI is measured in watts, not data”, the same logic is emerging on the ground in AI Factories.

The winners will be those that convert each watt into the most useful compute before heat robs their margins.