Why AI capacity is counted in gigawatts

The unit nobody chose

Ask someone how big a new AI data center is, and they will answer in gigawatts. They will not answer in servers, in square feet or in exaflops. They use a unit of electrical power, as if the facility were a power plant in reverse.

This is a strange choice. Nobody gives the size of an airport in tons of kerosene per hour, or the size of a hospital in oxygen consumption. For most of the history of computing, nobody gave the size of a data center this way either. Operators quoted facilities in floor area, and later in racks. The change to megawatts happened quietly, over about a decade. It happened because every other unit stopped meaning anything.

The reason is worth finding, because the same story explains most of what is physically unusual about AI infrastructure:

• why the buildings are plumbing projects
• why the industry is changing its power wiring to 800 volts
• why photonics stopped being a long-distance technology and became a rack-scale one.

The collapse of the old units

A decade ago the average rack drew 4 or 5 kW, about what two domestic kettles draw. Air conditioning built for 8 to 12 kW per rack cooled it with margin. The capacity of a data center really was its floor area. More area gave more racks and more compute. Quoting square feet was honest.

Then accelerators arrived. NVIDIA specifies about 120 kW for a GB200 NVL72 rack, and vendors rate their rack designs at 132 kW. That number survives a bottom-up check, and it is worth doing the check once yourself. Seventy-two accelerators at roughly a kilowatt each is 72 kW before any other part starts. Add the CPUs, the NVLink switch trays, the fans and the conversion losses. Then a total above 120 kW is not surprising. It is nearly inevitable.

The OCP ecosystem now designs for 1 MW racks. Google presented them, and NVIDIA and Delta showed the 800-volt power architecture intended to feed them. From 5 kW to a megawatt is a factor of two hundred. About twenty-five of that happened in the last decade. The roadmap asks for the other eight.

Apply this change to a fixed amount of compute, and the old units fail. Ten megawatts of IT load in 2015 meant two thousand racks and a warehouse. In GB200 racks it means seventy-six. On the megawatt-rack roadmap it means ten, in a footprint smaller than a tennis court. Floor area did not become less informative step by step. It became almost meaningless, because it stopped predicting anything a buyer cares about.

Density at this level does more than make the old unit obsolete. It meets three separate physical limits. This essay calls them walls.

The first wall: air

Air is a mediocre coolant. The volume of air that you must move increases with the heat that you remove. The published thresholds are consistent:

• Plain hot-aisle and cold-aisle air cools a rack up to about 12 kW.
• Full containment extends this to 40 or 50 kW.
• Rear-door heat exchangers bring water to the cabinet, but not yet to the chip. They extend the range to perhaps 75 kW. Sources frankly describe them as transitional. One reason is that the temperature differences start to condense water where nobody wants it.
• Somewhere past 100 kW, air cooling stops working, and cold plates must touch the silicon.

This is the wall that people notice, because it changes what the building is. An air-cooled data center is a room with fans. A liquid-cooled data center is a plumbing system with computers attached. It has manifolds in every rack, coolant distribution units in every row, and a facility water loop behind them. Floor loading and ceiling heights must match. This wall also makes old buildings hard to reuse, which is a story of its own.

×

What a liquid-cooled machine actually looks like from the aisle: ORNL's Frontier, plumbing below, interconnect bundles everywhere.

The second wall: the busbar

The second wall gets less attention. It is also purer, because it is only arithmetic.

Power is voltage times current. The bus inside a rack operated at 48 volts for a generation. If the voltage stays fixed, the current increases with the power of the rack:

For scale, the starter motor in a car draws a couple of hundred amps, for about one second, through cables as thick as a thumb. At its rating, a GB200 rack needs fourteen times that, continuously. At twenty thousand amps, the conductor of the megawatt rack is not a cable at all. It is a structural member that also conducts current.

The true cost is worse than the current suggests, because resistive loss increases with the square of the current. Move the same rack from 48 to 800 volts, and the current decreases by a factor of 16.7. The heat lost in any given conductor decreases by a factor of 278.

That factor of 278 is the entire argument for the 800 VDC transition, which is now moving through OCP. The reported results are concrete:

• Copper cross-sections become up to 45 percent smaller.
• Power shelves leave the compute rack, which gives more room for compute.
• Conversion moves into a sidecar cabinet, so the upstream AC plant can stay where it is.

The literature states the trigger point plainly. Above roughly 200 kW per rack, the 48-volt bus reaches a limit set by copper and heat. The industry did not want a new power architecture. The square law made one necessary.

The third wall: the signal

The first two walls are about how to feed and cool a rack. The third wall is about how to talk to it, and it is the wall that concerns this site most directly.

Copper that carries signals has its own scaling problem, and that problem is distance. As lane rates increase, the frequency that a trace must carry increases with them. Loss increases with frequency. So the span that a passive copper cable can cover becomes much shorter:

• roughly 2.5 meters at 112G lanes
• about 1 meter of passive cable at 224G
• under half a meter projected at 448G.

The full derivation, and the reason that fiber does not have this problem, is an essay of its own. This essay needs only the result.

Now look at what the first two walls did to the geometry. Rising rack power spreads a fixed compute job across more cabinets and more meters. At the same time, the usable meters of copper decrease. The two curves cross. Below the crossing, the wiring of a training domain is only a cabling task. Above it, no arrangement of copper connects the machine at all, and the fabric between racks must become optical.

For rack-to-rack links, the crossing already occurred, at 224G. In this sense, photonics now sits in the critical path of compute, not out at the end of a long wire.

What a good unit knows

So why did megawatts win? After those three walls, power is the one number that predicts everything else.

Say that a site is one gigawatt. Then you know these things to useful precision:

• how much compute the site can hold
• what its cooling plant must move
• what its switchgear must carry
• how long its utility queue will be
• roughly what it cost.

No other single figure does that. Floor area predicts nothing now. Rack count depends on the year of the racks. Even FLOPS lose their meaning with every chip generation, while a megawatt is a megawatt forever.

One ambiguity does remain inside the unit, and it is worth billions. “A gigawatt” can mean the IT load, the meter reading, or the redundant grid connection. The difference between those readings is a large fraction of the total cost of a site. But the industry agrees on the dimension.

This observation applies well beyond data centers. Industries measure capacity in the resource that limits them:

• Shipping counts containers, not cargo value.
• Foundries count wafer starts, not chips.
• Airlines count seats.

When this industry began to quote its size in gigawatts, it announced what it is actually short of, whether it meant to or not. It is not short of land, of servers, or even of capital. It is short of watts.

What to take away

1. Rack density increased more than twentyfold in a decade, and the megawatt-rack roadmap adds about eight times again. Floor area stopped predicting capacity. The unit followed the physics.
2. Air cooling ends near 50 kW per rack and stops working by 100. Liquid cooling is a structural fact of AI buildings, not a preference.
3. The 48-volt bus ends near 200 kW, where racks need thousands of amps. The 800 VDC transition is a direct result of the square law of resistive loss.
4. Copper signaling fails on distance, at the same time as compute domains spread across more distance. That crossing already occurred at 224G, and it is why rack-to-rack fabrics are becoming optical.
5. Units show scarcity. This industry counts watts because it cannot get enough watts.