How Data Centers Work

Key Takeaways

• Data centers are fundamentally heat removal problems—all electricity becomes heat that must be extracted
• Power density has increased tenfold in the AI era, from 5-15 kW per rack to 60-140+ kW
• Cooling technology determines facility design: air cooling at 20 kW, direct-to-chip at 80 kW, immersion at 300+ kW
• PUE (Power Usage Effectiveness) measures efficiency; modern facilities achieve 1.1-1.2, meaning only 10-20% overhead
• Redundancy systems (N+1, 2N) ensure reliability but can double infrastructure costs

The Hidden Factories

Behind every AI conversation, every streaming video, every cloud application is a building. Not a small building—a facility often the size of several shopping malls, filled with computers stacked in rows, cooled by industrial systems, and consuming enough electricity to power a small city.

These are data centers, the physical infrastructure of the digital world. And they've evolved dramatically over the past decade, driven by the explosion of cloud computing and, more recently, artificial intelligence.

In the early 2000s, a typical data center might occupy 50,000 square feet and consume 5 megawatts of power. Today's hyperscale facilities can exceed 1 million square feet and consume 200 to 300 megawatts. Some of the facilities being built for AI will reach 1,000 megawatts or more—equivalent to a large power plant dedicated to a single building.

To understand why these facilities are so large, so expensive, and so power-hungry, you need to understand their basic components and the engineering challenges they solve.

The Rack: The Fundamental Unit

Walk into a data center and you'll see rows of tall black cabinets. These are racks, and they're the fundamental building block of the entire facility.

The standard rack is 19 inches wide and 42U tall. A "U" is 1.75 inches, so 42U means about 73 inches—just over six feet. Inside that space, you mount servers, networking equipment, and power distribution units on sliding rails.

For decades, rack design was stable. A typical server was 1U or 2U tall, consumed 200-500 watts, and you might fit 20 to 40 servers in a rack. Total power per rack: 5 to 15 kilowatts. Data center designs optimized for this density. Raised floors with perforated tiles delivered cool air from below. Hot air exhausted above. Simple, reliable, well-understood.

Then AI arrived.

Modern AI servers are dramatically more power-dense. A single NVIDIA HGX H100 server—just 8U tall—can draw 10 kilowatts. A fully populated AI rack might contain four or five of these systems. Suddenly, your rack isn't 10 kW anymore. It's 40, 60, even 80 kilowatts.

Some next-generation designs are pushing to 140 kilowatts per rack. At that density, a single rack consumes more power than 100 typical American homes. And a data center might have thousands of these racks.

This tenfold increase in power density changes everything about data center design. The cooling systems designed for 10 kW racks can't handle 80 kW. The electrical distribution systems can't deliver enough power. The entire facility architecture needs to be rethought.

The Heat Problem

Here's an inescapable fact of physics: all the electricity that goes into a data center comes out as heat. A 100-megawatt facility doesn't just use 100 megawatts—it produces 100 megawatts of heat that must be removed, continuously, or the equipment will destroy itself.

Processors have temperature limits. An NVIDIA H100 chip might have a maximum junction temperature of 90°C (194°F). Go above that and the chip throttles performance to protect itself. Go too far above and it fails permanently. At $30,000 per chip, that's an expensive failure.

Heat accumulates quickly. A rack generating 60 kilowatts in a sealed enclosure would reach destructive temperatures in minutes without cooling. The challenge isn't just removing heat—it's removing it continuously, reliably, and efficiently at a scale that would make HVAC engineers blanch.

This is why data center engineering is fundamentally heat engineering. Everything else—power distribution, networking, physical security—is important but secondary. If you can't remove the heat, nothing else matters.

Cooling Technologies: From Air to Liquid

For decades, data centers used air cooling. Giant air conditioning units chilled air, fans pushed it through the facility, the air absorbed heat from servers, and the hot air exhausted outside or to cooling towers. This approach works well up to about 20 kilowatts per rack.

Above 20 kW, air cooling becomes impractical. Air has relatively poor thermal properties—it doesn't carry much heat per volume. To cool a 60 kW rack with air, you'd need to move massive volumes at high velocity, creating noise, turbulence, and inefficiency.

Enter liquid cooling. Water and specialized coolants can absorb far more heat per volume than air. This enables new cooling architectures.

Direct-to-chip cooling circulates liquid through cold plates mounted directly on processors. The coolant absorbs heat at the source, then flows to a heat exchanger where it transfers that heat to facility water or outside air. This approach can handle 40 to 80 kilowatts per rack—enough for current AI workloads.

Direct-to-chip cooling requires significant infrastructure. Every rack needs coolant supply and return lines. You need pumps, heat exchangers, and leak detection systems. The plumbing complexity increases dramatically. But for AI data centers, it's becoming standard.

Immersion cooling takes this further: submerge entire servers in dielectric fluid (a coolant that doesn't conduct electricity). Heat transfers directly from all components to the fluid, which then flows to heat exchangers. This can handle 300+ kilowatts per rack—densities impossible with any other technology.

Immersion cooling sounds exotic, and it is. The fluid is expensive. Servicing servers means pulling them out of tanks. Compatibility issues exist. But for the highest-density AI deployments, it's increasingly viable. Several companies are building immersion-cooled facilities specifically for AI training clusters.

The cooling choice determines facility design. An air-cooled facility needs massive HVAC systems and tall ceilings for airflow. A direct-to-chip facility needs extensive plumbing and heat exchangers. An immersion facility needs tanks, fluid handling, and specialized server designs. Each approach has trade-offs in cost, complexity, reliability, and density.

PUE: Measuring Efficiency

How do you measure data center efficiency? The industry standard is PUE: Power Usage Effectiveness.

PUE is a simple ratio: total facility power divided by IT equipment power. If your IT equipment uses 100 megawatts and your total facility uses 120 megawatts, your PUE is 1.2. The extra 20 megawatts goes to cooling, power conversion losses, lighting, and other overhead.

A PUE of 1.0 would be perfect—all power goes to IT equipment, none to overhead. That's impossible. But modern facilities are getting close.

Historical data centers often had PUEs of 1.6 to 2.0. For every megawatt of IT equipment, they used 0.6 to 1.0 megawatts of overhead. Today's hyperscale facilities routinely achieve 1.1 to 1.2, meaning only 10-20% overhead.

This improvement comes from several factors. Free cooling (using outside air when temperatures permit) reduces mechanical cooling load. More efficient power supplies waste less electricity as heat. Better airflow management reduces fan energy. Liquid cooling eliminates much of the HVAC overhead.

Why does this matter? At scale, efficiency differences translate to millions of dollars annually and megawatts of capacity. A 100 MW facility at PUE 1.5 needs 150 MW total. At PUE 1.2, it needs 120 MW—a 30 MW difference. That's 30 fewer megawatts of generation, transmission, and cooling infrastructure required.

For the Saline Township project in Michigan, which will reach 1.4 gigawatts of IT load, the difference between PUE 1.2 and 1.5 is 420 megawatts—more than many entire data centers consume.

Redundancy and Reliability

Data centers can't go down. Not for cloud services, not for AI inference, not for enterprise applications. The cost of downtime—in lost revenue, user trust, and service level agreement penalties—is enormous.

This drives redundancy, which means having backup systems for every critical component. But redundancy comes in levels, each progressively more expensive.

N+1 redundancy means you have one extra unit beyond what's needed. If you need four chillers to cool your facility, you install five. If one fails, the other four handle the load. This is the minimum for most data centers.

2N redundancy means you have double everything. Two complete, independent systems, either of which can handle the full load. One fails, the other takes over seamlessly. This is common for Tier IV data centers (the highest reliability classification).

2N+1 redundancy goes further: two complete systems plus an additional backup unit. This is overkill for most applications but exists in critical deployments.

Redundancy doesn't just apply to cooling. Power systems need it too. Uninterruptible Power Supplies (UPS) provide instant backup from batteries if utility power fails. Diesel generators kick in within seconds to provide longer-term backup. Fuel storage ensures generators can run for days.

Some facilities have multiple utility feeds from different substations, protecting against transmission failures. Others have redundant fiber paths for network connectivity. The principle is the same: eliminate single points of failure.

The cost is substantial. A 2N power system means you're buying, installing, and maintaining two complete electrical plants instead of one. Your capital costs roughly double. Your facility footprint increases. Your maintenance complexity multiplies.

But for hyperscale operators and enterprise customers, it's worth it. The industry measures uptime in "nines." 99.9% uptime (three nines) means 8.76 hours of downtime per year. 99.99% (four nines) means 52.6 minutes. 99.999% (five nines) means 5.26 minutes.

Achieving five nines requires aggressive redundancy. And every nine costs exponentially more than the last.

Water Consumption: The Hidden Cost

Even facilities that don't use liquid cooling consume water—often enormous amounts.

Evaporative cooling, used by many data centers, works like sweating. Water evaporates, absorbing heat in the process. Cooling towers use this principle to reject heat from facility water loops to the atmosphere. It's efficient and cost-effective, but the water evaporates and doesn't come back.

The typical ratio is about 1 to 2 million gallons of water per megawatt per year. For a 100 MW facility, that's 100 to 200 million gallons annually—enough to fill 300 Olympic swimming pools.

In regions with abundant water, this rarely raises concerns. But in arid areas like Arizona, New Mexico, or parts of Texas, data center water consumption has become controversial. Water that goes to data centers doesn't go to agriculture, residential use, or ecosystem preservation.

Some facilities use "dry cooling" that doesn't consume water, instead using large radiators like car cooling systems. This works but requires more land, higher capital costs, and reduces efficiency in hot weather. Other facilities recycle water, use municipal wastewater, or implement closed-loop systems that minimize consumption.

The water question is becoming more prominent as data center construction accelerates in water-stressed regions. It's not just about getting approval—it's about long-term sustainability and community relations.

Putting It All Together

A modern AI data center is a complex system where every component depends on every other component. The rack density determines cooling requirements. The cooling technology determines power requirements. The power requirements determine electrical infrastructure. The reliability requirements determine redundancy levels. The redundancy levels determine facility size and cost.

Consider the Saline Township project: 1.4 gigawatts of IT load, presumably at PUE around 1.2, means 1.68 gigawatts total. At 60-80 kW per rack, that's roughly 18,000 to 23,000 racks across 250 acres. The cooling system must remove 1.68 billion watts of heat continuously. The electrical system must deliver power at transmission voltages and step it down through multiple substations. The backup systems must handle failures without service interruption.

The construction timeline for such a facility is 2 to 3 years. The cost exceeds $7 billion. The operational complexity requires hundreds of skilled technicians, engineers, and operators.

And this is just one facility among hundreds being built across the United States to support AI and cloud computing growth.

Data centers are hidden infrastructure—most people never see them, never think about them. But they're as critical to modern life as power plants and water treatment facilities. Every digital interaction relies on them. And as AI becomes more integrated into daily life, that dependence will only deepen.