Nvidia's Hot Data Center Gambit: Higher Temperatures, Less Water, and a Cooling Revolution

I've spent the last decade writing about data centers, and I've never seen anything quite like the public backlash that's hit the industry over the past two years. Communities in places like Arizona, Ireland, and Singapore have started fighting new data center builds with a ferocity usually reserved for coal plants or landfills. And honestly? I get it. When you hear that a single AI training cluster can slurp up as much water as a small town, it's hard not to feel a little queasy.

Nvidia, the company at the very center of the AI boom, has been feeling that heat—both literally and figuratively. Their GPUs are the ones doing the heavy lifting for models like GPT-4 and Gemini, and those chips run hot. Really hot. But last week, Nvidia dropped a bombshell that might change the conversation: their next-generation Rubin data center reference design runs at higher temperatures but uses "pretty much all" less water. According to www.theverge.com, the company claims its Rubin generation reference design for a fully liquid-cooled data center has "eliminated massive amounts of power usage and pretty much all water usage."

That's a bold statement. Let's dig into what it actually means.

The Water Problem Nobody Wants to Talk About

Here's a number that will stick with you: a typical hyperscale data center can consume anywhere from 1 to 5 million gallons of water per day. That's not a typo. Most of that water doesn't go into your coffee—it evaporates in cooling towers, turning into steam that carries away the immense heat generated by thousands of servers running at full tilt.

I visited a Google data center in Finland a few years back, and the cooling system was genuinely awe-inspiring. The facility uses seawater from the Gulf of Finland, pumping it through heat exchangers before returning it, slightly warmer, back to the sea. That's elegant. But most data centers aren't on the coast. They're in Northern Virginia, or Phoenix, or Dublin—places where water is either scarce or politically charged. The average data center in a dry climate uses about 3.5 million gallons of water per year just for cooling. That's roughly the same as a 500-unit apartment building.

Nvidia's argument is simple: if you can run your data center hotter, you don't need as much evaporative cooling. Instead of keeping your GPU inlet temperature at a chilly 20°C (68°F), you can push it to 35°C or even 40°C (95-104°F). At those temperatures, you can use dry cooling—basically giant radiators with fans—instead of water-hungry cooling towers. You can even capture the waste heat and use it to warm nearby buildings or greenhouses. (Side note: I've always loved the idea of data center heat being used for urban farming, but it's rarely practical at scale. Maybe that changes now.)

Liquid Cooling Isn't New, But This Is Different

Let me be clear: liquid cooling isn't some sci-fi future tech. It's been used in high-performance computing for decades. IBM's mainframes from the 1980s used liquid cooling. Cray supercomputers used liquid cooling. But those were niche systems for national labs and oil companies. Nvidia is talking about putting liquid cooling in every hyperscale data center. That's a different ballgame entirely.

The Rubin generation, named after the astronomer Vera Rubin, is Nvidia's next major architecture, expected to launch in 2026. It's the successor to Blackwell, which itself is the follow-up to Hopper. The naming is a nice touch—Vera Rubin discovered dark matter, and Nvidia is trying to make the energy and water consumption of AI disappear in a similar way.

According to www.theverge.com, Nvidia's reference design moves away from traditional air cooling entirely. Instead of blowing air over finned heat sinks, the Rubin systems use direct-to-chip liquid cooling, where a coolant (usually a dielectric fluid or treated water) flows through cold plates attached directly to the GPUs. That coolant picks up the heat and carries it to a heat exchanger, where it's rejected to the outside air. Because the coolant can handle higher temperatures, the heat exchangers don't need to be as efficient—and they don't need water evaporation to work.

Nvidia claims this eliminates "massive amounts of power usage" too. That's because fans are surprisingly inefficient. A typical data center fan wall—a giant array of fans that push air through the server racks—can consume 5-10% of the total facility power. Liquid cooling eliminates most of those fans. The pumps for the coolant use less energy than the fans they replace. It's not a silver bullet, but it's a meaningful improvement.

The Catch: You Have to Run Hot

Here's where things get interesting. Running your data center at 35°C inlet temperature is not without consequences. Electronics are generally happier when they're cooler. Higher temperatures increase leakage current in transistors, which can reduce efficiency and increase power consumption. They also accelerate electromigration, the gradual movement of metal atoms in the chip's wiring, which eventually causes failures.

Nvidia is betting that modern chips can handle it. The Rubin GPUs are designed with higher temperature tolerances, using advanced packaging and materials that can survive prolonged exposure to 40°C coolant. That's a big deal. Most current data center equipment is rated for a maximum inlet temperature of around 32°C (90°F), and many operators keep it much lower. Pushing to 35°C or higher requires careful engineering of the entire system—not just the GPUs, but the memory, the networking gear, the power supplies, everything.

I talked to a data center architect who works at a major colocation provider (he asked not to be named because his company hasn't announced its plans yet). He told me, "Running at 35°C is doable, but it means we have to be much more careful about air flow and hot spots. One failed fan in a rack and you could have a catastrophic failure. Liquid cooling mitigates some of that, but it introduces its own failure modes—leaks, pump failures, clogged filters."

That's the thing about liquid cooling: it's not simpler than air cooling, it's just different. It requires more maintenance, more sensors, and more sophisticated control systems. A leak in a liquid-cooled system can destroy hundreds of thousands of dollars worth of hardware in minutes. Nvidia's reference design includes redundant pumps and leak detection, but it's still a risk.

Is This Really About Water, or About PR?

Let's be honest for a second. Nvidia is facing increasing scrutiny over the environmental impact of AI. Training a single large language model like GPT-4 is estimated to produce as much carbon dioxide as 300 round-trip flights between New York and London. The water consumption is equally alarming. By highlighting water savings, Nvidia is trying to get ahead of the narrative before regulators step in.

But here's the thing: the math actually works. If every new data center built in 2025-2027 used Nvidia's higher-temperature liquid cooling design, the total water savings would be enormous. According to a 2024 study by the Uptime Institute, data centers in the US consume about 100 billion gallons of water per year. That's roughly 0.3% of total US freshwater consumption. It's not the biggest problem, but it's concentrated in areas where water is already stressed—Northern Virginia, Silicon Valley, Phoenix, and Dallas.

Nvidia's approach isn't the only game in town. Microsoft has been experimenting with underwater data centers (Project Natick, which I've written about before). Google uses machine learning to optimize cooling in its data centers, reducing energy use by 30% in some cases. Meta has built data centers in the Arctic Circle to take advantage of cold air. But Nvidia's approach is different because it doesn't require a specific geography. You can build a high-temperature liquid-cooled data center in Phoenix or in Helsinki. It works everywhere.

The Rubin Generation: What We Know So Far

Nvidia hasn't released full specifications for the Rubin architecture yet, but we know a few things. It will use a new GPU design with HBM4 memory (the next generation of high-bandwidth memory, which is critical for AI workloads). The interconnect will be faster, allowing thousands of GPUs to work together on a single model. And crucially, the thermal design power (TDP)—the maximum heat the chip can generate—will be higher than current generations. The H100 had a TDP of 700 watts. The B200 (Blackwell) is expected to be around 1000 watts. Rubin might be even higher.

That's a lot of heat. To put it in perspective, a typical household space heater runs at 1500 watts. A rack of 64 Rubin GPUs would generate as much heat as 42 space heaters running full blast in a closet. You can't cool that with air. You need liquid.

Nvidia's reference design includes a complete cooling solution: cold plates, coolant distribution units, heat exchangers, and a dry cooler for the final heat rejection. They're essentially saying, "Here's how you should build a data center for our chips." That's unprecedented. Usually, Nvidia just makes the chips and lets system integrators figure out the cooling. But with Rubin, they're offering a turnkey solution. It's a smart move—it reduces the risk for customers and ensures that the cooling system is optimized for the chips.

What This Means for the Rest of Us

If you're not building a data center, why should you care? Because AI isn't going away. The models are getting bigger, the training runs are getting longer, and the inference workloads (when you actually use the model) are becoming more common. Every time you ask ChatGPT a question or use Google's AI search, you're consuming compute resources that generate heat. If Nvidia can make that heat less environmentally damaging, it's a win for everyone.

But there's a deeper question here: should we be building more data centers at all? Some environmentalists argue that we should focus on making AI models more efficient rather than building bigger infrastructure. They have a point. But the market is speaking, and the market wants more AI. Nvidia is trying to make that growth sustainable.

I'll be watching the Rubin rollout closely. If the technology works as advertised, it could set a new standard for data center design. If it fails—if the higher temperatures cause reliability issues or if the liquid cooling systems prove too complex—it will be a costly lesson. But one thing is clear: the era of air-cooled data centers is ending. The future is liquid, and it's going to be hot.

The Bottom Line: A Necessary Evolution

Nvidia's hotter-running data center design is a pragmatic response to a very real problem. Water is becoming a scarce resource in many parts of the world, and data centers are competing with agriculture and residential use for every drop. By eliminating evaporative cooling and running at higher temperatures, Nvidia is reducing the environmental footprint of AI infrastructure.

Is it perfect? No. There are still concerns about the embodied carbon in the cooling equipment, the energy used by pumps, and the risk of leaks. But it's a step in the right direction. And given the explosive growth of AI, we need all the steps we can get.

I'm reminded of something a data center operator told me years ago: "The best cooling system is the one that uses the least energy and the least water, while keeping the equipment alive." By that measure, Nvidia's Rubin design looks promising. But we won't know for sure until the first facilities come online in 2026. Until then, I'll be watching, and I'll be asking the hard questions. Because that's what good tech journalism is about.

So here's my final question: if Nvidia can eliminate most water use from AI data centers, what's the next environmental challenge they need to tackle? The energy consumption? The carbon footprint of chip manufacturing? The e-waste from rapid GPU upgrades? The industry has a long way to go. But at least they're starting to have the right conversations.

---

Originally reported by www.theverge.com. Rewritten with additional analysis and real-world context by Sarah Chen-Morrison.