How Can We Meet AI’s Insatiable Demand for Compute Power?

Of all the breathtaking observations about AI, few exceed this one: The growth rate for AI’s compute demand is more than twice the rate of Moore’s law (see Figure 1).

For decades, Moore’s law—that is, the number of transistors on an integrated circuit doubles about every two years—has been the unrivaled measure of technological progress. If you apply its rate of progress to other fields of endeavor, the results are equally mind-blowing and amusing. For example, an automobile with Moore’s law applied would travel at 300,000 miles per hour, achieve 2 million miles per gallon of gas, and cost four cents.

Now, AI’s compute demand—that is, the number of computations that must be performed to support evolving models—has grown at twice that rate over the past decade. With continued growth of these models and more adoption of AI by enterprises, Bain’s analysis suggests that the total global compute requirements could reach 200 gigawatts by 2030. In the US alone, total demand could reach 100 gigawatts by that time, which would increase new electricity demand on a grid that has seen relatively flat load growth for the past 20 years.

What are the implications of this for technology executives charged with allocating capital and managing investments? If you bet on continued growth and add lots of power generation or compute capacity while the trend slows down, you could be stuck with catastrophic unutilized power and compute capacity. If you bet that the trend will slow while it turns out to be durable, you may find yourself with insufficient capacity to capture a wave of growth and market share.

What could change the trajectory?

As companies form their strategic plans and determine investments, they should be considering four critical factors that could either impede or accelerate this growth: unaffordable economics, better graphics, technological breakthroughs, and supply chain shortages.

The economics become unaffordable. Bain’s research suggests that building the data centers with the computing power needed to meet that anticipated demand would require about $500 billion of capital investment each year, a staggering sum that far exceeds any anticipated or imagined government subsidies. This suggests that the private sector would need to generate enough new revenue to fund the power upgrade. How much is that? Bain’s analysis of sustainable ratios of capex to revenue for cloud service providers suggests that $500 billion of annual capex corresponds to $2 trillion in annual revenue.

What could fund this $2 trillion every year? If companies shifted all of their on-premise IT budgets to cloud and also reinvested the savings anticipated from applying AI in sales, marketing, customer support, and R&D (estimated at about 20% of those budgets) into capital spending on new data centers, the amount would still fall $800 billion short of the revenue needed to fund the full investment (see Figure 2).

The algorithms get better. When mathematical problems grow too complex or costly, step-change progress often comes not from incremental tuning but rather from entirely new algorithmic approaches. Innovations such as MapReduce (which popularized distributed data processing) and the Transformer architecture (which unlocked a more efficient way to process sequential data) exemplify this kind of breakthrough. Even at smaller scales, algorithmic innovation can unlock meaningful gains. In foundation models, techniques such as mixed-precision matrix computation improve training and inference efficiency. Logical methods, such as chain-of-thought prompting or large model distillation, boost performance while lowering computational load. DeepSeek is a recent example, pushing the compute efficiency frontier through smarter algorithmic design. Still, even with these innovations, the path forward requires a significant increase in infrastructure to reach the 100 gigawatts of additional compute power that will be needed.

Technological breakthroughs change the landscape. History is replete with unexpected leaps of progress in computational power. Sixty years of Moore’s law progress in semiconductors has given us handheld devices that far outperform the most powerful computers of the 1970s. Many speculate that quantum computing, for example, could displace the favored semiconductors trajectory of today, reducing the compute and power demands of tomorrow’s systems. Bain’s research suggests we are at least 10 to 15 years away from quantum computers stable enough to replace generative AI training and inference workloads. Other technological breakthroughs could include specially designed training and inference application-specific integrated circuits (ASICs), which could be more efficient than general purpose graphics processing units (GPUs), or new forms of memory or advanced packaging to improve power efficiency.

Shortages hit the supply chain. It will be difficult to build data centers fast enough to meet rising demand given constraints in four areas: power supply, construction services, compute enablers (such as GPUs), and the limited supply of data center equipment (including electrical switchgear and advanced cooling). Of these, increasing the supply of electricity may be the most challenging as bringing new power generation, transmission, and distribution online in a highly regulated industry can take four years or longer.

While no single issue will solve this deep challenge, innovation, government support, and efficient markets are all factors that could help close the gap. AI has the potential not only to improve productivity but also to spur the development of new businesses and revolutionary technological advancements in areas as diverse as drug discovery, autonomous vehicles, and logistics. Such large shifts frequently unlock new value in the economy, and they could help produce the revenue needed for the necessary capex. However, without such innovations or breakthroughs, general progress could slow, and the field could be left to only those players in markets with adequate public funding.