Tokens per megawatt-hour: the energy economics of frontier AI

For frontier AI companies, the labs training the most capable general-purpose models, infrastructure is as much the business as software. Training builds a model on a cluster of AI chips over weeks. Once trained, inferencing is a separate workload: each request arrives as input tokens, and the model generates output tokens one at a time until the answer is done. Labs charge for tokens per million. Data centres are sized by the power they can deliver to their computers, in megawatts or, for large ones, gigawatts.

A public count of that infrastructure comes from Epoch AI, an independent research group that tracks frontier data centres from satellite imagery, permits and filings. Its September 21, 2026 snapshot estimates 13.1 GW of operational IT capacity across 86 sites, with another 22.3 GW projected on those sites. Announced plans are larger still. OpenAI has announced chip agreements, some as letters of intent, for 26 GW: 10 GW with NVIDIA, 6 GW with AMD and 10 GW with Broadcom, all in 2025, and in 2026 an 8 GW campus in Pike County, Ohio, leased for 20 years under NVIDIA’s guarantees. Anthropic has signed about 13 GW: up to 5 GW with Amazon, nearly 1 GW of it online by the end of 2026; about 5 GW with Google and Broadcom from 2027; up to 2 GW of AMD chips from 2027; and up to 1 GW on Azure. In May 2026 it also took all of SpaceX’s Colossus 1, more than 300 MW, online within the month.

In energy terms, the IEA puts data-centre electricity use at 485 TWh in 2025, up 17% in a year, and expects it to roughly double to 950 TWh by 2030, about 3% of world demand. AI is the fastest-growing part: in the IEA’s April 2026 Base Case, AI-focused data centres reach about 465 TWh by 2030, more than three times 2025 and half of all data-centre use. Its 2035 cases for all data centres range from 942 to 1,637 TWh. Globally, data centres account for slightly less than a tenth of electricity demand growth to 2030. In the US they account for a third: Lawrence Berkeley National Laboratory’s June 2026 update puts US data centres at 649 TWh by 2030, 11.8% of US electricity, with AI servers drawing 55% of it. The IEA’s Base Case has 426 TWh for the US; the laboratory attributes the gap mainly to different assumptions about AI chip shipments.

The money is committed ahead of the megawatts. Amazon, Alphabet, Microsoft and Meta spent about $166 billion on capital in the second quarter of 2026 alone, and Meta expects $130 to 145 billion for the year. Contracted backlogs run larger still: Microsoft reports $678 billion and Oracle $664 billion of future revenue already booked across their cloud businesses, and Amazon’s filings record more than $200 billion of new compute commitments from OpenAI and Anthropic this year.

Building it is expensive. Chips, networking and the building come to about $40 million per megawatt, most of it chips. One filed benchmark for the campus side: SB Energy expects about $178 billion of capital spending to build its 8.8 GW of contracted campuses, leased mostly to OpenAI, roughly $20 million per megawatt of IT, excluding land. At the 10.2 kW rating of an eight-GPU DGX H100 server, one megawatt of server capacity holds 784 of NVIDIA’s H100 chips, or fewer of the newer, more powerful ones. Epoch’s record for IT power at a single site has doubled roughly every ten months, from 237 MW in 2024 to 946 MW today. That site is xAI’s Colossus 2 in Memphis, whose newer chips do the work of about 1.1 million H100s.

A megawatt-hour is an abstraction until you know what one answer costs. A short chat answer from a frontier model can use about 0.3 watt-hours, roughly one second of a microwave oven. One detailed estimate comes from a 2026 Microsoft study published in the journal Joule. Its frontier medians come from three open models above 200 billion parameters, DeepSeek-R1, Llama 3.1 405B and Nemotron Ultra 253B, on H100 servers: assumed node power and throughput fitted to published benchmarks determine the energy allocated to each answer, plus the overhead for cooling and the rest of the facility.

Input tokens are missing from the equation because it is an output-dominated approximation for short prompts. Prefill reads the whole prompt in one pass, while decoding writes the answer token by token, so the authors approximate a query’s energy by its output length and hold the prompt at 500 tokens. A longer prompt still costs something: it lowers the throughput term and raises the energy per answer. The authors flag the limit themselves. Prefill is under-modelled for long contexts, at 100,000 input tokens it would be substantially higher, and orchestration, tool calls and context management sit outside the study. Most agent input is not new, either. Serving systems keep the processed prompt in a cache, so the next turn reads it back instead of computing it again, and production traces show 90 to 96% of agent input tokens arriving as cache hits. The agent case below therefore counts cached and fresh input separately, at ratios taken from ML.ENERGY’s phase-split measurement and Irminsul’s cache-hit energy.

Chat is cheap in energy; reasoning costs about 13× more in the paper. Across its models and query distributions, chat with a median 300 output tokens uses a median 0.31 Wh, and reasoning with 5,000 uses 3.91 Wh. The 2.9 Wh per ChatGPT query from de Vries (Joule, 2023) used non-production assumptions; Oviedo et al. put such estimates 4 to 20× too high. A megawatt-hour, a thousand units on a power bill, corresponds to about a billion output tokens in this worked H100 case. The actual rate depends on request shape and the serving stack: long contexts lower throughput, larger batches raise it, and tight latency targets cap it. MLPerf measures throughput per system at a stated latency target; the figure here is back-solved from the paper’s medians. These figures describe large generative models; recommender, vision and classical machine-learning workloads use different amounts of energy per task. The paper’s public code samples PUE from 1.05 to 1.6, wider than the 1.05 to 1.40 its text states; at the code’s median of 1.30 the back-solved throughput would be 7% higher and the tokens per megawatt-hour unchanged.

Provider figures are in the same ballpark. Google’s full-stack measurement of a median Gemini Apps text prompt is 0.24 Wh in May 2025, of which active accelerators account for 0.14 Wh; its June 2026 environmental report repeats the figure rather than re-measuring. Sam Altman put the average ChatGPT query at 0.34 Wh in June 2025, without stating a method, and OpenAI has published nothing since. The independent academic ML.ENERGY benchmark finds reasoning responses at 25× the energy of chat. Its longitudinal analysis shows why throughput and energy diverge: fifteen months of serving-stack upgrades gave 3 to 5× more tokens per second on the same H100s but only 15 to 41% less energy per token, because GPU power rises with utilisation.

The agent row uses the reasoning calibration and adds input energy at assumed ratios informed by other serving measurements: an uncached input token at one-tenth of an output token, near the median phase split ML.ENERGY measures for Llama models on H100s, and a cached read at a quarter of that, from the 14 to 37% of prefill energy a cache hit costs in Irminsul’s 4,096-token tests. Applying those to an 80,000-token frontier context is an extrapolation. Moving the cache-hit rate between 80 and 95% and the two input ratios across their measured ranges puts the same task between about 65 and 400 Wh, so 153 Wh is a central case, not a measurement. The 153 Wh includes node and facility overhead, so it is not directly comparable with GPU-only measurements of agent episodes, which span about 3 to 34 Wh across configurations in a metered SWE-bench study. The task shape is illustrative, informed by production traces of coding agents from GitHub Copilot and TraceLab.

Training is the other big use of power, and some of it is publicly disclosed. Meta’s model card discloses 30.84 million H100 GPU-hours for Llama 3.1 405B. At the chips’ 700 W rating, that implies 21.6 GWh, or about 35 GWh with assumed node and cooling overhead of 1.6×. At an assumed $2.50 per GPU-hour, the rental-equivalent compute cost is $77 million; the estimated electricity, at $92 per MWh, is $3.2 million. It remains the largest fully disclosed run: Meta’s Llama 4 card gives 7.38 million GPU-hours across two smaller models, and no frontier model released in 2026 discloses its hours.

These sites are gigawatt-scale. Stargate Abilene is planned at about 1.2 GW, roughly 10 TWh a year at full draw, with 421 MW of IT running there in September 2026. A training run holds a flat load for weeks and can pause at a checkpoint. Interactive inference follows users, so it cannot pause. According to the Joule authors, the pressure on grids mostly comes from the three things mentioned below, and not from the energy used to give a single answer:

• Training loads: a run draws near-full power for weeks and arrives as one large block, not as gradual growth.
• Pace of adoption: demand is growing faster than transmission can be planned, permitted and built.
• Concentration: gigawatts land at a handful of substations rather than spreading across the network.

Take one megawatt of server capacity and add up what it costs for a year. Filled with H100 servers, rated at 10.2 kW for eight GPUs each, that is 98 servers and 784 chips. The servers average an assumed 70% of that rating, so 0.7 MW, and PUE 1.21 adds cooling and the rest of the building to make 0.847 MW at the meter. Over 8,760 hours that is 7,420 MWh a year. Own it and depreciation dominates the annual budget.

At $50 per MWh electricity is 5% of the total; at $120, 11%. A B200 megawatt costs about the same, near $8 million, with 546 GPUs at 1.83 kW of system power each and $45,000 apiece, though each of those chips does several times the work. Facility overhead varies too: Google’s fleet averaged PUE 1.09 in 2025, against the 1.21 assumed here. Renting at assumed on-demand rates of $2.50 to $4 per H100-hour, a range that brackets Lambda’s September 2026 list price of $3.99, would cost $17 to 27 million a year. For context, CoreWeave’s second-quarter 2026 revenue of $2.58 billion comes to about $6.9 million a year per megawatt of the 1.5 GW it had active at the end of June, close to the $7.7 million a year our budget says a megawatt costs to own; it added nearly 500 MW during the quarter, so revenue per megawatt actually in service was higher. Its second-quarter net interest expense of $640 million, annualised, adds roughly $1.7 million per megawatt-year, the financing our budget leaves out. These are company-wide ratios, not H100 rental quotes.

Spread the annual budget over output and the unit cost appears. At an assumed 75% of the calibrated throughput, the H100 case produces 5.4 trillion output tokens a year. Dividing $7.68 million by that output gives about $1.43 per million output tokens. This is a simplified accounting cost; a full levelized cost would also include financing and the other omitted costs.

Revenue is what a megawatt earns by running.

At the list prices of Anthropic’s Sonnet 5, the worked H100 case would sell for about $72 million a year. That assumes three-quarters productive output, selling output tokens at $10 per million and the accompanying 500 input tokens per 300 output at $2. Free tiers, subscriptions, cached input and discounts change what is collected. Dividing reported run-rates, which annualise a recent period’s revenue, by an assumed 2.5 GW of live compute gives a different view in the top-down rows. Anthropic said only that run-rate revenue crossed $47 billion in May. Bloomberg reported in August that Anthropic passed $65 billion at the end of July and that OpenAI reached $40 billion; TechCrunch summarises both reports. Two cautions. Anthropic books the full token price paid through Amazon’s Bedrock marketplace as its own revenue and pays Amazon separately, so the run-rate includes money that goes to the cloud (SemiAnalysis, May 2026), and the two labs may not define run-rate the same way. Both figures also cover whole companies on mixed hardware, so dividing by assumed capacity gives a scenario, not what an inference megawatt earns.

Filed figures are thinner, because both labs’ IPO drafts are confidential. The clearest sit on the other side of the ledger: Microsoft’s audited accounts show $24.1 billion of revenue from OpenAI in the year to June 2026, and Amazon’s July filing records compute commitments expanded by $100 billion over eight years from OpenAI and by more than $100 billion over ten from Anthropic. Per megawatt, IREN reports that its recent three-year GPU contracts earn more than $20 million per megawatt of IT a year, about three times CoreWeave’s company-wide $6.9 million.

Break-even sits below that $1.43 because input tokens are sold too: $1.07 per million output tokens, once 500 input tokens per 300 output are charged at a fifth of the output price. Flash’s $3.75 price is 3.5× that threshold, though its actual serving cost depends on the model. Smaller models can support lower prices. In the company-ratio scenarios, electricity is 4.3% of revenue for OpenAI and 2.6% for Anthropic at 2.5 GW, or 1.7% at 1.6 GW.

The Sonnet list-price scenario implies an 89% margin over the simplified budget; the batch row implies 79%, and renting the chips at Lambda’s $3.99 per GPU-hour list price instead of owning them, 62%. Reported lab margins use different cost boundaries: Anthropic’s inference gross margin was in the mid-sixties per cent in early 2026, by SemiAnalysis’s estimate, and OpenAI’s compute margin on paid products, revenue minus the cost of running its models, was reported at about 70% in October 2025 (The Information, via SaaStr). By May 2026 Anthropic’s gross margin was reported at about 70% on first-quarter results (SaaStr, third-party). Free tiers, flat-rate subscriptions, discounted cached and batch tokens, enterprise deals, cloud rental costs and differences between models all affect the outcome.

A healthy margin on serving does not settle company profitability. Training the next model, research compute and researchers’ pay must also be funded, and capacity is bought ahead of demand. The two labs now differ here: Bloomberg reported in August that Anthropic was profitable on an adjusted operating basis in the second quarter of 2026, on more than $11.5 billion of revenue, while OpenAI said in July that its planned infrastructure spending through 2030 had reached $750 billion (Wall Street Journal, via TechCrunch). The scale helps explain the capital raises: OpenAI closed a $122 billion round in March 2026, and Anthropic raised $65 billion in May.

An hour of the worked megawatt uses $78 of electricity and earns $1,800 to $3,000 in the company-ratio scenarios. The expensive input is time without power. On a 100 MW campus, a month’s delay puts $133 to 217 million of sales at risk, if that work cannot be made up elsewhere or later. For a real project the figure to use is smaller: the margin on that work, net of alternatives such as renting capacity or routing traffic elsewhere. A whole year of cheaper electricity, $50 instead of $120 per megawatt-hour, saves about $52 million. Fixed charges of $58 million a month run either way, so they are a separate comparison, not an extra loss.

And the waits are long. Building a data centre takes one to three years; new grid infrastructure can take five to fifteen, according to the IEA. JLL puts the average wait for a grid connection in major markets at over four years. In Texas, ERCOT had more than 474 GW of large-load requests by August 2026, about 90% of them data centres, against a record peak demand of 86 GW. In August, Texas’s governor ordered an audit of every data centre in that queue before more approvals.

The shortage shows in prices. On PJM, the largest US grid market, the capacity price rose from $28.92 per MW-day for 2024/25 to its $325 cap for 2028/29, and PJM puts most of the load growth behind the 2027/28 auction down to large new loads. Buyers pay to get connected. Talen expects premium prices on its Amazon contract, AEP has data centres commit to pay for up to 90% of their contracted demand for as long as 20 years, used or not, and Entergy says Meta pays directly for the extra transmission at its Louisiana site.

How flexibility buys time

One way through the queue is to give the grid something back: using less power for a few hours on its hardest days. Whether a data centre can do that depends on its work. Training runs save checkpoints and can pause. Batch jobs, such as bulk document processing or agents working in the background, can wait hours; Anthropic and OpenAI sell them at half price for results within 24 hours. Real-time answers, in chat, search or a coding assistant, cannot wait.

Grids pay for this through demand-response programmes. One study of California’s grid, by Du et al., models a reward of $150 per MWh and a penalty of $375 for falling short. That is small next to what real-time work earns, but deferring training or batch work costs little if it catches up later, though checkpoints and catching up are not free.

It works, so far at small scale. Emerald AI, Oracle and NVIDIA cut the power of a 256-GPU cluster by 25% for three hours during two real peaks in Phoenix, while meeting service targets (Nature Energy). Google has written 1 GW of such flexibility into its utility contracts in return for faster connections. Texas law now requires many new large loads to be ready for curtailment. And Duke University estimates that if new loads accepted cuts for about 177 hours a year, around 0.5% of their energy, the existing US grid could take 98 GW more without new power plants, before transmission limits.

How much of a campus can flex depends on its mix. Half the training load and all batch work can step back; more real-time work lowers that share, and idle servers, networks and cooling keep drawing regardless. Batteries cover the rest. Grid events usually last a few hours, and Chen and Zheng find the grid gains little from deferrals beyond three. On a 100 MW site where 25 MW can pause, a call to cut a quarter needs no battery; a call to cut half needs 25 MW of storage for three hours, or 75 MWh, before losses and reserves. The IEA expects 20 to 25 GW of batteries and 15 to 27 GW of on-site gas at data centres by 2030.

Energy per token and cost per token share a denominator: the tokens a megawatt-hour produces. A lever that doubles throughput halves both, from the same power and the same $7.68 million. The Joule authors put the combined opportunity at 8 to 20× against today’s energy per query. They assess each category separately, then combine selected gains rather than multiplying the full ranges:

• Models, 1.5 to 10×: distillation, training a smaller model on a larger one’s outputs, has cut energy 5 to 10×. Quantisation, running the model at lower numerical precision, adds more, as do mixture-of-experts designs that fire only part of the model for each token and controls that make reasoning models write up to 40% fewer tokens. Small models use one to two orders of magnitude less energy on narrow tasks such as maths and coding.
• Serving software, 1.5 to 5×: scheduling to each service’s latency target, caching, splitting prompt reading from answer writing, a small model drafting tokens for the large one to check, and routing each request to the smallest model that can handle it.
• Chips and facilities, 1.5 to 2.5×: newer chips such as NVIDIA’s Blackwell (B200/GB200); liquid cooling; and power capping. NVIDIA’s MLPerf v5.1 results from September 2025, the last round to run Llama 3.1 405B on both generations, put GB200 NVL72 at 3.0× the per-GPU throughput of an eight-GPU H200 system offline and more than 5× on the interactive test. For Vera Rubin, NVIDIA projects up to 10× more tokens per megawatt than GB200 NVL72 on one reasoning model; its first MLPerf preview in September 2026 posted up to 2.5× GB300 NVL72 per GPU on DeepSeek-R1. These are single benchmarks, not universal multipliers.

The levers work in two ways. Better chips, serving software and smaller models raise throughput, cutting energy and cost per token together. Shorter reasoning and caching instead cut the tokens each task needs, so the cost per task falls while the cost per token stays put. The 8 to 20× mixes both, so it is not all extra tokens from the same machine. AI is also starting to do some of the research behind these gains, which could speed up the models lever; that research runs on compute too.

Utilisation moves cost alone. About nine-tenths of the budget is depreciation and other fixed charges, so an idle megawatt costs nearly as much as a busy one. At half utilisation instead of three-quarters, the $1.43 per million output tokens rises to roughly $2, while energy per token barely moves.

None of these levers lowers what a megawatt costs; they lower the cost of what it produces. Whether that becomes revenue depends on price. With throughput doubled, the Sonnet-price case earns $144 million a year if prices hold, the same $72 million if they halve, and less if they fall further, against an unchanged $7.68 million budget. Revenue per megawatt rises only when throughput grows faster than prices fall, and when demand grows enough to use the extra tokens. So far it has. Google cut energy per prompt 33× between May 2024 and May 2025, mostly through model changes, yet it reported over 3.2 quadrillion tokens a month at I/O 2026, 7× a year earlier, and the IEA puts the growth in AI-focused data-centre electricity use at 50% in 2025. Cheaper tokens have filled megawatts rather than freed them, which keeps power the constraint.

Efficiency alone will not shrink the footprint: with data-centre demand set to double by 2030, whether that load is clean is a question of its own.

1. 01Electricity is a small cost share. It is 9% of the ownership budget at $92 per megawatt-hour, and 1.7 to 4.3% of the company-ratio revenue scenarios.
2. 02Megawatts on a date matter. A month’s delay outweighs a year of cheaper power, $133 to 217 million against $52 million on a 100 MW campus, when the delayed work is lost.
3. 03The flexible share depends on the work. Training and batch work can step back; real-time answers cannot. That flexibility is what buys a faster connection, and batteries and on-site generation cover what the workload cannot shed.

Frontier AI has made electricity a small line on a very large bill. What a premium still cannot guarantee is a connection on time, and that is where the competition has moved.