How much energy does a single LLM API call consume? Data from Epoch AI, the IEA, and peer-reviewed research breaks down the real numbers by model size.
Every call to an LLM API draws power from a GPU cluster in a data centre somewhere. The question is how much power, and whether the commonly cited figures are even close to accurate.
For most of 2024, the number floating around was 3 watt-hours per ChatGPT query, roughly ten times the energy of a Google search. Epoch AI revisited this claim in detail, applying updated hardware specs and realistic token counts. Their conclusion: typical ChatGPT queries using GPT-4o likely consume roughly 0.3 watt-hours. Ten times less than the old estimate. The gap comes from more efficient models and hardware compared to early 2023, combined with an overly pessimistic assumption about how many tokens a typical query generates.
A separate peer-reviewed study published in Joule in April 2026 arrived at a similar figure through a different method. Researchers modelled frontier-scale inference (models with over 200 billion parameters) on H100 nodes and found a median energy of 0.31 Wh per query, with an interquartile range of 0.16 to 0.60 Wh. Their conclusion was direct: widely cited public estimates overstate AI energy consumption by 4 to 20 times.
So the per-query cost of a standard text call is small. Around 0.3 Wh. Less than running an LED lightbulb for a few minutes.
The problem is what happens when you multiply that number by billions of daily requests, and when you factor in the models that cost far more than 0.3 Wh per query.
Not all models cost the same to run. A benchmarking study from the University of Rhode Island tested 30 commercial LLMs and measured energy per query across short and long prompts. The spread is wide.
Stay in the sub-watt-hour range. A short GPT-4o query consumes approximately 0.43 Wh according to that study. GPT-4.1 nano requires only 0.454 Wh even for long prompts, making it one of the most energy-efficient options available.
A different category entirely. DeepSeek-R1 and o3 each consume over 33 Wh per long prompt — more than 70 times the consumption of GPT-4.1 nano. These models generate extended chains of thought, producing thousands of output tokens per query instead of hundreds. The Joule study quantified this directly: reasoning queries with approximately 5,000 output tokens raise energy use roughly 13 times compared to standard queries.
Sits in between. It supports chain-of-thought reasoning but consumes 17.045 Wh for long-form input, less than half the energy of o3. The Rhode Island study ranked it highest in overall eco-efficiency across all 30 models tested.
The math here matters for teams choosing which model to call. A workflow that routes 90% of queries to a lightweight model and reserves reasoning calls for the remaining 10% will consume a fraction of the energy compared to routing everything through o3. The Joule study modelled this scenario at data-centre scale: serving 1 billion queries per day requires approximately 0.7 GWh. If 10% of those are long reasoning queries, total demand rises to 1.7 GWh per day. More than double.
Individual queries are small. The aggregate is not.
The International Energy Agency published its "Key Questions on Energy and AI" report in 2026, building on its landmark April 2025 analysis. The numbers are stark. Global data centre electricity consumption grew by 17% in 2025. Electricity consumption from AI-focused data centres grew even faster, surging 50% in that same year. The IEA projects electricity consumption from data centres will double by 2030, with AI-focused facilities tripling.
Growth in global data centre electricity consumption in 2025
Growth in electricity from AI-focused data centres in 2025
Projected growth in AI-focused electricity consumption by 2030
Capital expenditure tells the same story from the investment side. The five largest technology companies exceeded $400 billion in capital expenditure in 2025. The IEA expects that figure to jump by another 75% in 2026.
The IEA also noted that energy efficiency per AI task is improving at a rate unprecedented in energy history, with energy use per AI task dropping by at least an order of magnitude annually. Simple text queries now consume less electricity than running a television over the same period of time. But new energy-intensive applications (video generation, reasoning, agentic workflows) consume hundreds or thousands of times more energy per query than simple text generation. Efficiency gains per task are real. They are also being outpaced by growth in usage volume and task complexity.
Understanding AI energy consumption at the component level helps identify where waste can be eliminated.
An LLM generates output in tokens. For OpenAI models, one token represents approximately 0.75 words on average. Generating each token requires roughly two floating-point operations (FLOP) per active parameter in the model. GPT-4o has an estimated 200 billion total parameters but uses a mixture-of-experts architecture, meaning only a fraction (roughly one quarter) are activated per token. That gives approximately 100 billion active parameters, requiring 200 billion FLOP per output token.
A typical query generating 500 output tokens needs around 100 trillion FLOP. Running that on Nvidia H100 GPUs (rated at 700 watts, but consuming up to 1,500 watts per GPU including server and data centre overhead) takes a fraction of a second. Epoch AI walked through this calculation in detail and arrived at their 0.3 Wh estimate.
Input tokens also cost energy, though less per token than output generation. Long context windows (uploading entire documents for summarisation, for example) add to the compute bill. The Joule study found that the gap between short and long prompts was significant but smaller than the gap between standard and reasoning models.
The overhead multiplier matters too. Raw GPU power draw is only part of the picture. Data centres add cooling, networking, storage, and power distribution losses on top. The industry-standard metric for this is Power Usage Effectiveness (PUE). A PUE of 1.2 means for every watt consumed by compute, an additional 0.2 watts goes to cooling and infrastructure.
Here is where the numbers become actionable for engineering teams.
Production AI applications generate a large volume of repeated or near-identical queries. Traffic analysis across production applications shows that 35 to 45 percent of prompts are either identical or semantically equivalent to previous requests. Every one of those redundant calls triggers the same GPU computation, draws the same power, and produces the same answer that already exists in a previous response.
A semantic caching layer intercepts these repeated queries before they reach the LLM provider. Instead of re-running inference, the cache returns the stored response. The energy cost of serving a cached response is negligible compared to running a full inference pass on an H100 cluster.
10,000 queries per day × 0.43 Wh each = 4.3 kWh of daily inference energy
40% cache hit rate = 4,000 queries served from cache
Energy saved = 1,720 Wh per day
Monthly avoided inference energy = 51.6 kWh
Scale that across an organisation running multiple AI products, or across the industry's billions of daily queries, and caching becomes one of the most direct mechanisms for reducing aggregate AI energy consumption. The Joule study identified model design, serving systems, and hardware as three pathways that could together reduce per-query energy by 8 to 20 times. Caching operates upstream of all three: it eliminates the query before any of those systems need to process it.
The 70× energy gap between GPT-4.1 nano and o3 means model routing is an energy policy, whether teams think of it that way or not.
Most production workloads do not need reasoning-class models for every request. A customer support chatbot answering common questions, a document classifier sorting incoming emails, a summarisation tool processing meeting notes: these can run on smaller, faster models that consume a fraction of a watt-hour per call.
Reasoning models earn their energy cost when the task demands multi-step logic, complex analysis, or extended generation. Using them for everything is the equivalent of running a diesel generator to charge a phone.
The practical approach is tiered routing. Classify incoming requests by complexity. Send the majority to efficient models. Reserve expensive reasoning calls for the queries that need them. The Joule study showed that even a modest 10% share of reasoning queries more than doubles total energy consumption at scale. Reducing that share from 10% to 5% through better routing cuts the energy surplus roughly in half.
Three interventions have measurable impact on AI energy consumption at the application level:
Semantic caching can eliminate redundant GPU computation for applications with repetitive query patterns. Implement it in your application or a cache layer designed for the reuse and retention constraints of the workload; SHIM does not provide a semantic response cache.
Match model capability to task complexity. Use lightweight models for straightforward tasks and reserve reasoning models for queries that justify the compute.
Prompt engineering that produces concise responses directly reduces energy per query. The relationship is roughly linear: half the output tokens means roughly half the inference energy.
These are not abstract sustainability commitments. They are engineering decisions that reduce both cost and energy consumption in direct proportion. A query that never reaches a GPU costs nothing to run and draws zero watts.
Epoch AI estimates that a typical GPT-4o query consumes approximately 0.3 watt-hours of electricity. A peer-reviewed study published in Joule in April 2026 found a similar median of 0.31 Wh for frontier-scale models. Both figures are roughly ten times lower than the commonly cited 3 Wh estimate from 2023.
Reasoning models like o3 and DeepSeek-R1 generate extended chains of thought, producing thousands of output tokens per query instead of hundreds. A University of Rhode Island benchmarking study measured these models at over 33 Wh per long prompt, more than 70 times the energy of lightweight models like GPT-4.1 nano.
Semantic caching stores responses to previous queries and serves them when a new query is semantically equivalent. Since the cached response requires no GPU inference, the energy cost drops to near zero for that request. Production applications typically see 35 to 45 percent of prompts repeat or closely match previous queries, making caching a direct way to cut inference energy.
According to the IEA's 2026 "Key Questions on Energy and AI" report, global data centre electricity consumption grew 17% in 2025, with AI-focused facilities surging 50%. The IEA projects data centre electricity consumption will double by 2030, with AI-focused consumption tripling.