What an LLM API actually costs to run

Ask what an LLM API costs and you get a number that means nothing on its own - "a few dollars per million tokens." That is a unit price, not a bill. What you actually pay depends on four things the pricing page never puts in one place: the model tier you choose, how many tokens each call burns, how many calls you make, and whether you use the discounts sitting in plain sight. Here is how an LLM API bill is really built in 2026, driver by driver, with a worked monthly estimate, so you can size your own before a proof of concept quietly turns into a four-figure invoice.

Why "how much does the API cost" has no single answer

Providers price large language models per token, not per request or per seat. A token is roughly three-quarters of a word, so a page of text is about 500 tokens. Every model has two rates - one for the tokens you send in, one for the tokens it generates - quoted per million tokens. That sounds simple until you realise the bill is a product of four independent variables, and changing any one of them can move your monthly cost several-fold:

• Model tier - which model you call. The widest-spread lever: frontier models cost multiples of small ones per token.
• Tokens per call - prompt plus context plus the generated answer. The line teams underestimate.
• Call volume - how many times a day your app hits the API. Linear, and easy to forecast wrong.
• Discounts used - prompt caching and batch pricing. The free money most teams leave on the table.

Get those four straight and every provider's pricing page becomes readable. Miss one - usually context or caching - and your proof of concept looks cheap while your production bill does not.

Driver 1: the model tier - the widest lever you control

Every major provider ships a family of models at different price points, and the spread inside one family is large. A top-tier frontier model typically costs on the order of five to ten times a small, fast model per token - and the small model is often good enough for classification, extraction, routing, and simple replies. Picking the model is the single biggest cost decision you make, and the instinct to reach for the most capable model for everything is what blows up bills.

The pattern that wins in production is tiered routing: send the easy, high-volume calls to a cheap fast model and reserve the expensive frontier model for the genuinely hard reasoning. A support assistant might classify and route 90 percent of messages on a small model and escalate only the 10 percent that need deep reasoning - cutting the model bill by more than half with no drop in answer quality where it matters. If you are weighing which model fits which job, our head-to-head on Claude vs GPT-4o vs Gemini for business compares them on capability, not just price - the two decisions go together.

Driver 2: tokens per call - and why output is the expensive half

Each API call bills you for two things: the input (everything you send) and the output (everything the model writes back). These have different rates, and on most frontier models output costs roughly four to five times the input rate. That single fact reshapes how you budget:

• Long prompts are cheaper than long answers. A big system prompt or a stuffed context window is billed at the cheaper input rate. A model that rambles for a thousand tokens when a hundred would do is billed at the expensive output rate - so cap output length and ask for concise responses.
• Context is re-billed every single call. This is the tax teams forget. Your system instructions, the retrieved documents in a retrieval-augmented app, and the running chat history are all input tokens - and they are re-sent and re-billed on every request. A 4,000-token context sent on a million calls a month is 4 billion input tokens of pure overhead before the model answers anything.
• Reasoning and thinking tokens count. Models that "think" before answering generate internal tokens you pay for as output, even when they are summarised or hidden. More thinking means better answers and a bigger bill - a real trade-off to tune, not ignore.

The practical move is to measure, not guess. Count the tokens in a representative call - prompt, context, and a typical answer - and multiply by the two rates. That one calculation, done honestly, is worth more than any pricing-page skim.

Driver 3: call volume - the multiplier that turns cents into invoices

Per-call costs look trivial in a demo. A single call might cost a fraction of a cent. The bill is that fraction multiplied by your real volume, and volume is where estimates go wrong. Ten thousand users each making twenty calls a day is 200,000 calls a day, six million a month - and a "cheap" call at half a cent is thirty thousand dollars a month. The economics of an LLM feature live in the volume column, not the unit price.

Two things inflate volume quietly. First, chatty agent loops: an autonomous agent that makes five tool calls to answer one question multiplies your call count by five. Second, retries and multi-step chains that re-send context each hop. Before you commit to an architecture, model the call count end to end - our breakdown of what a custom AI agent really costs to build and run shows how the agent loop, not the model rate, is usually what drives the bill.

Driver 4: the discounts most teams never turn on

Two provider-side discounts routinely cut a real bill by a large margin, and both are opt-in - you get nothing unless you build for them:

• Prompt caching. If the front of your prompt is stable - a long system prompt, a fixed set of instructions, a document you ask many questions about - the provider can cache it and charge the repeated part at roughly a tenth of the input rate on later calls. For any app that re-sends the same big context on every request (most retrieval and agent apps), caching is the highest-leverage cost fix there is. It needs the stable content at the front of the prompt and the volatile content at the end - a design choice, not a toggle.
• Batch processing. For work that does not need an instant answer - overnight summarisation, bulk classification, embedding a back-catalogue - most providers run a batch endpoint at about half the standard price. If a workload can wait an hour, batching halves its cost for nothing but a queue.

These are not micro-optimisations. On a context-heavy production app, caching plus batching can be the difference between a bill you sign off and one you escalate. If your workload runs on AWS, the same token math shows up one layer down - our note on AWS Bedrock pricing explained walks the same drivers through the Bedrock line items, including provisioned throughput.

A worked monthly LLM bill

Take a mid-size support-assistant feature: 100,000 conversations a month, each averaging three calls (classify, retrieve-and-answer, follow-up). Each answer call sends a 3,000-token context (system prompt plus retrieved docs) and generates a 400-token reply. Illustrative, not a quote - your rates and mix will differ, but the proportions are the lesson:

• Volume: 100,000 conversations x 3 calls = 300,000 calls a month.
• Input tokens: most of the cost is the re-sent 3,000-token context on every answer call - the largest single line, and the one caching attacks.
• Output tokens: only 400 tokens per answer, but at four to five times the input rate they punch above their token count.
• Tier choice: run the classify step on a small model and the answer step on a mid-tier model, and you pay frontier rates on none of it.

Run those numbers naively - frontier model, no caching, no batching - and the same feature can cost several times what it costs built well. The lesson is the same as any usage bill: the unit price on the pricing page is the part you can almost ignore, and the token math - context length, output length, tier, and the discounts - is the number worth engineering. Always price your real prompts against the live pricing page for the model you choose, because the rates move.

How to keep the LLM bill down without hurting quality

1. Route by difficulty. Send easy, high-volume calls to a small fast model and reserve the frontier model for hard reasoning. This is the biggest lever and rarely costs you quality where it matters.
2. Cache the stable prefix. Put your system prompt and fixed context at the front and let the provider cache it - later calls bill the repeated part at a fraction of the rate. Essential for retrieval and agent apps.
3. Cap and trim output. Set a sensible max output length and ask for concise answers. Output is the expensive half - a rambling model is a costly one.
4. Batch the non-urgent. Anything that can wait - overnight jobs, bulk processing - runs at roughly half price on the batch endpoint. Free money for a queue.
5. Measure before you scale. Count tokens on a real call and multiply by your true volume before launch. The demo bill and the production bill are different animals.

None of this is about spending less for its own sake. It is about matching each of the four cost drivers to the value it returns, so the LLM feature earns its keep instead of becoming a bill you resent - the same discipline we bring when we design and cost these systems, whether you run them on your own stack or as part of your AI development and agent build roadmap. And if that feature is a chatbot specifically, the build-versus-buy maths in our AI chatbot cost guide sits right on top of this token math.

FAQ

How much does an LLM API cost in 2026? It depends on four things, not one: the model tier (frontier models cost roughly five to ten times a small model per token), the tokens per call (prompt plus context plus output, with output billed four to five times higher than input), your call volume, and whether you use prompt caching and batch discounts. A small, well-routed feature can run tens of dollars a month; a naive frontier-model deployment at high volume can run into four or five figures. Model your real prompts and volume against the live pricing page.

Why is output more expensive than input? Generating tokens is more compute-intensive than reading them, so providers price output higher - commonly four to five times the input rate on frontier models. The practical effect is that long answers cost more than long prompts, so capping output length and asking for concise responses is a direct cost lever.

What is prompt caching and how much does it save? If the front of your prompt is stable across calls - a system prompt or a document you query repeatedly - the provider can cache it and bill the repeated part at roughly a tenth of the input rate on subsequent calls. For context-heavy apps that re-send the same large prompt every request, it is the single highest-leverage cost fix. It requires putting stable content first and volatile content last.

Is it cheaper to self-host an open model than to use an API? Sometimes, at high and steady volume - but you trade the per-token API rate for GPU, ops, and engineering cost, and the break-even is higher than most teams expect. For most workloads the managed API, with tiered routing and caching, is cheaper and far less work than running your own inference. It is a real decision to model, not a default.