How does LLM cost per token work?

LLM providers charge per token for both input (what you send) and output (what the model generates), usually with output priced higher than input. Frontier models cost more per token than smaller models. Because agents send large contexts and make many calls, per-token pricing that looks cheap in isolation compounds quickly at agent scale.

How do you reduce AI agent costs?

Make spend visible with per-call attribution, route simple tasks to cheaper models, cache repeated prompts, trim context and tool schemas, and bound retries. Most teams that measure their agent spend find a large share of tokens are wasted and can be recovered without reducing capability.

How Much Do AI Agents Cost to Run? LLM Cost per Token in 2026

Q: How much do AI agents cost to run?

It depends almost entirely on token consumption. Agents cost far more than chatbots because a single agent task can trigger many model calls, tool invocations, and retries. Costs are driven by how many tokens flow through the models, which model tier handles each call, and how much of that consumption is wasted on verbose context and dead-end reasoning.

The question with no sticker price

“How much do AI agents cost to run?” is the question every team asks after the first pilot and before the second. It has no clean answer, because unlike a SaaS subscription an agent has no sticker price. Its cost is a function of how it behaves — and agents behave in ways that are hard to predict and easy to underestimate.

What can be answered clearly is the shape of the cost: what drives it, why agents multiply it, where the waste hides, and how to bring it under control. That is what this post does. The one-line version: LLM cost per token looks trivial in isolation and becomes the dominant line item at agent scale, because agents consume tokens in volumes that chatbots never approached. Understanding why is the first step to controlling it. For the broader risk picture, see enterprise AI agent deployment risks.

How LLM cost per token actually works

The unit of cost is the token — roughly a word-piece. Providers charge separately for input tokens (the prompt, context, and tool definitions you send) and output tokens (what the model generates), and output is typically priced higher than input. There is a further axis: model tier. Frontier models cost meaningfully more per token than smaller, faster models.

Two properties of this pricing matter enormously once you move from chatbot to agent:

It is linear in tokens but tokens are not linear in tasks. A harder task does not cost a bit more; it can cost a lot more, because the model reasons longer, the context grows, and more calls are made.
Input is not free. Teams fixate on output length, but for agents the input side — large contexts, long tool schemas, accumulated conversation history — often dominates. Every call resends context, so context bloat is a per-call tax.

Per-token pricing that reads as fractions of a cent feels negligible in a single call. The trap is that agents do not make single calls.

Why agents multiply the cost

A chatbot is one model call per user message. An agent is something else entirely. Give an agent a goal and it decomposes the task, selects tools, executes steps, handles errors, and iterates — and each of those steps is one or more model calls.

The multipliers stack:

Many calls per task. A single agent task routinely triggers a stack of model calls, not one. The number is often several times what the architecture diagram implies, once tool use and sub-agent delegation are counted.
Context grows with the task. Multi-turn agent work accumulates context, and because context is resent on every call, cost grows faster than linearly as a task runs long.
Tool schemas are a standing tax. The definitions that tell the agent what tools exist consume input tokens on every call, whether or not the tools are used.
Retries compound quietly. A modest tool failure rate combined with retry logic inflates real call volume well above the intended number.

The result is that agents consume many times more tokens than the chatbots they resemble on the surface — and the multiplier is largely invisible until the bill arrives.

Where the money leaks

Here is the part that should encourage rather than alarm: a large share of what agents spend is waste, not necessary work. Independent analysis and hands-on experience both point to the same conclusion — a substantial fraction of tokens consumed by production agents does nothing useful.

The common leaks:

Leak	What it looks like
Verbose context	Resending far more history and instruction than the task needs, on every call
Dead-end reasoning	The agent explores paths that lead nowhere, paying for every token along the way
Tool schema bloat	Large, always-on tool definitions consuming input tokens the task never uses
Wrong model tier	Trivial classification or extraction routed to an expensive frontier model
Uncontrolled retries	Failed calls retried without limit, multiplying spend on already-failing work

Every one of these is recoverable without reducing what the agent can do. That is the crucial point: cutting agent cost is usually not about doing less — it is about removing waste.

What it costs to not measure

The reason cost overruns cancel so many agentic projects is not that agents are inherently unaffordable. It is that the spend is invisible until it is a crisis. Budgets for agent projects are routinely underestimated, and the underestimate is discovered a month after launch, when the pattern of waste is already baked in and running.

The teams that survive are not the ones spending the least. They are the ones spending visibly. When every call is attributed to a feature and a team, waste becomes obvious and fixable. When it is not, the same waste hides inside a single aggregate bill that no one can decompose.

How to bring agent cost under control

The optimisation playbook is well understood, and teams that apply it typically recover most of their wasted spend. The moves, in rough order of leverage:

Instrument cost per call first. You cannot optimise what you cannot see. Attribute every call to a feature, user, and team before touching anything else.
Route by difficulty. Send simple, high-volume tasks to smaller models and reserve frontier models for genuine reasoning. This alone often produces the largest single saving.
Cache repeated prompts. Stable context that appears across many calls should be cached rather than resent and re-billed.
Trim context and schemas. Send the model what the task needs, not everything you have. Prune tool definitions to what is actually reachable.
Bound retries. Cap retry loops and treat a rising failure rate as an alert, not a cost to absorb.

Done together, these routinely cut agent spend by a large margin while leaving capability untouched. The prerequisite for all of them is visibility.

Visibility is the whole game

Every optimisation above depends on one thing: knowing where the tokens go. That is precisely what AI FinOps tooling provides — drop-in Python and Node.js SDKs that wrap your existing Anthropic and OpenAI calls with per-call attribution, anomaly detection, and budget alerts, without changing your architecture. The instrumentation is the hard prerequisite; the optimisations follow from it.

So, how much do AI agents cost to run in 2026? As much as you let them — and far less than you fear, once you can see the bill line by line. The teams treating cost as an afterthought are the ones getting cancelled. The teams treating it as a first-class metric are the ones quietly shipping agents that stay within budget.

If your agent spend is climbing and you want to find where the tokens are going, email atin@aivyuh.com or book a 30-minute call.

Part of AI Vyuh’s coverage of the AI agent economy — security, code quality, cost, and provenance for teams deploying AI agents in production.