AI agent costs: why price per token misses the finished task

The price tag is not the receipt

In a chat box, price per token feels like a useful number. You ask, it answers, you count the tokens. The router sees a price tag and selects the cheapest tier. That number never changes during the task. But the receipt does. One finished task accrues line items the price tag never mentions: a first attempt, the context re-read again and again, a schema error and a retry, tool calls that read files and run them and read again, a fallback to a stronger model when the cheap one stalls, and finally a human fixing the patch by hand. The price tag stays small while the receipt grows.

The mistake is not "cheap models are bad". That is lazy and false. The mistake is subtler: we optimise the cost of a step instead of the cost of finishing the job. Cheap tokens are not cheap if they do not finish. Route on the price tag and you have a price preference, not a cost strategy: a guess wearing the costume of a decision.

One scope note carries through everything below. The measured evidence comes from coding agents, where trajectories, tools, tests and success conditions are observable. The discipline very likely applies more broadly; the exact magnitudes do not.

Where an agent's cost actually goes

Chat is a line: prompt in, answer out. An agent is a loop: plan, call a tool, read what it returns, then fold the whole growing context back into the next request and do it again. The output stays a small visible sliver; the input side snowballs.

This part is measured, and the conditions matter. Bai and colleagues (2026) instrumented eight frontier models running on SWE-bench Verified, a benchmark of real GitHub issues, through the OpenHands agent framework: 500 instances, four runs per instance per model. In that setting they found that agentic coding is far more expensive than single-shot work; that input tokens, not output, dominate the overall cost, even with caching enabled; and that what looks like one task is really a long trajectory through setup, exploration, editing, validation and closeout.

Two of their numbers are easy to quote and easy to misquote.

The first is the headline multiplier. The paper's summary says agentic tasks consume about 1000x more tokens than code reasoning and code chat. Its own body splits that: about 3500x more than a typical single-round reasoning task, and about 1200x more than a multi-round chat. The summary compresses two different comparisons into one number. Use the split, and cite the conditions.

The second is run-to-run variance. The paper's abstract notes that runs on the same task can differ by up to 30x in total tokens. Its own body reports the typical spread across repeats as roughly 2x, the most expensive run against the cheapest. The 30x is the extreme tail, not the norm. An honest estimate has to carry both: usually about double, occasionally far more.

The rest of the measured picture holds up. Higher token use did not reliably improve accuracy; accuracy tended to rise with cost early and then saturate, with extra tokens going into repeated file views and edits rather than progress. And models were poor at predicting their own consumption: the strongest correlation the study found was 0.39, for output-token prediction by one model (Sonnet 4.5), with input-token prediction weaker across the board. The agent cannot price the journey before it walks it. That is the argument for a meter after the run, not confidence before it.

A complementary industry read points the same way. Cursor's Developer Habits Report (Spring 2026, published July 2026) reports input tokens above 90 percent of non-cache token volume, and the ratio of input to output tokens rising from about 4.5x in January 2026 to roughly 11x to 13x by April and May. One honest nuance the paper adds: cached input is cheaper per token than fresh input, but the sheer volume of input can still dominate the bill.

One worked receipt

Here is a single worked example, and it is illustrative by construction: the numbers show the shape of an honest comparison, not a verdict about any named model.

Take a bug fix that must pass a test suite. The cheap-first route starts on the small model. It produces a format error, retries, falls back to a stronger model when it stalls, and finally needs a human to repair the patch. The measured route starts on the stronger model where the history says it pays, and finishes in one pass. Counted to completion, the cheap-first route costs visibly more than the measured route, even though its tokens were cheaper. Same models, same list prices, a different bill once completion is counted.

That is the whole task-type distinction. The cheap model genuinely wins simple work: on a small, well-specified task it finishes in one clean pass and costs a fraction of the alternative. What it loses is the hard work, and it loses it to its own retries. The routing rule that follows is not "which model is cheapest?" but "which model has the lowest measured cost to finish this kind of task?", and the answer differs for an edit, a bug fix, and a repository migration.

The missing meter is an outcome ledger

Model-versus-model comparison is a crowded lane that goes stale every release. A better router keeps a live ledger, by task type, and lets measured history decide. The ledger's numbers are only as good as what it counts, so the gate at the end is a measurement list. Before changing a routing policy, measure eight things:

1. Tokens to completion, including the re-reads.
2. Retries.
3. Format failures.
4. Fallbacks to a stronger model.
5. A defined success, not an assumed one.
6. Latency.
7. Human repair time.
8. Results split by task type.

Until those are measured, a routing change is a preference. A higher per-token price is justified only after measured tokens-to-completion show it lowers cost per successful outcome.

The verdict: do not route to the cheapest model. Route to the cheapest completed outcome. Log the attempts, the fallbacks, the failures and the human repairs, then route the next task from the ledger, not the leaderboard. A price tag is not a receipt.

The data