Why are AI agents so expensive? Do more tokens actually lead to better performance? Which models are more token-efficient? Can agents predict their own token costs before execution?
These were the questions bugging us, so we wrote a paper to find out.
Excited to share "How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks", led by
@Longju_Bai, with co-authors at Stanford, MIT, Google DeepMind, Microsoft AI, and All Hands AI.
A few findings that surprised us:
๐น Agentic coding tasks consume ~1000ร more tokens than chat or reasoning workloads. And input tokens, not output, become the dominant cost driver, because each round re-feeds the entire trajectory back into the model.
๐น More tokens โ better outcomes. Runs on the same task can vary by up to 30ร in token use, and accuracy often peaks at intermediate cost. Beyond that, extra spending tends to reflect redundant exploration and does not bring further performance gain.
๐น Models differ substantially in token efficiency. On the same successfully solved tasks, Kimi-K2 and Claude Sonnet-4.5 use roughly twice as many tokens as GPT-5.2. The gap becomes even larger when all the models fail.
๐น Human-rated task difficulty weakly predicts actual cost. "Easy" tasks for humans can be surprisingly expensive for agents, and vice versa. The classic "Moravec's Paradox" is also true for coding agents!
๐น Agents struggle to predict their own costs. Self-prediction correlations top out around 0.39, and every model we tested systematically underestimates what a task will cost. Result-based pricing still has a long way to go when we cannot even figure out the token cost beforehand.
Together, these results suggest that cost prediction is a genuinely challenging task for current agents. We think this opens up real research questions around self-modeling, calibrated cost estimation, and pricing mechanisms that work under residual uncertainty.
Huge thanks to my collaborators:
@Longju_Bai, Zhemin Huang,
@sunjiao123sun_ ,
@xingyaow_ ,
@radamihalcea ,
@erikbryn ,
@alex_pentland
paper:
website: