Register and share your invite link to earn from video plays and referrals.

Jiaxin Pei
@jiaxin_pei
Assistant Professor @UTAustin, Research Fellow @StanfordHAI & @DigEconLab Researching AI Agents and Human-AI Interaction
954 Following    1.8K Followers
Why are AI agents so expensive? Do more tokens actually lead to better performance? Which models are more token-efficient? Can agents predict their own token costs before execution? These were the questions bugging us, so we wrote a paper to find out. Excited to share "How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks", led by @Longju_Bai, with co-authors at Stanford, MIT, Google DeepMind, Microsoft AI, and All Hands AI. A few findings that surprised us: 🔹 Agentic coding tasks consume ~1000× more tokens than chat or reasoning workloads. And input tokens, not output, become the dominant cost driver, because each round re-feeds the entire trajectory back into the model. 🔹 More tokens ≠ better outcomes. Runs on the same task can vary by up to 30× in token use, and accuracy often peaks at intermediate cost. Beyond that, extra spending tends to reflect redundant exploration and does not bring further performance gain. 🔹 Models differ substantially in token efficiency. On the same successfully solved tasks, Kimi-K2 and Claude Sonnet-4.5 use roughly twice as many tokens as GPT-5.2. The gap becomes even larger when all the models fail. 🔹 Human-rated task difficulty weakly predicts actual cost. "Easy" tasks for humans can be surprisingly expensive for agents, and vice versa. The classic "Moravec's Paradox" is also true for coding agents! 🔹 Agents struggle to predict their own costs. Self-prediction correlations top out around 0.39, and every model we tested systematically underestimates what a task will cost. Result-based pricing still has a long way to go when we cannot even figure out the token cost beforehand. Together, these results suggest that cost prediction is a genuinely challenging task for current agents. We think this opens up real research questions around self-modeling, calibrated cost estimation, and pricing mechanisms that work under residual uncertainty. Huge thanks to my collaborators: @Longju_Bai, Zhemin Huang, @sunjiao123sun_ , @xingyaow_ , @radamihalcea , @erikbryn , @alex_pentland paper: website:
Show more