Haseeb Qureshi ＞|＜(@hosseeb):Excellent primer on the economics of token caching, and why fancy token model routers aren't as economical as you'd think. As workloads lean more agentic (way more tokens in than out vs chat), caching dominates as the most important input into token costs.

2026.09.09 18:01

Excellent primer on the economics of token caching, and why fancy token model routers aren't as economical as you'd think. As workloads lean more agentic (way more tokens in than out vs chat), caching dominates as the most important input into token costs.

Diogo Almeida@CompleteSkeptic

2026.09.09 17:25

Wrote about the incredibly important KV cache: - Why are people spending so much on agents - How is inference so profitable all of a sudden (+ why it won't last) - Why routing doesn't work P.S. they also constrain the design space for coding agents A LOT

Forward to community