Vals AI(@ValsAI):How do model routers compare to individual models on real agentic tasks? Introducing the Vals Router Index. Routers promise to balance quality and cost across a fast-moving model frontier. We put that promise to the test with an independent evaluation of five routers (OpenRouter Auto, FireRouter, Ramp Router, Jev, Fugu Max) against every single model on the Vals Index, using the same agents, ~1,250 tasks and graders across finance, coding, law and tax. We found that routing didn’t consistently deliver better value. Every router was beaten on both score and cost by at least one single model.

2026.10.20 19:18

How do model routers compare to individual models on real agentic tasks? Introducing the Vals Router Index. Routers promise to balance quality and cost across a fast-moving model frontier. We put that promise to the test with an independent evaluation of five routers (OpenRouter Auto, FireRouter, Ramp Router, Jev, Fugu Max) against every single model on the Vals Index, using the same agents, ~1,250 tasks and graders across finance, coding, law and tax. We found that routing didn’t consistently deliver better value. Every router was beaten on both score and cost by at least one single model.

显示更多