Agents on Rails: lemans is now open source, and today four new models hit the benchmark:
@openai’s Terra,
@Alibaba_Qwen 3.8-27B,
@AnthropicAI's Sonnet 5, and ox-alpha, a new model just released in stealth mode.
As of August 24, 2026:
- Fastest:
@openAI Terra is now the fastest model, at a median of 3 minutes 2 seconds per task.
- Most accurate: Still
@claudeai Opus 5 by
@AnthropicAI. Solved 92% of runs (58 of 63), and has the best API recall of all the models (35%).
- Cheapest:
@OpenAI GPT-5.6 Luna is still the cheapest from the models we can accurately price - just .012 cents per run.
Read the latest benchmark report from
@evilmartians to learn about lemans and see how the new models stacked up: