The model didn't change. The interfaces did.
- same open 27B weights
- 3.8x the success rate on OSWorld V2
- $1.10 a task vs $25.50 for GPT-5.5 and $72.40 for Opus 4.8
This is why I run Hermes with whatever model fits the job. The harness is half the game
Our agent scientist, Agent S4, just got an open 27B model to beat GPT-5.5 and Claude Opus 4.8 on OSWorld V2, with the model's weights left exactly as they were.
With the interfaces S4 evolved, Qwen3.8-27B completes 21.3% of the 108 tasks, the highest binary success of any open-source computer-use agent and 3.8x the same model with fixed interfaces.
It costs $1.10 per task, against $25.50 for GPT-5.5 and $72.40 for Opus 4.8 💸
Full write-up: