TL;DR: A new five-platform environment lets you train and evaluate agents that combine GUI operation with coding, and it reveals that even top models look far more capable than they actually are under strict behavioral testing.
Title: RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents
URL:
Key points
🖥️ Covers Ubuntu, macOS, Windows, Android, and Web
🔍 "Recreation" tasks: rebuild a running reference app from scratch, no source code
🧪 RecreationBench: 250 tasks scored via both programmatic and visual assertions
📈 Fine-tuning on 35k recreation trajectories lifts OOD benchmarks by up to 17.9 points
🏆 Top model GPT-6 Astra scores 58.1% overall
⚠️ Yet it passes every programmatic test on only 2.8% of tasks
🔧 Agents flexibly swap Electron apps for native GTK/AppKit without losing fidelity
It's a sharp reminder that surface-level recreation and true behavioral fidelity are still worlds apart.
#
AIAgents# #
ComputerUse#