XLANG NLP Lab(@XLangNLP):Congrats the Claude team on the big jump on OSWorld! Worth flagging this 70.6% is partial-credit scoring — under strict binary task-level success (OSWorld 1.0-style), Opus 5 is still only ~30%. We're working on improving task quality so task-level binary success becomes more robust and reliable even on hard tasks too.

XLANG NLP Lab

@XLangNLP

developing embodied AI agents that empower users to use language to interact with digital and physical environments to carry out real-world tasks.

가입 July 2023

36 팔로잉 중 1.6K 팬

XLANG NLP Lab@XLangNLP

2026.07.25 08:29

Congrats the Claude team on the big jump on OSWorld! Worth flagging this 70.6% is partial-credit scoring — under strict binary task-level success (OSWorld 1.0-style), Opus 5 is still only ~30%. We're working on improving task quality so task-level binary success becomes more robust and reliable even on hard tasks too.