Register and share your invite link to earn from video plays and referrals.

XLANG NLP Lab
@XLangNLP
developing embodied AI agents that empower users to use language to interact with digital and physical environments to carry out real-world tasks.
Joined July 2023
35 Following    1.5K Followers
Two years ago, we built OSWorld 1.0 โ€” the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved? Not even close. ๐Ÿš€Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks. What's new: ๐ŸŽฏ 108 real-world workflows, each ~1.6 hours โฑ๏ธ for a skilled human โš™๏ธ ~318 tool calls/task vs. ~30 in OSWorld 1.0 ๐ŸŒ Grounded in authentic artifacts & stateful user profiles โšก Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more ๐Ÿ“Š Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use. ๐Ÿ  Homepage: ๐Ÿ“„ Paper: ๐Ÿ’ป Code: ๐Ÿค— Dataset: ๐Ÿงต [1/8]
Show more
0
16
441
109
Forward to community