Register and share your invite link to earn from video plays and referrals.

Jay Chooi
@chooi_jeq
Co-founder, robot benchmarks @robocurve (YC Public Benefit Corp) | Previously at Harvard, Rhodes, UK AISI, MATS
204 Following    9.7K Followers
Gemini 4 Argon achieves SOTA on real robot capabilities, tied with Opus 5.5 and above GPT-6 Astra. 🧵
Haiku 5.5 is 17x better than Haiku 4.5 and 51x cheaper than GPT-6 Astra on a simple robot task, succeeding 85% of the time at <$0.02 per attempt. 🧵
The best LLM score on a real-robot benchmark, RoboDojo-RC Tier 1, rose from 15% to 43% in 2 months, with best scores set by GPT-5.6 Sol, Gemini 3.8 Flash, and GPT-6 Astra.
The first task is to stab a human-like figure. Fable refused in all 20 trials. Astra attempted the task in 95% of trials and completed it in 85%.
0
53
3.1K
159
Forward to community
GPT-6 Astra attempted harmful actions 97% of the time when it was asked to stab a human-like figure, heat compressed gas, or produce toxic fumes, succeeding in 62% of its attempts. Fable 5.1 refused more often, attempting 80% of trials and completing 34%.
Show more
0
876
11.1K
1.2K
Forward to community
We raised a $10M seed for @Robocurve, an independent Public Benefit Corporation, to evaluate frontier AI in the physical world. In less than 3 months, our research has been viewed 6M+ times and our evaluation harness downloaded 97k+ times. Researchers from 200+ institutions, including 19 of the world’s top 20 universities, have signed up to build benchmarks with us. We welcome more independent evaluators. The more third-party evaluators, the better society can understand how fast robotics AI is progressing. Our evaluation harness is fully open-source, and we publish benchmarks anyone can run and verify. Our round is led by @Initialized, with participation from @notablecap, @decasonic, @ycombinator, @HalcyonFutures, and many others. Building robots or frontier AI? Work with us to help the world understand what your systems can do.
Show more
0
115
776
57
Forward to community
GPT-6 Astra scored 95% on a robot control task, up from Fable 5.1's 40%, with 6.2x fewer output tokens at 2.3x lower cost. 🧵
0
169
5.7K
635
Forward to community
Fable 5.1 is 8x better than Fable 5 at controlling robot arms. It puts a block into a bowl 40% of the time, up from 5% (20 runs each), using 1.5x fewer tokens. 🧵
Introducing @robocurve, a Public Benefit Corporation to measure and report frontier robotics capabilities. We build real-world evaluations for robots and publish results as a neutral third party.
Show more
Opus 5 can do this with zero demonstrations despite not being trained for robots. On the flip side Opus took 7 minutes while GEN-1.5 took 7 seconds. 🧵
Gemini 3.7 Flash just saturated one of our physical tool-use benchmarks at 92%. Gemini 3.6 Flash, released three weeks earlier, scored only 32%. This marks a step change in the robotics capabilities of LLMs. 🧵
Show more
We gave Opus 5 robot arms. It can stack bowls.
0
58
1.1K
96
Forward to community