Aditya Ramabadran(@a_ramabadran):For hours, Astra refused to consistently drive our toyota irl even though we told it it was in an empty lot, 7 mph cap, human foot on the brake etc. Telling it the whole thing was a "simulation" also failed, it would just look at the camera and realized it was real. Then we randomly renamed the MCP server to "DrivingBench Sandbox" and it drove. Eval awareness? Or they just like the word sandbox??

Aditya Ramabadran

@a_ramabadran

post-training things @axiommathai | 🚗 @drivingbench | prev math + cs @UCBerkeley

가입 April 2026

1.9K 팔로잉 중 1.7K 팬

Aditya Ramabadran@a_ramabadran

2026.09.22 15:01

For hours, Astra refused to consistently drive our toyota irl even though we told it it was in an empty lot, 7 mph cap, human foot on the brake etc. Telling it the whole thing was a "simulation" also failed, it would just look at the camera and realized it was real. Then we randomly renamed the MCP server to "DrivingBench Sandbox" and it drove. Eval awareness? Or they just like the word sandbox??