DrivingBench
@DrivingBench
frontier LLMs drive a toyota corolla -
3 Following 839 Followers
DrivingBench origins 🚗
@tobiges: "We were all on Twitter a lot, and we saw all these demos of crazy things recent models like Astra were capable of doing in Blender, spatial reasoning, all these things. We were hanging out on a weekend and chatting about what would be possible."
"We were thinking: are we actually at a point where we can take general-purpose models? Obviously these driving systems are super specialized. But what if we took a general model and gave it a car to drive, and see what happens?"
"So we decided to make it into a benchmark and do more sophisticated evaluations of the capabilities these models have now."
@MTSlive @nautsimon_ @a_ramabadran
Show more
We gave ChatGPT, Claude, and Grok control of a real Toyota Corolla 🚗, steering/gas/brakes, no human driving (just a foot over the brake).
Only one model was able to complete our entire driving course. Introducing DrivingBench. 🔥⌛️🏁
Show more
