🤖 What if you could pilot a robot with a VLM that never sees a single robot training example? RoboDawn tackles exactly that question.
Title: Transferring the Intelligence of VLMs to Robotic Control (RoboDawn)
URL:
It gives a frozen, pretrained VLM a human-intuitive interface of translation, rotation, and gripper commands, plus a handful of in-context demonstrations, and lets it directly drive a robot arm.
Three things stand out.
🎮 A game-like control interface
The VLM issues discrete move, rotate, and gripper commands, all defined relative to the gripper interaction point. This lets it reuse spatial manipulation knowledge it already picked up from web-scale pretraining.
📚 One demo makes a huge difference
No parameter updates at all. Just a command primer plus task demonstrations as context lift success on RoboTwin 2.0 from 53.2% zero-shot to 73.6% one-shot.
🏆 It beats robot-trained policies outright
With zero task-specific training, RoboDawn's zero-shot performance already surpasses policies trained on dedicated data, like π0.5 (46.0%) and LingBot-VLA (50.4%). On a real Franka robot it hits a 90% success rate.
It suggests the real bottleneck may not be collecting more robot data, but designing the interface that unlocks the intelligence VLMs already have.
#
Robotics# #
VLM#