A robot that runs a video generation model as its policy! 🧤
I ran into mimic at NVIDIA GTC in Berlin.
@mimicrobotics's five-finger hands built in house, two wrist cameras plus a workspace camera, their own whole-body stack doing impedance control on the arms.
The policy is FLUX-mimic, developed with Black Forest Labs on the FLUX 3 backbone.
The action decoder is lightweight and reads intermediate features from the backbone’s video prediction path. Over 95% of training compute went into predicting video. Actions are a thin readout on top.
And it works with the backbone frozen. Black Forest Labs reports the decoder beating prior VLAs with no backbone finetuning at all, a setting where VLAs reportedly fail outright.
The manipulation-relevant structure is already sitting in the video representation. Nobody taught it contact or cause and effect as a separate objective. It fell out of predicting tens of millions of hours of video.
One detail from the demo is the drop-in teleop. The operator takes over mid-episode to correct a wrong move, then hands back and the policy resumes.
That’s the safety story, and every correction is a labelled intervention at precisely the moment the policy was wrong.
Running at Audi on kitting, ECU insertion, and soft parts like seals and cables.
Thanks for explaining
@elvisnavah! 🦾
It was awesome during
@NVIDIARobotics GTC, thanks for inviting me!
~~
♻️ Join the weekly robotics newsletter, and never miss any news →