가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Lukas Ziegler
@lukas_m_ziegler
robotics evangelist | riding the wave of robotics | angel investing 🕵🏼‍♂️
가입 September 2021
983 팔로잉 중    65.5K 팬
A robot that runs a video generation model as its policy! 🧤 I ran into mimic at NVIDIA GTC in Berlin. @mimicrobotics's five-finger hands built in house, two wrist cameras plus a workspace camera, their own whole-body stack doing impedance control on the arms. The policy is FLUX-mimic, developed with Black Forest Labs on the FLUX 3 backbone. The action decoder is lightweight and reads intermediate features from the backbone’s video prediction path. Over 95% of training compute went into predicting video. Actions are a thin readout on top. And it works with the backbone frozen. Black Forest Labs reports the decoder beating prior VLAs with no backbone finetuning at all, a setting where VLAs reportedly fail outright. The manipulation-relevant structure is already sitting in the video representation. Nobody taught it contact or cause and effect as a separate objective. It fell out of predicting tens of millions of hours of video. One detail from the demo is the drop-in teleop. The operator takes over mid-episode to correct a wrong move, then hands back and the policy resumes. That’s the safety story, and every correction is a labelled intervention at precisely the moment the policy was wrong. Running at Audi on kitting, ECU insertion, and soft parts like seals and cables. Thanks for explaining @elvisnavah! 🦾 It was awesome during @NVIDIARobotics GTC, thanks for inviting me! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
더 보기