Instead of choosing between training a world model and training a language conditioned robot policy, why not do both? LDA-1B s a new foundation model that is trained on 30,000 hours of human and robot interaction data. Part of the secret is that LDA-1B jointly learns forward dynamics, action prediction, and visual forecasting, all in a structured DINO latent space which avoids the pitfalls of redundant pixel-level prediction which isn’t necessarily aligned robot action. This approach works on both dexterous hands and simple robot grippers; it also generalizes across objects, tasks, and scenes. @JiangranLyu joins us to explain. Learn more on Episode 98 of RoboPapers, with @micoolcho and @chris_j_paxton!

Excitement about world-action models and robot learning has never been higher — they promise a way to use human egocentric data to train massive robotics models which can provide the “GPT” moment for robotics and unlock general-purpose embodied intelligence. And yet there’s been little concrete demonstration of scaling in robot learning. @DynaRobotics aims to change that, with an in-depth look at how scaling works as they approach 1 million hours of training data. @JasonMa2020 @tianyurobot @_anhquanpham and @ChetBhateja joined us to tell us more. They show that as the amount of data they use in pretraining increased, they saw predictable, statistically significant gains on accuracy metrics on held-out data (data not seen during training). They go on to talk about what they learned, and show how this can be applied to many different problems. Watch Episode #99# of RoboPapers, with @micoolcho @chris_j_paxton @DJiafei today to learn more!

Full episode dropping soon! Geeking out with @JasonMa2020 @tianyurobot @_anhquanpham @ChetBhateja on Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Co-hosted by @micoolcho @chris_j_paxton @DJiafei

Many tasks cannot be completed by one robot alone. But coordinating multiple robots performing complex manipulation tasks is very difficult. Many solutions rely on complicated centralized control, which tends towards bespoke methods that do not scale well with team size. Enter CHORUS by @riadoshi21, @leto__jean, and team. They train a single VLA to control multiple, diverse robots, given only local observations and identifying information. This allows robots to collaborate on tasks like using a tape measure, handing over a book, or lifting a laundry basket. To learn more, watch Episode #93# of Robopapers now, with @micoolcho, @DJiafei, and @chris_j_paxton!

Full episode dropping soon! Geeking out with @riadoshi21 @leto__jean on CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy Co-hosted by @micoolcho @DJiafei