Register and share your invite link to earn from video plays and referrals.

Yilun Du
@du_yilun
Assistant Professor at Harvard @KempnerInst + CS. PhD @MIT_CSAIL, BS MIT. Generative Models, Compositionality, Embodied Agents, Robot Learning.
351 Following    14.2K Followers
Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents? We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration. A coding agent connects directly to the robot’s perception and control APIs, learns from feedback, and builds programs that perceive, plan, and act. The model stays frozen. As exploration scales, the agent improves its programs and builds up reusable skills. The results: • ZERO teleoperation or action demonstrations on real robots—from insertion with 0.1 mm clearance and air-fryer manipulation to deformable objects and long-horizon mobile whole-body control. • +37 percentage points on average over leading learned policies across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo’s bimanual tasks, and LIBERO-Pro. • One framework spanning single-arm, bimanual, dexterous, humanoid, and mobile robots, with new embodiments connected through APIs. The craziest part: on each real-world embodiment, going from scratch to mastering the tasks took roughly two hours. No teleop. No model finetuning. We’re open-sourcing the full framework AND every experiment session. We can’t wait to see what people build with it! Project + sessions: Paper: GitHub: N Project fully led by @RunhanH and @xczhangxx who are absolutely awesome. How it works, experiments, and robot videos below 🧵 (1/N)
Show more
0
35
570
105
Forward to community