Ilir Aliu(@IlirAliu_):A robot arm can now be trained over a weekend by three students in San Francisco. A bulldozer cannot. Spent part of the day on the floor at Fully Connected, @CoreWeave's conference in San Francisco. Good energy, plenty of cool demos. My favourite panel: From World Models to Working Machines: Physical AI in the Real World Caterpillar Inc., @nvidia, and CoreWeave, talking about what it actually takes to get physical AI out of the lab and onto a job site. Brandon Hootman, who runs construction autonomy at Caterpillar, said the company's development cycle used to run in years, timed to when a new machine model launched. It now runs in months. His reasoning: Caterpillar stopped waiting for the next product generation to test what it was learning, and started iterating with customers directly on real sites instead. NVIDIA's Amit Goel explained why this only became possible in the last year or two. Models turned multimodal, so they read video and lidar, not just text. Simulation got trustworthy, combining physics engines with models trained on real footage. And the compute to run both finally showed up. His number for the data gap... one hour of real machine data can now generate somewhere between 100 and 1,000 hours of usable simulation. The hardest part to simulate, according to Goel, is contact. What happens physically the moment a bucket hits soil or a truck bed meets rock is where simulations used to fall apart. His team now feeds real field data back into the simulator constantly to correct that. Hootman added one interesting thing. Caterpillar didn't rip out its existing safety and control systems to make room for AI. The model operates inside them, with hard limits on what it's allowed to decide on its own. Richard Ahlfeld, Ph.D. from CoreWeave made the infrastructure case: the data volume here runs far beyond what language models deal with, and evaluation has to sit right next to training, or you're flying blind on whether the model actually works. Working demo and something running unattended in the field... what's your best story here?

2026.09.30 23:41

A robot arm can now be trained over a weekend by three students in San Francisco. A bulldozer cannot. Spent part of the day on the floor at Fully Connected, @CoreWeave's conference in San Francisco. Good energy, plenty of cool demos. My favourite panel: From World Models to Working Machines: Physical AI in the Real World Caterpillar Inc., @nvidia, and CoreWeave, talking about what it actually takes to get physical AI out of the lab and onto a job site. Brandon Hootman, who runs construction autonomy at Caterpillar, said the company's development cycle used to run in years, timed to when a new machine model launched. It now runs in months. His reasoning: Caterpillar stopped waiting for the next product generation to test what it was learning, and started iterating with customers directly on real sites instead. NVIDIA's Amit Goel explained why this only became possible in the last year or two. Models turned multimodal, so they read video and lidar, not just text. Simulation got trustworthy, combining physics engines with models trained on real footage. And the compute to run both finally showed up. His number for the data gap... one hour of real machine data can now generate somewhere between 100 and 1,000 hours of usable simulation. The hardest part to simulate, according to Goel, is contact. What happens physically the moment a bucket hits soil or a truck bed meets rock is where simulations used to fall apart. His team now feeds real field data back into the simulator constantly to correct that. Hootman added one interesting thing. Caterpillar didn't rip out its existing safety and control systems to make room for AI. The model operates inside them, with hard limits on what it's allowed to decide on its own. Richard Ahlfeld, Ph.D. from CoreWeave made the infrastructure case: the data volume here runs far beyond what language models deal with, and evaluation has to sit right next to training, or you're flying blind on whether the model actually works. Working demo and something running unattended in the field... what's your best story here?

Forward to community