註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

keerti manney
@keertimanney
Towards spatial intelligence, founding @antimlabs
加入 December 2011
347 正在關注    91 粉絲
I think “can an LLM control a robot?” is slightly the wrong question. With enough compute and enough time between actions, probably yes. What’s more interesting is what has to change in the architecture, training, and inference for these models to operate at robot speed. Robotics operates at multiple timescales. Deciding “Pick up the pan and place it on the stove.” can happen relatively slowly. Maintaining balance, correcting a slipping grasp, reacting to contact, or adjusting a joint trajectory cannot. A model that has to autoregressively generate a sequence of tokens before every physical correction is paying an enormous latency tax for capabilities you often don’t need at that layer. Making the LLM 2x or 5x faster helps, but it doesn’t really change the shape of the problem.
顯示更多