가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

huaijiang
@huaijiangzhu
가입 July 2020
0 팔로잉 중    0
ok, so if i understood this correctly, the “in-context learning” ability was essentially baked into the model at training time. roughly speaking, the policy is trained to do something like: (demo video, current obs) → action this is quite different from in-context learning in llms, where the model is simply trained on next-token prediction over arbitrary context. fwiw, this is pretty clever because it could also solve another problem: they could pair an egocentric video-only demonstration with a UMI/teleop trajectory for the same task. the target trajectory provides the action labels, while the video-only demonstration provides the task context. that way, the policy can learn from action-labeled robot data while also learning how to translate an unlabeled video demonstration into robot actions.
더 보기