注册并分享邀请链接,可获得视频播放与邀请奖励。

Sam Denton
@samueldenton
Platform Research @appliedcompute prev: @scale_ai; @amazon
加入 July 2015
620 正在关注    439 粉丝
Super excited to share this work we've been doing at AC (major kudos to @AtharvNaphade)! It allows us to take a legitimate step towards taking our hands of the wheels during model training. Looking forward to sharing more about how we were able to use it for automating a full end to end training loop 👀
显示更多
We're sharing a key step towards enabling Continual Learning at Applied Compute. In On-Policy Self-Distillation, the teacher model supervises training with privileged information: a hint. The training signal depends heavily on hint quality, but manually refining hints across runs is impractical, and isolating their individual impact is difficult. We instead optimize hints automatically using efficient proxies for their eventual training value.
显示更多