註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Sam Denton
@samueldenton
Platform Research @appliedcompute prev: @scale_ai; @amazon
加入 July 2015
620 正在關注    439 粉絲
Super excited to share this work we've been doing at AC (major kudos to @AtharvNaphade)! It allows us to take a legitimate step towards taking our hands of the wheels during model training. Looking forward to sharing more about how we were able to use it for automating a full end to end training loop 👀
顯示更多
We're sharing a key step towards enabling Continual Learning at Applied Compute. In On-Policy Self-Distillation, the teacher model supervises training with privileged information: a hint. The training signal depends heavily on hint quality, but manually refining hints across runs is impractical, and isolating their individual impact is difficult. We instead optimize hints automatically using efficient proxies for their eventual training value.
顯示更多