Register and share your invite link to earn from video plays and referrals.

Sam Denton
@samueldenton
Platform Research @appliedcompute prev: @scale_ai; @amazon
Joined July 2015
620 Following    439 Followers
Super excited to share this work we've been doing at AC (major kudos to @AtharvNaphade)! It allows us to take a legitimate step towards taking our hands of the wheels during model training. Looking forward to sharing more about how we were able to use it for automating a full end to end training loop 👀
Show more
We're sharing a key step towards enabling Continual Learning at Applied Compute. In On-Policy Self-Distillation, the teacher model supervises training with privileged information: a hint. The training signal depends heavily on hint quality, but manually refining hints across runs is impractical, and isolating their individual impact is difficult. We instead optimize hints automatically using efficient proxies for their eventual training value.
Show more