Applied Compute(@appliedcompute):We're sharing a key step towards enabling Continual Learning at Applied Compute. In On-Policy Self-Distillation, the teacher model supervises training with privileged information: a hint. The training signal depends heavily on hint quality, but manually refining hints across runs is impractical, and isolating their individual impact is difficult. We instead optimize hints automatically using efficient proxies for their eventual training value.

Applied Compute

@appliedcompute

The best AI is built, not bought. Our platform, Applied Compute Agent Cloud, is now in private beta. Book a demo below.

Joined July 2012

24 Following 8.3K Followers

Applied Compute@appliedcompute

2026.10.07 18:44

We're sharing a key step towards enabling Continual Learning at Applied Compute. In On-Policy Self-Distillation, the teacher model supervises training with privileged information: a hint. The training signal depends heavily on hint quality, but manually refining hints across runs is impractical, and isolating their individual impact is difficult. We instead optimize hints automatically using efficient proxies for their eventual training value.