OPID does skill distillation for agentic RL using the agent's own completed trajectories as the skill source, so the supervision stays on-policy and matched to the current state distribution. Hand-written or off-policy skill libraries drift from what the policy actually visits in multi-turn interaction, which is the gap it targets. Skills are extracted in hindsight from finished rollouts and split into two levels: episode-level skills that capture global workflows and failure-avoidance rules, and step-level skills for local decisions, with a critical-first routing that distills the high-leverage ones back into the policy. OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning Paper:

On-policy distillation has the same systems bottleneck as RL: rollouts dominate training time on reasoning workloads. Going async fixes throughput but feeds the learner stale-policy data, and what staleness does to OPD specifically was unstudied. The clean finding is that KL direction decides robustness. Teacher-weighted forward KL shrugs off stale rollouts, student-weighted reverse KL breaks under them, and for the reverse-KL case nothing from async RL beats just recomputing the signal under the current student. Finite teacher-score caches then turn the estimator into a bias-variance tradeoff, which is the case for multi-sample Monte Carlo. AsyncOPD: How Stale Can On-Policy Distillation Be? Paper:

Adding an on policy distillation section to the RLHF book and it’s remarkable how bad LLMs / coding agents are at it, despite me giving them the core papers and 250 pages of context on how I present ideas.

Anthropic says Alibaba ran the largest known distillation attack on its models, training their own systems on Claude's outputs without authorization. The problem with closed models is that there’s no verifiable trail showing which outputs ended up in downstream training. Onchain provenance is the fix because it can create receipts for what a model was actually built on. Without that, model theft stays almost impossible to prove cleanly.

We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax. These labs created over 24,000 fraudulent accounts and generated over 16 million exchanges with Claude, extracting its capabilities to train and improve their own models.

⚡️BREAKING: Zhang Yiming says ByteDance will not use distillation as a shortcut to catch up in AI, even if that means its LLMs temporarily fall behind Chinese rivals. According to The Information, Zhang made the remarks at a Seed team all-hands last month, telling the team to sacrifice short-term gains for longer-term goals. Distillation allows a new model to learn from the outputs of a stronger model, saving training time and compute. But ByteDance is choosing a harder path: building its own frontier models more independently. The bigger concern may be political risk. ByteDance has spent years under U.S. scrutiny over TikTok. If the company were accused of extracting capabilities from leading American AI models at scale, it could give Washington another reason to pressure TikTok. Anthropic has previously accused several Chinese AI firms of extracting capabilities from Claude, including DeepSeek, Moonshot AI, MiniMax, Zhipu and Alibaba, but not ByteDance. #ByteDance# #distillation# #seedance# #LLMs# #TikTok#

Warning my friends: for the sake of saying that distillation is a good thing, and not a bad one, you are falling in the trap of admitting that frontier Chinese models are *mainly* the result of distillation (which is not just not true, but also not possible).