we perform post-training via multiple stages: SFT, RL, RFT, and on-policy self-distillation (OPSD). we use RFT and OPSD as the last stage of training to bridge the sim2real gap. RL environments, while very realistic, still lack the full diversity and complexity of the real-world use cases we receive from production. in this research we describe how OPSD can be quite effective at learning from user feedback in follow-up conversations, as well as from tool call errors. we design a training method that combines RFT (forward KL/CE) and OPSD (reverse KL) to learn from both successful and problematic trajectories. for problematic trajectories, we construct hints from tool call errors or negative user feedback and use self-distillation to alleviate those errors. in this specific work we post-trained GLM-5.2 with this approach and observed a substantial 21% reduction in tool call errors, as well as positive improvements in user satisfaction (not yet statistically significant, which we are currently addressing). it also made the trajectories more cost-efficient. another reason i'm excited about this direction is that it seems to be a potential way of doing online/continual learning. for more details, take a look at the blog: