TL;DR: "The model you use is not the model that was pretrained." An 87-slide lecture that maps post-training — turning a base model into a useful policy — across six stages: imitate, compare, explore, verify, transfer, anticipate. Title: Post-Training LLMs (Kawin Ethayarajh, AI and Economics Summer Institute 2026) URL: Key points 📝 SFT: teach by demonstration. Can imitate whole trajectories (plan, tools), but it's fundamentally imitative ⚖️ Offline preference optimization: learn from win/loss. DPO drops the reward model; KTO learns per-outcome, no SFT needed 🎲 Online RL: the current policy explores. REINFORCE/PPO/GRPO; rewards are mostly sequence-level ✅ RLVR: replace the reward model with a checker. Binary pass/fail is enough; scales more predictably than RLHF 🧪 Environments: an RL env is task + data + interface + tests. The bottleneck shifts from examples to environments 🔁 On-policy distillation: the teacher scores the student's own rollout. Copies an RL policy in 7–10x fewer steps 🌐 World adaptation: environments adapt back to agents. "Mecha-nudges" raise machine-readability A systematic map, threaded with an economist's lens. #LLM# #PostTraining#