가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Dongxi 东锡 NLP
@dongxi_nlp
Prev. PhD @Stockholm_Uni | Alumni @KTHuniversity @uppsalauni Sharing insights on AI
가입 January 2022
1K 팔로잉 중    42.7K 팬
推荐阅读! TreeOPSD: Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning
(1/5) New preprint: TreeOPSD, critic-free RL for long-horizon LLM agents 🌳 Long-horizon agents receive a single outcome reward after many turns, and each rollout batch feeds several sequential mini-batch updates while the policy drifts from the one that sampled it. TreeOPSD backs terminal rewards up a rollout tree and splits each update into two steps: construct a target for every turn, then match the current policy to it with forward KL. This leads to: • closed-form, bounded turn targets that commute with tree backup, so shared prefixes are trained once with no loss of credit • the exact tree policy gradient on the first mini-batch, and updates that stop at the target and reverse after overshoot • consistent gains over GRPO, GSPO and a matched tree baseline, largest on the longest-horizon tasks On Deep Research, TreeOPSD lifts Qwen3-30B-A3B-Thinking-2507 from 16.23 (GRPO) to 21.20 average accuracy. Title: Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning Paper: Together with @TencentHunyuan #LLM# #RLVR# #AIAgents#
더 보기