登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

LMSYS Org
@lmsysorg
Large Model Systems Organization: We developed SGLang @sgl_project ( Chatbot Arena (now @arena), and Vicuna!
参加 August 2024
204 フォロー中    17.5K ファン
SGLang + Miles: Rollout Routing Replay (R3) is Now Live! 🎉 We're excited to announce that SGLang and Miles now support Rollout Routing Replay (R3) for stable reinforcement learning training on MoE models! Training MoE models with RL has been notoriously unstable, often leading to catastrophic collapse. The problem? Routing inconsistency between inference and training engines. R3 fixes this by recording expert routing decisions during inference and replaying them during training. The impact is significant: dramatically reduced training-inference discrepancy by reusing inference routing decisions, preventing training collapse. R3 has full distributed training support with DataParallel Attention and all parallelism strategies, supported models include Qwen3-30B-A3B, deepseek_v2, etc. Try it out and let us know your results! 🚀
もっと見る