註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Rohan Pandey
@khoomeik
descending cross-entropy to ascend entropy @PeriodicLabs || prev research @OpenAI @CarnegieMellon '23
加入 February 2020
3K 正在關注    43.3K 粉絲
You saw the AI & science. Let's talk about the RL infra it took to build @periodiclabs Neon. To minimize training-inference mismatch in RL, SGLang captures inference's MoE routing decisions for each rollout and we "replay" them while training. In agentic (multiturn tool-use) settings, SGLang exports these router decisions in response to each decoding request, i.e. after each conversation turn. So when *any* data-parallel rank finishes a conversation turn, *all* other ranks must wait until routing data finishes exporting. This slowdown is exacerbated because we export routing decisions from the *entire* conversation rather than just the most recent turn! When @hsu_byron @vwxyzjn discovered this in our Kimi K2 RL setup, they introduced Delta Router Replay: cache previous turns' router decisions on the training client, so you can export only the delta (most recent turn's router decisions) upon each decoding request. Delta Router Replay significantly speeds up our long-context agentic RL runs, and @hsu_byron upstreamed it to SGLang (#24851#) a few months ago.
顯示更多
0
13
215
22
轉發到社區