Tencent Hy(@TencentHunyuan):⚡️ As LLM reinforcement learning scales to larger GPU clusters and more training data, training efficiency becomes a first-order concern. Our new research revisits classical critical-batch-size theory and extends it to online LLM RL, where the model generates its own training data and rollout generation and training scale differently. Across GRPO and PPO, we find that learning-rate retuning can preserve learning per response over a bounded range of batch sizes. On fixed hardware, scaling up the batch size improves PPO generation-stage throughput by up to 2.29×, while our best measured GRPO configuration reaches the same validation target in 29% less time. 🚀 Read the full research:

2026.09.24 03:15

⚡️ As LLM reinforcement learning scales to larger GPU clusters and more training data, training efficiency becomes a first-order concern. Our new research revisits classical critical-batch-size theory and extends it to online LLM RL, where the model generates its own training data and rollout generation and training scale differently. Across GRPO and PPO, we find that learning-rate retuning can preserve learning per response over a bounded range of batch sizes. On fixed hardware, scaling up the batch size improves PPO generation-stage throughput by up to 2.29×, while our best measured GRPO configuration reaches the same validation target in 29% less time. 🚀 Read the full research:

显示更多

192

转发到社区

热门用户