注册并分享邀请链接,可获得视频播放与邀请奖励。

Wenhao Chai
@wenhaocha1
加入 January 2022
2.2K 正在关注    7.6K 粉丝
we brought PPO back just for one reason: get way more out of each rollout GRPO is hard to scale, you can't just pay for as many rollouts as you want on every task. for math/coding an extra rollout is a couple more compiles and checks, whatever. for RSI it's a training run that costs thousands of dollars. and in bio we literally don't have multiple identical embryos to run in parallel we did some early stuff on keeping PPO training stable, hopefully some of it's useful to people
显示更多