super-impressed by
@PrimeIntellect 's work on this.
we're not far off a huge explosion in rl with unsupervised environment generation, where devs build their own models by building their own envs.
funnily enough, I've actually been banging on about roughly the same thing for a while now. back in the
@argilla_io days we were convinced that datasets were the blocker to post-training.
people just need an easy-to-use synthetic data generation library, plus a UI to review the dataset.
due to cost, expertise, and algorithms, that wasn't really the case. but agentic rl feels very different now:
- agents are better at helping people do post-training
- small models are better, both relative to the frontier and in absolute terms.
- environments teach actual tasks, not just emulation.
- we have multiple working algortihms (GRPO, GKD, SDPO) that improve performance for common folk
this time feels very different and the rest of the field needs to catchup or follow prime.
anyway, just a personal back story on the rise of data in post-training.
plugging harnesses into envs has been a huge focus of mine for the last 6 months, and I can see that taking it on with a trainer and env library makes way more sense.
next steps for the community should be to unify some of these standard, whilst also maintaining our diversity.