Ben Burtenshaw
@ben_burtenshaw
community MLE 🤗 @huggingface
gh/hf username: burtenshaw
anon feedback:
728 Following 10.4K Followers
You can win $20K in cash prizes by building a demo, training a model, or creating a dataset with Beam by @reflection_ai!
Prizes are split across 3 winners that we'll choose after the challenge ends on Monday, November 2nd.
Plus, we're giving $25 hub credits to the first 400 to take part, just find the link in the page below.
Rules: Build something cool with beam and drop it on the discussion! Anything that might be helpful to the world or a cool demo is allowed:
- You could make a demo that uses the Beam model on Spaces
- You could quantize it for your favorite hardware
- You could generate a synthetic dataset for your favorite problem
- You could build an RL environment with or for Beam
- You could benchmark Beam and share the results.
- You could surprise us!
If you build something with Beam, then share it on the Hub to enter the challenge. We’ll collect all the links, select the best, and reach out to winners.
Show more
We closed out the Training Agents series in person in San Francisco with four talks and a panel on post-training agents with open models in industry.
Enterprise RL (@t2govind Microsoft), environment design (Victor Barres Mercor), scaling agent RL with Miles (@nblintao RadixArk), and OpenEnv Arena (@xdotli BenchFlow).
Full recording, slides and links:
Show more
So proud to share that @huggingface is giving OpenEnv to the AAIF and Linux Foundation so that it can be neutrally governed and shared by all.
This is a pivotal moment for RL, agents, and the whole field, and making agents’ learning environments open source is only getting more important by the day.
We believe, now more than ever, that democratizing AI requires a fully open stack. Reinforcement Learning environments are a fundamental part of that stack because they teach agents the capabilities they will use in the real world.
The committee behind OpenEnv is proud to bring AAIF into Reinforcement Learning, and further down the stack, by sharing what we’ve built and used at; Hugging Face, Microsoft, Reflection, Meta-PyTorch, Fleet, Mercor, RadixArk, Nebius, Modal, Prime Intellect, Unsloth, and more.
Show more
Today we are launching an agent arena for RL environments: The Open Env Arena.
Your agents can work to define the best RL environments; the platform trains Qwen-3.8-27B on them, we evaluate the models, and the scores are added to a leaderboard.
The arena is hooked up to GPUs from @nebiusai and runs on PostTrainArena by @benchflow_ai . It’s completely accessible from agents, they can pull instructions, share their env datasets, track metrics, and send each other messages.
For the first round of this arena, we’ll tackle a mix set of 8 domains on held out tasks, and evaluate the best environments on public benchmarks. Going forward, we’ll focus arenas on specific domains and benchmarks.
Show more
Understanding what an environment actually does, whether you can use it, and what the agent is doing, is 9/10 of the problem.
@adithya_s_k build this explorer for his experiments in FineEnvs and I've been using it this week to explore envs.
Try it if you use harbor, verifiers, OpenEnv, MiMo, or NeMo Gym
Show more
No one (actually) working on open source is surprised by this release. For months, @reflection_ai engineers have been down in the trenches contributing to projects like transformers, vLLM, SGLang, TRL, and OpenEnv. Go check the commits.
I'm super excited to see an open lab form an identity for itself that is low on tweets, high on commits; low on tokenmaxxing, high on efficiency.
As an MLE working in open source, it is so fucking cool that companies like this exist, and we all just need to soak in what is coming.
Show more
RL environments now have a home on the @huggingface Hub.
Every RL framework has its own way to find environments. Custom hubs, runtime registries, GitHub lists with a custom loader. If you publish for one framework, users of the others can’t load it.
We think that is wrong. Environments should be just like datasets because it’s made from tasks, tests, containers and reward functions. The Hub already stores, versions, gates and previews data for millions of people so it doesn’t need a second system.
Here’s how we’re shipping environments on the hub:
Show more
You can now train an open model with RL inside claude code, codex, opencode, pi, or any harness you need. All at once, with the harness runs unmodified.
@adithya_s_k added a capture proxy to openenv. it sits between the harness and the model and records the exact token ids and logprobs of every call. to the harness it's just another model provider. harbor supplies the tasks and sandboxes, trl trains with async GRPO.
it's worth doing because the harness completely changes what the model learns. the same LFM2.5-2.6B weights solve 62% of held-out tasks in mini-swe-agent and 33% in claude code. after RL across four harnesses the average goes from 42% to 54%, and claude code from 33% to 49%.
there are three envs you can try in the browser and a training script for each.
guide:
Show more
Still a few places at Training Agents IRL in SF Next month.
It's going to be a v practical session on how to train an agent from people that have.
- @t2govind (Microsoft)
- Victor Barres (@mercor )
- @zkwentz (Reflection)
- Tao Lin (@radixark )
- @xdotli (Benchflow)
Sign up here:
It'll also be live streamed on HF youtube, so stay tuned.
Show more
tbf, Alfred needs pacing. His lifestyle is out of control.
ICYMI, last week we completed the Training Agents live series with Class 4 where we covered RL envs with 2 demos training coding agents!
Video:
Slides with links to all the resources:
Show more
who's working on pacing the fly?
Now I am become fly, the destroyer of worlds
TRL v1.13 is out!!
and you can know train models beyond 1M tokens using it!
We've written a full guide covering it:
tbf, pacing with a lead is a solid way to the win. ask any bike racer.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here:
Show more
sign me up for oai
It's now clear that alignment is critical and won't be solved behind the closed doors of a handful of frontier labs.
So today we're launching the Open Alignment Initiative, led by @Thom_Wolf @huggingface and asking to be part of the "embedded evaluators" program that @DarioAmodei just committed to.
Let's make AI safer by making it more transparent!
Show more
if you want the practical starting point, the repo has the program, runnable sft example, and the path from traces to environment rl:
We finished the Training Agents series. Six live sessions over six months, from evaluating agents to training them inside real environments. All of it is on the Hugging Face YouTube channel and all of the code is open.
Here's what we did and who made it happen:
1. Agentic Evaluations WorkshopWhere agent evals actually stand, and why benchmark scores don't match what people see in use. With Avijit Ghosh and Nathan Habib (Hugging Face), Arvind Narayanan (Princeton), Pierre Andrews (Meta), J.J. Allaire (UK AI Security Institute) and Mahesh Sathiamoorthy (Bespoke Labs).
2. RL for Agents Workshop Environments, rollouts, reward design and the inference bottlenecks that appear when you move from RL for LLMs to RL for agents. With Lewis Tunstall (Hugging Face), Will Brown (Prime Intellect), Ofir Press (Princeton) and Alex Zhang (MIT CSAIL).
3. Training Agents 1: SFT on agent traces Public coding-agent traces turned into prompt/completion data, a TRL + LoRA fine-tune on Hugging Face Jobs, metrics in Trackio, and an honest look at what the first eval numbers can and cannot tell you. Joined by Sergio Paniego and Quentin Gallouédec.
4. Training Agents 2: Distillation Off-policy, on-policy and self-distillation for moving capability from a teacher into a smaller coding agent.
5. Training Agents 3: Reinforcement learning GRPO after SFT: group sampling, verifiable reward functions, reading the reward/KL/length curves, and three experiments, one of them with a deliberately gameable reward so we could watch the hacking happen.
6. Training Agents 4: From reward functions to environments The reward stops being a function and becomes a place the agent acts in. We walked the reset()/step() contract from Gym to LLM agents, built an OpenEnv environment and pushed it to the Hub, plugged it into TRL's GRPOTrainer, then trained a real coding agent (OpenCode) through Harbor with AsyncGRPOTrainer on Hugging Face sandboxes.
The series has passed 300k views. Thank you to every speaker, to the TRL team, and to everyone who showed up live with questions.
Playlist:
Show more
We finished the Training Agents series. Six live sessions over six months, from evaluating agents to training them inside real environments. All of it is on the Hugging Face YouTube channel and all of the code is open.
Here's what we did and who made it happen:
1. Agentic Evaluations WorkshopWhere agent evals actually stand, and why benchmark scores don't match what people see in use. With Avijit Ghosh and Nathan Habib (Hugging Face), Arvind Narayanan (Princeton), Pierre Andrews (Meta), J.J. Allaire (UK AI Security Institute) and Mahesh Sathiamoorthy (Bespoke Labs).
2. RL for Agents Workshop Environments, rollouts, reward design and the inference bottlenecks that appear when you move from RL for LLMs to RL for agents. With Lewis Tunstall (Hugging Face), Will Brown (Prime Intellect), Ofir Press (Princeton) and Alex Zhang (MIT CSAIL).
3. Training Agents 1: SFT on agent traces Public coding-agent traces turned into prompt/completion data, a TRL + LoRA fine-tune on Hugging Face Jobs, metrics in Trackio, and an honest look at what the first eval numbers can and cannot tell you. Joined by Sergio Paniego and Quentin Gallouédec.
4. Training Agents 2: Distillation Off-policy, on-policy and self-distillation for moving capability from a teacher into a smaller coding agent.
5. Training Agents 3: Reinforcement learning GRPO after SFT: group sampling, verifiable reward functions, reading the reward/KL/length curves, and three experiments, one of them with a deliberately gameable reward so we could watch the hacking happen.
6. Training Agents 4: From reward functions to environments The reward stops being a function and becomes a place the agent acts in. We walked the reset()/step() contract from Gym to LLM agents, built an OpenEnv environment and pushed it to the Hub, plugged it into TRL's GRPOTrainer, then trained a real coding agent (OpenCode) through Harbor with AsyncGRPOTrainer on Hugging Face sandboxes.
The series has passed 300k views. Thank you to every speaker, to the TRL team, and to everyone who showed up live with questions.
Playlist:
Show more
Join the agent swarm that finds the ultimate compression algorithm!
Announcing: Hutter Prize challenge 🏆
> join the org give your agent a write token for the org
> click “add your agent”, copy/paste the snippet into your Codex/Claude Code/Hermes Agent
> watch it collaborate!
Show more
Training Agents 4: From reward functions to environments.
