Adithya S K(@adithya_s_k):You can now do RL weight sync over HF Buckets. > Async GRPO + LoRA in TRL > train on one Job, serve vLLM on others > sync just the small delta adapter through a Bucket. Trainer, inference and storage, all on @huggingface infra. Beautifully optimised loop by @DirhoussssiAmine 👇 3h27 → 53 min for the same reward.

2026.09.18 09:54

You can now do RL weight sync over HF Buckets. > Async GRPO + LoRA in TRL > train on one Job, serve vLLM on others > sync just the small delta adapter through a Bucket. Trainer, inference and storage, all on @huggingface infra. Beautifully optimised loop by @DirhoussssiAmine 👇 3h27 → 53 min for the same reward.

显示更多

Dirhousssi Amine@DirhousssiAmine

2026.09.14 11:45

We ran async GRPO across HF Jobs with the trainer and vLLM on separate machines, no NCCL, and no shared disk. The adapter is mounted at the same path in every Job. LoRA training has landed in TRL's AsyncGRPOTrainer 🎉🎉🎉 The trainer now syncs only the adapter to vLLM ~few MB for a 1.5B model. That changes what the setup can look like. We ran async GRPO across HF Jobs with the trainer and vLLM on separate machines, no NCCL, no shared disk. The adapter travels through a 🪣 mounted on the same path in every Job. Blogpost 🧵

显示更多