Do pretraining, SFT, and RL really need to be separate stages? Title: Training Foundation Models with a Single Algorithm URL: This paper shows a framework that runs everything from pretraining to RL together, with one likelihood-based loss and one trainer. Highlights 🧩 RL is likelihood fitting too The MaxRL gradient has the same form as MLE on the distribution of successful completions. So pretraining, midtraining, SFT, and RL differ only in data mixture and timing, and staged training is a special case. 📈 It beats staged training On reasoning puzzles with a 124M model, the best unified configuration beat staged training on all three evaluations. Staged training degrades when midtraining runs past its best budget. 🛡️ Forgets less, learns the next task better With a 0.76B model, held-out corpus loss went below the base model, while staged training raised it. After math, coding RL reached a higher final accuracy for every unified model than for the staged one. Experiments are small-scale, but I find the view of training as "which data, when, in what proportion" compelling. #FoundationModels# #ReinforcementLearning#