Ali Hatamizadeh(@ahatamiz1):I don't believe (https://t.co/KGXo5sSi3s) establishes anything about Muon for RLVR. They run GRPO with Qwen3-1.7B/4B over GSM8K, and report that Muon consistently fails. But as always, the devil is in the details. Just quickly checking their setup in their repo, Muon's per-element update RMS comes out at 5×lr. AdamW's is typically 0.2 to 0.4×lr. So they are effectively running Muon at roughly 20x AdamW's step size at the same LR. And of course they did not sweep the optimal LR for Muon in this recipe. We have seen massive success in both pre-training and post-training experiment for Muon and there's no reason to believe otherwise.

2026.09.14 03:47

I don't believe ( establishes anything about Muon for RLVR. They run GRPO with Qwen3-1.7B/4B over GSM8K, and report that Muon consistently fails. But as always, the devil is in the details. Just quickly checking their setup in their repo, Muon's per-element update RMS comes out at 5×lr. AdamW's is typically 0.2 to 0.4×lr. So they are effectively running Muon at roughly 20x AdamW's step size at the same LR. And of course they did not sweep the optimal LR for Muon in this recipe. We have seen massive success in both pre-training and post-training experiment for Muon and there's no reason to believe otherwise.

显示更多