jianlin.su(@Jianlin_S):Steepest Descent on Manifolds: 6. Muon + Double Rotation https://t.co/tGfnlzfYv6 Introduces MuonR — a Muon variant that constrains updates to left & right rotation matrices. This preserves the singular value distribution of weights, providing a clean, elegant way to maintain training stability.

jianlin.su

@Jianlin_S

Grad&Clip&EM is all you need @Kimi_Moonshot Blog: , Cool Papers: Interesting link:

加入 February 2025

21 正在關注 11.6K 粉絲

jianlin.su@Jianlin_S

2026.06.08 04:14

Steepest Descent on Manifolds: 6. Muon + Double Rotation Introduces MuonR — a Muon variant that constrains updates to left & right rotation matrices. This preserves the singular value distribution of weights, providing a clean, elegant way to maintain training stability.

157

轉發到社區

熱門用戶