注册并分享邀请链接,可获得视频播放与邀请奖励。

Yaroslav Bulatov
@yaroslavvb
@southparkcommons (early OpenAI, Google Brain, Meta)
加入 February 2011
1.2K 正在关注    11.8K 粉丝
@_arohan_ I've been thinking why Muon wasn't tried earlier, I suspect it's because it didn't fit into traditional preconditioning mindset. It's weird to precondition current gradient by using statistics computed on the same batch, you would expect these statistics to be quite bad
显示更多