註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Yaroslav Bulatov
@yaroslavvb
@southparkcommons (early OpenAI, Google Brain, Meta)
加入 February 2011
1.2K 正在關注    11.8K 粉絲
@_arohan_ I've been thinking why Muon wasn't tried earlier, I suspect it's because it didn't fit into traditional preconditioning mindset. It's weird to precondition current gradient by using statistics computed on the same batch, you would expect these statistics to be quite bad
顯示更多