Register and share your invite link to earn from video plays and referrals.

Yaroslav Bulatov
@yaroslavvb
@southparkcommons (early OpenAI, Google Brain, Meta)
Joined February 2011
1.2K Following    11.8K Followers
@_arohan_ I've been thinking why Muon wasn't tried earlier, I suspect it's because it didn't fit into traditional preconditioning mindset. It's weird to precondition current gradient by using statistics computed on the same batch, you would expect these statistics to be quite bad
Show more