Register and share your invite link to earn from video plays and referrals.

Yaroslav Bulatov
@yaroslavvb
@southparkcommons (early OpenAI, Google Brain, Meta)
1.2K Following    11.8K Followers
@_arohan_ I've been thinking why Muon wasn't tried earlier, I suspect it's because it didn't fit into traditional preconditioning mindset. It's weird to precondition current gradient by using statistics computed on the same batch, you would expect these statistics to be quite bad
Show more
Google has an internal "let it break" essay about a hero engineer whose hard work ends up being a net negative (by masking the underlying issues). My manager sent me that essay when I was trying too hard to get the collective TensorFlow unit test suite green.
Show more
0
67
6.3K
436
Forward to community