They also use Muon throughout the network
- Router AdamW for stability
- GR projections AdamW (suspected due to their weird shape)
- ngram table Adam without weight decay
- per head orthogornalization
They trained the model with TP so for the parameter update, they come up with a parameter assigner to balance flops across ranks. The ranks then exchange via all to all
This year, Dragons’ Den (Shark Tank) Greece has a cryptographer. All in for crypto + AI investments, even outside the show.
Premiere today at SKAI TV Greece, 22:10 GR time (20:10 UK, 12:10 California)
AMEC 688012: we survived earnings, always dicey w/ A -Share names. 38% top line gr (vs. LRCX 30%-->50% in Sep q) stable 40% mgns (vs LRCX 52%). News that Sammy & Hynix (China) were testing AMEC machines conveniently dropped on earnings day. Looks good thematically.
I've spent the whole day cleaning up all the people I follow who have used that stupid AI. You’re getting blocked if you tag that stupid ass Gr*k bot under my tweets