Keller Jordan
@kellerjordan0
CIFAR-10 fanatic
Pretraining @OpenAI OpCo LLC.
452 Following 19.1K Followers
PSA: Most biglab people now read almost zero papers and understand ICLR/ICML/NeurIPS to be mainly full of overclaims & fraud. (but there are a few diamonds in the rough of course)
New modded-NanoGPT optimization benchmark result: @wen_kaiyue has improved upon both the Muon and AdamW baselines, by replacing their weight decay with hyperball optimization. The new record is 3325 steps.
Show more
