It's official : We trained NanoGPT in 39.9 seconds.
By percentage reduction in training time, this is a greater improvement than the previous 45 world records combined.
The merged Github PR:
Here's how we do it :
Credit
@DevenPzak who did this alone while simultaneously playing baseball for the polish national squad, taking them to the European Finals.