Larry Dial(@classiclarryd):New NanoGPT Speedrun WR at 73.8s (-0.8s) from Mister-dev-oss, CerovazS, MarioPaerle, GabrieleCirillo, and crisostomi on GitHub. They added an incredibly sophisticated fp8 implementation on the MLP down proj fwd pass. 280 lines that include putting the activation quantization step into the prior kernel, delayed amax scaling, and fusing the weight quantization and transpose together.

2026.08.09 22:36

New NanoGPT Speedrun WR at 73.8s (-0.8s) from Mister-dev-oss, CerovazS, MarioPaerle, GabrieleCirillo, and crisostomi on GitHub. They added an incredibly sophisticated fp8 implementation on the MLP down proj fwd pass. 280 lines that include putting the activation quantization step into the prior kernel, delayed amax scaling, and fusing the weight quantization and transpose together.