Sebastian Raschka
@rasbt
ML/AI research engineer. Ex stats professor.
Author of "Build a Large Language Model From Scratch" ( & reasoning (
๊ฐ์
October 2012
1.2K ํ๋ก์ ์ค 511K ํฌ
Nice case study on using optimized functions whenever possible (except for educational purposes, though ๐)
By switching from a hand-rolled GELU to PyTorch's built-in one, I improved my LLM training speed -- and much more than I expected, from 21,000 tokens/second to 25,000!
