new course just dropped: CS 312: Deep Learning Alchemy w/ @stanfordnlp !
A very interesting new class concept from @tatsu_hashimoto :
- Assignments are worth 0% of the grade
- 85% of your course grade is 40-minute quizzes where you get a code diff for an experiment you've never run and ask you to predict validation loss differences
- The premise? that prediction == understanding. If we don't have any intuition as to the outcomes of an experiment, then it's often safe to say we don't understand it (or at least, it means we don't have beyond surface level knowledge).
And also, everything is public (in the style of cs336!)
(Answer to the question below in, and links 🧵)
Show more
On KDA prefill (Kimi K3's attention), llms independently writes kernel that gets a 2.05x speedup over official FlashKDA on B200; and it's already merged and running in sglang. this kernel has 23 shape-specialized dispatch options and is the best performing kernel open source'd (in this workload).
But how?
We studied flashinfer-ai/flashinfer to figure out exactly how these kernels are produced.
Inside: Many thousands of lines of straightline frozen cuda + 1/3 inline ptx
Explanining why in 🧵, and code explainers (1/n)
Show more
