Yifan Qiao(@yifandotqiao):I did not see TPUs beating GPUs on decode any time soon, until @woosuk_k and the cracked team pulled it off with a single Pallas kernel running all of Kimi K3 at 709 tok/s 🤯 First TPU inference megakernel I know of, and it is open source today! The numbers speak for themselves, and the blog is definitely worth your time 🚀

Yifan Qiao

@yifandotqiao

MTS @inferact | ex-Postdoc at Sky Computing Lab @UCBerkeley | PhD @UCLA | Building efficient systems for AI

Joined January 2023

420 Following 632 Followers

Yifan Qiao@yifandotqiao

2026.09.23 18:42

I did not see TPUs beating GPUs on decode any time soon, until @woosuk_k and the cracked team pulled it off with a single Pallas kernel running all of Kimi K3 at 709 tok/s 🤯 First TPU inference megakernel I know of, and it is open source today! The numbers speak for themselves, and the blog is definitely worth your time 🚀