Yifan Qiao
@yifandotqiao
MTS @inferact | ex-Postdoc at Sky Computing Lab @UCBerkeley | PhD @UCLA | Building efficient systems for AI
420 Following 632 Followers
I did not see TPUs beating GPUs on decode any time soon, until @woosuk_k and the cracked team pulled it off with a single Pallas kernel running all of Kimi K3 at 709 tok/s 🤯
First TPU inference megakernel I know of, and it is open source today! The numbers speak for themselves, and the blog is definitely worth your time 🚀
Show more
DeepSeek V4 Pro on @vllm_project can be 106x cheaper than Opus 5 on the same @SemiAnalysis_ AgentX workload.
We got here with a stack of optimizations, all open in vLLM, and a blog that covers what works and, perhaps more interestingly, what doesn't.
This is why I'm excited about open source. Proud of the @inferact team. Try it on your own agents 🚀
Show more
