Tri Dao
@tri_dao
Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
Joined May 2012
665 Following 45.4K Followers
SSMs had the nice asymptotics but kept missing out on the inference goodies Transformers get for free like spec decode and quantization. Turns out we were just not caching the right thing. Same IO-aware trick behind FlashAttention, Mamba scan, paged KV. A lot of the big wins weren’t new architectures, just people taking memory IO seriously.
Show more
