Zyphra
@ZyphraAI
Full stack open superintelligence
Joined March 2021
40 Following 14.9K Followers
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.
ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
Show more
