Zyphra
@ZyphraAI
Full stack open superintelligence
加入 March 2021
40 正在關注 14.9K 粉絲
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.
ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
顯示更多
