Zyphra
@ZyphraAI
Full stack open superintelligence
加入 March 2021
40 正在关注 14.9K 粉丝
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.
ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
显示更多
