Zyphra
@ZyphraAI
Full stack open superintelligence
参加 March 2021
40 フォロー中 14.9K ファン
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.
ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
もっと見る
