Steve Jurvetson(@FutureJurvetson):As in the brain, "ultra-long contexts may require subquadratic attention. Attend to everything, and the signal gets lost in noise. Too sparse, and we lose the signal. Hierarchical attention may be essential not just for efficiency, but for attention to work at all."

Steve Jurvetson

@FutureJurvetson

Co-founder of Future Ventures and DFJ, supporting passionate founders to forge a better future. Early VC investor in Tesla, SpaceX, Planet, Commonwealth Fusion.

参加 March 2010

70 フォロー中 112.4K ファン

Steve Jurvetson@FutureJurvetson

2026.11.02 23:53

As in the brain, "ultra-long contexts may require subquadratic attention. Attend to everything, and the signal gets lost in noise. Too sparse, and we lose the signal. Hierarchical attention may be essential not just for efficiency, but for attention to work at all." — Zyphra

Zyphra@ZyphraAI

2026.11.02 22:04

Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention. ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵