Alexander Whedon
@alex_whedon
Building better algorithms.
Co-Founder at @subquadratic
59 Following 24.5K Followers
We've partnered with Appen to evaluate the benchmarks we published last week.
Results are in and we've actually improved across the board.
Link below to the full report.
@AppenResearch independently evaluated @subquadratic's SSA kernel - a learned sparse attention mechanism designed to reduce the quadratic scaling limitations of full attention.
Results at 1M-token context lengths:
- 56.2× wall clock speedup vs. FA2
- 62.8× FLOP reduction (validated via torch.profiler, <4% variance from theoretical)
- 95.6% average score across RULER tasks at 128K
- 86.2% average score on the hardest MRCR 8-needle bucket (512K–1M contexts)
- 81.8% SWE-Bench Verified resolved rate
Full report:
Show more
Hey, folks! We have been blown away by the response to SubQ and the SSA breakthrough over the last 48 hours.
It is awesome to see how many people are responding to our mission of creating more efficient algorithms to create better models.
We are working hard to firm up our release timeline and will share more very soon.
We will also share additional data and third-party validation in our model card next week.
If you have questions, please post them in the thread, and I'll do my best to respond!
Above all, THANK YOU! The support, feedback, and discussion from this community have been inspiring.
Show more
Introducing SubQ - a major breakthrough in LLM intelligence.
It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA),
And the first frontier model with a 12 million token context window which is:
- 52x faster than FlashAttention at 1MM tokens
- Less than 5% the cost of Opus
Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention).
Only a small fraction actually matter.
@subquadratic finds and focuses only on the ones that do.
That's nearly 1,000x less compute and a new way for LLMs to scale.
Show more
We finally have swag.
Watching training jobs be like…
#ICLR2026# is this week! Join Subquadratic during the conference to chat about research, the future of the industry, and the underlying assumptions of AI that need to be challenged.
Drinks on us 🍻
Show more
