Alexander Whedon
@alex_whedon
Building better algorithms.
Co-Founder, CEO at @subquadratic
85 Following 26.2K Followers
SubQ Snap does well in what TypeSafe calls the "Danger Zone", context above 32k. 85% of tokens processed by Snap are from requests above 32k, with many shipping data-intensive features to prod. SSA keeps long context fast and cheap.
We are open-sourcing Decision LCR (Snap 74%, Jev 58%), which mimics real-world tasks and is adapted from AA-LCR.
Because long-context benchmarks for decision models don't really exist, we built one of our own by adapting Artificial Analysis' Long-Context Reasoning benchmark, which we think does a better job mimicking real-world tasks than many LLM long-context benchmarks, to a decision format. We are open-sourcing this today.
Decision LCR:
Adapted from Artificial Analysis's AA-LCR. 100 human-written questions over real 72K–115K-token document sets, converted into 118 decisions, each run with three option orderings to control for position bias. Chance is 19.2, and models score 24–25 with the documents removed.
We'd love for others to run it, use it and improve it. If you work on long-context evaluation or decision models, we'd value your feedback and collaboration.
To learn more about SNAP and our evaluation process, check out the blog:
Show more
Yesterday, we released Snap, the first fully sub-quadratic language model that is generally available and in production at scale, and it is consistently winning at the frontier of decision model space in the real world, and in prod!
Transformers have inverse economies of scale. As the input size grows, the compute you need even on a per-token basis also grows. 10x the input requires 100x the attention compute.
Our model has constant compute-per-token no matter what the input size is, during both training and inference.
This enables us to make smarter models with divergent capabilities and economics, no matter what the model category is.
The community response so far has been amazing! Here are some of the cool things we have seen people build:
- coding agent memory
- UI design
- recommendation systems
- legal contract review
- entity recognition
- academic research curation
- bug finder
- etc.
We can't wait to see what you build next with Snap.
Show more
We use SkyPilot to help manage our multi-cloud GPU clusters for training and inference. The team goes above and beyond to address our niche feature requests as we address problems other labs don’t face. Thanks for helping make this launch a success!
Show more
Congratulations @subquadratic team on launching SubQ Snap! 🎉
Snap is served on SubQs GPU clusters by SkyPilot Platform, scaling to 11B+ tokens/hr through custom routing and inference optimizations tuned for long context.
Proud to be the stack for teams that want to own their intelligence.
Show more
This is a great thread from Jon. Jon and CRED were the first to not only test Snap but ship it to prod, and it took them only an HOUR! These results were exciting to us, because we’ve been doing a lot of work in the finance space with our LLM. Snap was equal to or better than Jev on six out of seven use cases while having better probability calibration (common feedback). The different was even bigger against the OpenAI Decisions API. More on this exploration together is coming soon!
Show more
The race to build the best AI decision-making model just got interesting!
We ran @subquadratic’s new model in production before launch, head-to-head against @typesafeai Jev, @OpenAI and Laya. The winner was the one that knew when it didn’t know. 🧵
Show more
Jake's analyses are always thorough and intriguing! Thanks for benchmarking our model!
NEW SOTA DECISION MODEL: SubQ Snap
SubQ Snap absolutely crushed the S1 Bench, beating Jev!! Insane.
1,999 decisions, 0 errors. Macro accuracy 0.825, the best full run the bench has ever recorded. Jev scores 0.775. Mercury Decide, the previous best API model, scores 0.802.
Subset scores: paws 0.940, vitaminc 0.917, boolq 0.910, massive 0.909, aegis2 0.856. Only helpsteer2 lags at 0.416, and that subset drags everyone down. To be fair, none of the other decision models do well on helpsteer2... And it seems to have done better than Jev.
Subquadratic claimed 61.7 vs Jev's 57.9 on their own Decision Index. On an independent battery it is not even that close. It beat its own benchmarks by a lot.
It didn't hit the speed benchmark on my bench but the accuracy more than makes up for it. About a second per decision, and the entire 1,999-question battery cost roughly 3 cents of input tokens, output free. To be fair, I felt it was the fastest but my agent said it was a bit slower. Nevertheless, this is the SOTA decision model.
Hosted API only and still a preview model, so the open questions are weights and what the 384K context does on long inputs. But as a decision model, this is the new bar.
This is my new go to decision model.
Full results:
Show more
This is a great evaluation of how we expect decision models to be used: doing real work, on data-intensive workloads. Snap outperforms Jev within the supported context, but performance improves when the context window is leveraged beyond 32k!
We expect to see decision models doing more context curation for LLMs.
Show more
Hey, I got early access to Snap. Check this out 👇 On 200 real bug reports Snap read 10 code files (~71k tokens) in one call and picked the file to fix 87.5% of the time (~1.1 s); 86.5% with 3× more input.
Show more
Introducing SubQ Snap, the first long-context decision model for real enterprise work.
Snap is the only decision model or LLM where compute per token stays constant as context grows.
• 384K tokens of multimodal context, 12x larger than Jev.
• 61.7 on Decision Index v0.2.1 vs Jev’s 57.9, outperforming on 13 of 16 enterprise benchmarks.
•Processes 100K tokens and returns a decision in under one second.
•$0.05/MM input tokens. Output is free.
Available today to anyone.
The first 10,000 sign-ups get 100M free input tokens.
We’ll likely hit capacity fast and will add users as we add compute. Compute offers welcome 🙂.
Latency will be variable as we tune the platform to this new model and category of users.
Model speeds are 80ms for minimal context, 250ms for 32k, 660ms for 100k, and 2.5 seconds for 384k.
Visit:
Show more
Grateful for this group of radicals! and dessert.
Great conversation with @craigss at @EyeOn_AI about how the industry spends too much time talking about compute and data-centers, when we should be talking about more creative, more efficient ways to scale AI.
We cover:
- The hidden cost-multiplier within every model in production today.
- How the scaling laws of transformer-based models have become the biggest bottleneck in AI.
- How our team at Subquadratic solved the problem with a new model architecture that is 128x more efficient than frontier models at 2M tokens without compromising on performance.
Thanks to Craig and the Eye on AI team for the thoughtful conversation.
Check out the full episode below.
Show more
New Podcast Alert.
Today on #GAEATalks#, @alex_whedon and Graeme Scott go deep on the macroeconomics of AI, why an efficiency-focused architecture is not a nice-to-have but an inevitability, and the two fundamental flaws of Transformer based models that are quietly capping what AI can do.
Click here for the full episode.
Show more
"The more data you send to the model the less efficient it gets.
This is a major problem that leads to deficiencies in the model's ability to reason over large amounts of data, it limits the amount of training that we can do on a fixed amount of compute, and it creates skyrocketing costs which we're seeing very much in the market today.
As someone who has been working in the space for 10 years, this was a very important problem to solve.”
@alex_whedon & @Justin_SubQ from @subquadratic had a great conversation with @sophiadew on @MTSlive.
Check out this clip from their interview:
Show more
We had a great conversation with @sophiadew x on @MTSlive.
We talked about why scaling AI isn't just about bigger models, but better architecture and what we're building next at @subquadratic.
Check it out.
Show more
Thanks for joining us in SF! We met folks working on all sorts of interesting algorithms research and hardware problems. We will do another event soon!
One of the key insights from our SubQ 1.1 Small technical report ( ) was that super long-context pre-training decreased the reliance on super-long-context post-training to enable super-long-context modeling capabilities. Million-token-plus pre-training enabled the model the extrapolate post-trained capabilities to larger-than-trained lengths. For example, we pre-trained a model with one-million-token inputs and then did post-training at or below one million tokens, and it was able to perform NIAH with high accuracy at multi-million-token lengths. Read more in our technical report!
Show more
We appreciate MIT Technology Review taking a look at what we're building at Subquadratic! We're just getting started.
We've partnered with Appen to evaluate the benchmarks we published last week.
Results are in and we've actually improved across the board.
Link below to the full report.
@AppenResearch independently evaluated @subquadratic's SSA kernel - a learned sparse attention mechanism designed to reduce the quadratic scaling limitations of full attention.
Results at 1M-token context lengths:
- 56.2× wall clock speedup vs. FA2
- 62.8× FLOP reduction (validated via torch.profiler, <4% variance from theoretical)
- 95.6% average score across RULER tasks at 128K
- 86.2% average score on the hardest MRCR 8-needle bucket (512K–1M contexts)
- 81.8% SWE-Bench Verified resolved rate
Full report:
Show more
Hey, folks! We have been blown away by the response to SubQ and the SSA breakthrough over the last 48 hours.
It is awesome to see how many people are responding to our mission of creating more efficient algorithms to create better models.
We are working hard to firm up our release timeline and will share more very soon.
We will also share additional data and third-party validation in our model card next week.
If you have questions, please post them in the thread, and I'll do my best to respond!
Above all, THANK YOU! The support, feedback, and discussion from this community have been inspiring.
Show more
Introducing SubQ - a major breakthrough in LLM intelligence.
It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA),
And the first frontier model with a 12 million token context window which is:
- 52x faster than FlashAttention at 1MM tokens
- Less than 5% the cost of Opus
Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention).
Only a small fraction actually matter.
@subquadratic finds and focuses only on the ones that do.
That's nearly 1,000x less compute and a new way for LLMs to scale.
Show more
We finally have swag.
Watching training jobs be like…
