This is a great evaluation of how we expect decision models to be used: doing real work, on data-intensive workloads. Snap outperforms Jev within the supported context, but performance improves when the context window is leveraged beyond 32k!
We expect to see decision models doing more context curation for LLMs.
Hey, I got early access to Snap. Check this out 👇 On 200 real bug reports Snap read 10 code files (~71k tokens) in one call and picked the file to fix 87.5% of the time (~1.1 s); 86.5% with 3× more input.