It's official, Astra has saturated some of the most advanced benchmarks out there.
Excited to test it on our new eval suite once we have access.
Really hoping we can stump it, but also happy for the challenge of iterating and building new evals for the beast of a model Astra sounds like it is.
Live long and benchmark 🖖
I love waking up to running benchmarks, and checking in on them.
Every morning kinda feels like Christmas.
Here's a quick update on the Fable 5.1 benchmark on my new eval suite, feels good to be able to stump this model a bit.