Ivan Zhou (@ivanzhouyq)

2026.09.03 20:02

We evaluated GPT 6 Astra at @databricks and it claims the new SOTA on our OfficeQA Pro & Pro V2 benchmarks, using our Genie harness. It also improves significantly from gpt 5.6 sol on the $ per task. Throughout our benchmarks, it shows a clear step up on data reasoning and document understanding for enterprises. It also has become my daily driver on @omnigent_ai. It is a great model to collaborate with and get things done reliably. Congrats to @OpenAI. The model will come to our Unity Gateway and smart routing soon!

436

Forward to community

Ivan Zhou@ivanzhouyq

2026.08.13 20:50

We are in a Cambrian explosion of frontier models. Even as a researcher who spends a lot of time evaluating models, I find it hard to keep up. New models are arriving constantly, and the mental overhead of choosing the right setup for each task keeps growing. That motivated us to build Smart Routing: an intelligent layer that automatically matches each task to the right model based on its complexity and the capabilities it needs. Our early results are promising: frontier level quality with 30%+ lower cost, and more than 50% savings on public benchmarks.

Forward to community

Ivan Zhou@ivanzhouyq

2026.08.12 17:07

We worked with @SpaceXAI to evaluate Grok 4.6 on the latest OfficeQA Pro V2 from @DbrxMosaicAI. It achieves the SOTA performance with @databricks's Genie harness! The model is strongest on our document understanding and data reasoning tasks, and it is a very efficient driver!