Okay my GPT-6.1 Terra benchmark with @VulcanBench is now finished 🖖 And before I talk about the key insights from this one, let me also remind everyone that VulcanBench is a coding benchmark. 100% of the tasks in my eval suite are coding tasks. Having now used Terra all week I can tell you, it's a stellar model for computer use, I'm crazy impressed. But VulcanBench isn't about seeing how good a model is at computer use, it's all about seeing how well a model completes more complex coding tasks. So now for analysis of GPT-6.1 Terra when it comes to coding. I typically say a score between 80 - 90 on Frontier v4 is the sweet spot for routine daily coding tasks. For GPT-6.1 Terra, this means that for coding, you'll want to use Extra High or Max only. Low, Medium, and High don't really make the cut. Looking at cost-per-task, it's a very inexpensive model to use, but when you look at it side-by-side with GPT-6.1 Sol, I'd probably just use Sol. For me, after looking at this data, I still plan to use GPT-6.1 Terra as my go-to for computer use, but I probably won't use it for coding. As usual, as a completely independent benchmark, everything I do at VulcanBench is available for you to review, inspect, and even run yourself if you want to. And the traces now include tool calls so you can really examine exactly how every model I benchmark thinks! Benchmark report: Model page: Public traces on GitHub: Live long and benchmark 🖖