Morgan(@morganlinton):Building evals is hard, and only getting harder, but I also think it's some of the most interesting and exciting stuff I've ever done. Going much deeper with my eval suites at @VulcanBench, very excited to share what's next. But first, within the next hour, I will be sharing the results from my first Harness Benchmark, really excited about this one, and it was hard in a different way since I had to keep updating guardrails. Now, I have what I think is one of the cleanest ways to benchmark not just a model itself, but a model in a harness. Stay-tuned!

2026.08.17 14:42

Building evals is hard, and only getting harder, but I also think it's some of the most interesting and exciting stuff I've ever done. Going much deeper with my eval suites at @VulcanBench, very excited to share what's next. But first, within the next hour, I will be sharing the results from my first Harness Benchmark, really excited about this one, and it was hard in a different way since I had to keep updating guardrails. Now, I have what I think is one of the cleanest ways to benchmark not just a model itself, but a model in a harness. Stay-tuned!