登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

VulcanBench
@VulcanBench
Benchmarking models across effort levels and harnesses on eval suites with real coding tasks. Because you don't need High or Max effort as much as you think.
参加 March 2020
71 フォロー中    2.7K ファン
Three generations of Sol, and I'll just come out and say it, very impressed with GPT-6.1 Sol, this is an exceptional model. More details below:
Okay, this is definitely one of the most interesting benchmark comparisons I have ever done. GPT 5.6, 6.0, and 6.1 Sol, across all effort levels on @VulcanBench Frontier v4. And honestly, I don't think I even need to give any analysis or explanations because the results really do speak for themselves. But one thing I will highlight is that GPT-5.6 Sol and GPT-6.0 Sol all struggled at Low effort, GPT-6.1 is a beast at Low effort. I am now convinced, GPT-6.1 Sol is a very good model, and you can comfortable use it, at Low effort, and see strong, insanely cost-effective results. I don't say this often but I can tell you, after seeing this data, I'd be crazy not to make this one of my daily drivers. Full details from the benchmark now live on the site: Live long and benchmark 🖖
もっと見る