登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

VulcanBench
@VulcanBench
Benchmarking models across effort levels and harnesses on eval suites with real coding tasks. Because you don't need High or Max effort as much as you think.
参加 March 2020
63 フォロー中    2.3K ファン
Seeing a high volume of posts on X about Astra being nerfed. Putting together a plan to test this with VulcanBench to move from anecdotal evidence to a more data-driven approach. Since I benchmarked it launch week, I do have data to compare how it performed at launch vs. now. But VulcanBench only looks at coding, not things like 3d rendering, game design, etc. and a lot of the nerf reports seemed to be focused on this so I might be somewhat limited. Still, worth seeing if we can notice any kind of difference. Let's let the data do the talking and answer the question: is Astra really nerfed?
もっと見る
🚨 GPT-6 Astra (Launch) Vs GPT-6 Astra (Today) Same exact prompt But today’s result looks noticeably worse It genuinely feels like OpenAI nerfed Astra