VulcanBench(@VulcanBench):I wanted to share a bit more about the benchmark results of our first voice model comparison with VulcanBench. We asked Grok Voice Think Fast 2.0 and GPT Realtime the same 200 questions, typed vs spoken. Here's the results: Grok: 99.0% text → 95.7% audio (+3.3 pp) GPT Realtime: 97.5% → 93.5% (+4.0 pp) Real, but small. And here's a little thread for anyone that wants to dive deeper into the details. A voice model benchmarking thread 🧵

VulcanBench

@VulcanBench

Open Source LLM benchmarking tool, focused on real world tests, large codebases, full transparency. An Open Source project by @morganlinton.

가입 March 2020

28 팔로잉 중 842 팬

VulcanBench@VulcanBench

2026.08.01 16:05

I wanted to share a bit more about the benchmark results of our first voice model comparison with VulcanBench. We asked Grok Voice Think Fast 2.0 and GPT Realtime the same 200 questions, typed vs spoken. Here's the results: Grok: 99.0% text → 95.7% audio (+3.3 pp) GPT Realtime: 97.5% → 93.5% (+4.0 pp) Real, but small. And here's a little thread for anyone that wants to dive deeper into the details. A voice model benchmarking thread 🧵