VulcanBench(@VulcanBench):I wanted to share a bit more about the benchmark results of our first voice model comparison with VulcanBench. We asked Grok Voice Think Fast 2.0 and GPT Realtime the same 200 questions, typed vs spoken. Here's the results: Grok: 99.0% text → 95.7% audio (+3.3 pp) GPT Realtime: 97.5% → 93.5% (+4.0 pp) Real, but small. And here's a little thread for anyone that wants to dive deeper into the details. A voice model benchmarking thread 🧵

VulcanBench

@VulcanBench

Open Source LLM benchmarking tool, focused on real world tests, large codebases, full transparency. An Open Source project by @morganlinton.

Joined March 2020

28 Following 842 Followers

VulcanBench@VulcanBench

2026.08.01 16:05

I wanted to share a bit more about the benchmark results of our first voice model comparison with VulcanBench. We asked Grok Voice Think Fast 2.0 and GPT Realtime the same 200 questions, typed vs spoken. Here's the results: Grok: 99.0% text → 95.7% audio (+3.3 pp) GPT Realtime: 97.5% → 93.5% (+4.0 pp) Real, but small. And here's a little thread for anyone that wants to dive deeper into the details. A voice model benchmarking thread 🧵