Okay, well something really neat happened today,
@glwillen offered to run
@VulcanBench on his local machine, which has quite a it more horsepower than mine, so we can run a solid Qwen3.8-27B benchmark across effort levels on eval suite 3.
Excited to see the results, will also benchmark on my machine but I've had a lot of ppl tell me that my little M1 Studio with 32GB of RAM might not be quite up to snuff to run Qwen3.8-27B locally, and I want to make sure to allow it to really show what it can do.
Will be sharing the benchmark on my machine as well, and looking at running in Qwen Coder to compare all three!
So soon(ish) I'll have:
1. Qwen3.8-27B running locally on decent specs
2. Qwen3.8-27B running locally on specs that match more closely to what most ppl have at home today
3. Qwen3.8-27B running through Qwen's own harness
If anyone knows someone on the Qwen team, would love to see if I can get some credits/a sub account for testing 🙏