Qwen3.5-4B running CPU-ONLY at up to ~11+ tps on a Ryzen 5 laptop. No GPU. Running models in RAM and CPU is something the Qwen4 team is already thinking about.
๐ก I built this standalone .exe because businesses are already deploying local models to save money and ensure privacy. (thumb drive friendly - just drag drop and doubleclick)
My llama.cpp recipe ๐
๐ง Qwen3.5-4B Q4_K_M GGUF
โ๏ธ Ryzen 5 7540U โ 6C/12T
๐งต --threads 9
๐งต --threads-batch 12
โก --prio 2
๐ --poll 50
๐ฆ --batch-size 2048
๐ฆ --ubatch-size 512
๐ --flash-attn on
๐ง KV cache: q4_0 / q4_0
๐ง --repack
๐พ --mmap
๐ค --parallel 1
๐ซ --device none
๐ซ --gpu-layers 0
๐ซ KV/op GPU offload
๐ซ MTP OFF
Interesting result ๐ MTP=3 was slower (~10 tps in benchmarks).
Plain decode + 9 threads + Q4 KV hit ~11.4 tok/s.
That's about 10% faster just from tuning llama.cpp โ on a basic laptop CPU.