A new dense 32B model ran a 239K-token conversation inside a 16GB GPU-memory budget.
It compares well to Qwen3.8-27B.
And the trick isn't just the ~2.8-bit quant.
54 of its 72 layers don't build a growing KV cache at all. ๐
Blockway released Agens Volundr 32B v1 today.
Stats
๐ง 32B parameters
๐ 262K configured context
๐พ ~2.82 bits/weight on the 16GB build
๐ Apache 2.0 weights
โ๏ธ MIT-licensed Agens Engine
๐ฆ Official GGUFs already available
Special Source ๐ instead of every layer accumulating KV as context grows
72 layers
โ
54 keep fixed-size recurrent state
โ
only 18 layers maintain growing KV cache
That helps with long-context memory
Using Agens Engine's hard 16 GiB GPU-memory cap, Blockway fed it a 219K-token document and reports at T=0.6
โก Chat โ 57 tps
โก Code โ 79 tps
โก Thinking โ 71 tps
โก Tool calls โ 67 tps
They pushed the limits of the 239K-token envelope while GPU memory peaked around 15.5 GiB.
At short context, the same profile reaches roughly 86โ123 tps depending on workload.
And there's already a separate 24GB profile using the 4-bit weights for cards in the RTX 4090 class.
๐ฏ Volundr attacks the other monster in the room, : the memory cost of long context.
โ ๏ธ Those published 16GB and 24GB performance numbers were measured on a 48GB GPU running under hard memory caps, not yet on physical 16GB/24GB consumer cards.
The official GGUFs work today with Blockway's llama.cpp fork.
Stock llama.cpp support is still being upstreamed.
Link in ALT