登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | 🔔 Follow for AI & Vibe Coding Tips 👇
参加 July 2023
552 フォロー中    12.5K ファン
A new dense 32B model ran a 239K-token conversation inside a 16GB GPU-memory budget. It compares well to Qwen3.8-27B. And the trick isn't just the ~2.8-bit quant. 54 of its 72 layers don't build a growing KV cache at all. 👀 Blockway released Agens Volundr 32B v1 today. Stats 🧠 32B parameters 📚 262K configured context 💾 ~2.82 bits/weight on the 16GB build 🔓 Apache 2.0 weights ⚙️ MIT-licensed Agens Engine 📦 Official GGUFs already available Special Source 👉 instead of every layer accumulating KV as context grows 72 layers ↓ 54 keep fixed-size recurrent state ↓ only 18 layers maintain growing KV cache That helps with long-context memory Using Agens Engine's hard 16 GiB GPU-memory cap, Blockway fed it a 219K-token document and reports at T=0.6 ⚡ Chat → 57 tps ⚡ Code → 79 tps ⚡ Thinking → 71 tps ⚡ Tool calls → 67 tps They pushed the limits of the 239K-token envelope while GPU memory peaked around 15.5 GiB. At short context, the same profile reaches roughly 86–123 tps depending on workload. And there's already a separate 24GB profile using the 4-bit weights for cards in the RTX 4090 class. 🎯 Volundr attacks the other monster in the room, : the memory cost of long context. ⚠️ Those published 16GB and 24GB performance numbers were measured on a 48GB GPU running under hard memory caps, not yet on physical 16GB/24GB consumer cards. The official GGUFs work today with Blockway's llama.cpp fork. Stock llama.cpp support is still being upstreamed. Link in ALT
もっと見る