注册并分享邀请链接,可获得视频播放与邀请奖励。

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | 🔔 Follow for AI & Vibe Coding Tips 👇
加入 July 2023
552 正在关注    12.5K 粉丝
This is a crazy, weird Qwen3.8-27B “uncensored” release. It isn't because it's uncensored, but because the author measured what the guardrail surgery cost to get it uncensored and then preserved MTP. Jonathan Coletti took Qwen3.8-27B and used Heretic model surgery to reduce its refusal behavior. Measured on ~100 held-out test prompts ... 🛑 Base Qwen3.8-27B → w/98 refusals 🔓 Modified model → w/only 12 refusals Meanwhile, the model's capability barely changed ... 🧠 MMLU 83.4 → 83.3 🎯 HellaSwag 82.8 → 82.9 📊 Mean across 4 tests → ~0.5 point drop Abliteration initially dropped Qwen's MTP head and the author then grafted all 15 MTP tensors back from the original Qwen3.8-27B and then verified that the MTP block actually survived GGUF quantization. So the modified model you still get ... 🧠 27B dense Qwen 👁️ multimodal/vision 📚 262K context architecture 🔮 native MTP speculative decoding 🦙 llama.cpp GGUF And mradermacher already has iMatrix quantizations 📦 IQ3_S → 12.7GB 📦 IQ3_M → 12.9GB 📦 Q3_K_M → 13.6GB 👀 📦 IQ4_XS → 15.4GB 📦 Q4_K_M → 16.9GB That makes ~13GB versions especially interesting for 16GB GPU testing. 🔥 ⚠️ File size ≠ VRAM requirement, so don't assume the 15.4GB IQ4 automatically fits comfortably in 16GB with useful context. Also, I have found that “uncensored” model variants don't mean capability is proven unchanged. For example, their published comparison doesn't yet cover coding, math, multilingual, or vision quality. 😁
显示更多
0
4
169
20
转发到社区