가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Benjamin Marie
@bnjmn_marie
Independent AI researcher (AI, LLM). Support my work:
가입 June 2019
227 팔로잉 중    7.7K 팬
New release: Two smaller Qwen3.8 27B SLQ models! - A new one targeting EAR 98 - And another one (EAR 97.8) just aiming to beat the official FP8 while being ~10 GB smaller. This time, I searched with smaller group sizes, especially for INT2/INT3, but none were selected. Smaller groups at higher precision are more useful here. GPTQ just isn't good enough at very low precision. I'll try smarter (and more expensive...) quantization for Qwen4. These models using smaller groups could also be slightly slower than previous versions. I didn't check. I think we're close to the limit of what GPTQ can achieve at EAR 98. The models could be even smaller with quantized token embeddings, but a vLLM bug prevents this for Qwen3.5. I avoided relying on a patched vLLM to keep deployment simple. Models:
더 보기