AJ(@ItsmeAjayKV):if you're on llama.cpp run Qwen3.8-27b with -spec-default --spec-type draft-mtp Since it does take its sweet time to think, might as well let it think fast. Run with mtp, you don't need a sep drafter, getting 2x speed on decode now, totally worth it. full command i'm using on my 3090 ./build/bin/llama-server -m "/qwen-3.8/Qwen3.8-27B-Q4_K_M.gguf" --host 127.0.0.1 --port 8080 -ngl 999 -fa on --jinja -np 1 -t 12 --alias qwen3.8-27b-q4 --spec-default --spec-type draft-mtp --cache-type-k q8_0 --cache-type-v q8

@ItsmeAjayKV

Bullish on local AI, llm finetuning, abliteration, llama.cpp OSS contributions GPUmaxxing: 1x 3060, 1x 3090 N4 (日本語勉強中) 🎌

加入 September 2016

683 正在关注 3.5K 粉丝

AJ@ItsmeAjayKV

2026.08.14 18:35

if you're on llama.cpp run Qwen3.8-27b with -spec-default --spec-type draft-mtp Since it does take its sweet time to think, might as well let it think fast. Run with mtp, you don't need a sep drafter, getting 2x speed on decode now, totally worth it. full command i'm using on my 3090 ./build/bin/llama-server -m "/qwen-3.8/Qwen3.8-27B-Q4_K_M.gguf" --host 127.0.0.1 --port 8080 -ngl 999 -fa on --jinja -np 1 -t 12 --alias qwen3.8-27b-q4 --spec-default --spec-type draft-mtp --cache-type-k q8_0 --cache-type-v q8_0

显示更多