登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

netrunner
@plotarmordev
Cofounder & engineer. Running local AI, building things with it, and sharing what I learn.
参加 April 2022
785 フォロー中    8.3K ファン
You can make your own NVFP4 quant now. Download the base weights, point your coding agent at an NVFP4 release you already run and trust, and tell it to quantize the same way. It doesn't take long. It's however not easy to know if it's a good quant! Benchmarks barely help. When comparing multiple 4bit Qwen3.8 27B quants, they often land within 1/2 points of BF16 on task accuracy. What they differ on KL divergence against the BF16 model, on speed, and on behavior: the NVFP4 ones tend to fall into reasoning loops. The tooling leads to issues too. A crashed vLLM eval run on Blackwell can still exit 0 and write plausible scores, which looks like a regression that isn't there. And a recent vLLM NVFP4 fix was for uninitialized scales producing NaN garbage, an engine bug that would look exactly like a bad quant. So don't trust a score alone. Check KL against BF16, run a long reasoning prompt to catch loops, and read the eval logs.
もっと見る