Andrei Panferov @ICLR(@black_samorez):Disaggregated quantization for LLMs 🧵 LLM prefill and decode can be processed by separate weights in different precisions. Boosts speed AND accuracy. Offloading makes it zero-overhead on long sequences for dense models. Applicable to existing quantized checkpoints. Paper: https://t.co/zr7fQm5SPF Code: https://t.co/S3SGKBprjZ Qwen3.8 NVFP4 prefillers:

Andrei Panferov @ICLR

@black_samorez

ML PhD Candidate @ISTAustria and @EllisForEurope

参加 September 2020

185 フォロー中 212 ファン

Andrei Panferov @ICLR@black_samorez

2026.09.28 18:07

Disaggregated quantization for LLMs 🧵 LLM prefill and decode can be processed by separate weights in different precisions. Boosts speed AND accuracy. Offloading makes it zero-overhead on long sequences for dense models. Applicable to existing quantized checkpoints. Paper: Code: Qwen3.8 NVFP4 prefillers: