Andrei Panferov @ICLR
@black_samorez
ML PhD Candidate @ISTAustria and @EllisForEurope
185 Following 212 Followers
Disaggregated quantization for LLMs 🧵
LLM prefill and decode can be processed by separate weights in different precisions. Boosts speed AND accuracy. Offloading makes it zero-overhead on long sequences for dense models. Applicable to existing quantized checkpoints.
Paper:
Code:
Qwen3.8 NVFP4 prefillers:
Show more
