注册并分享邀请链接,可获得视频播放与邀请奖励。

Andrei Panferov @ICLR
@black_samorez
ML PhD Candidate @ISTAustria and @EllisForEurope
加入 September 2020
185 正在关注    212 粉丝
Disaggregated quantization for LLMs 🧵 LLM prefill and decode can be processed by separate weights in different precisions. Boosts speed AND accuracy. Offloading makes it zero-overhead on long sequences for dense models. Applicable to existing quantized checkpoints. Paper: Code: Qwen3.8 NVFP4 prefillers:
显示更多
0
5
50
17
转发到社区