注册并分享邀请链接,可获得视频播放与邀请奖励。

Yannick Monye
@keennay
加入 April 2017
1.2K 正在关注    11.6K 粉丝
nvidia/GLM-5.3-Flash-NVFP4 (DFlash2) Fits on 2x H200s. Pre-fill & decode throughput data were much closer to RTX Pro 6Ks than I expected given the lack of native FP4 2x H200 - 195 tok/s: c1, 512k ctx - SGLang - 150 tok/s: c1, 512k ctx - vLLM - 619 tok/s: c8, 64k ctx - SGLang - 621 tok/s: c8, 64k ctx - vLLM 4x RTX Pro 6000 - 167 tok/s: c1, 512k ctx - SGLang - 137 tok/s: c1, 512k ctx - vLLM - 620 tok/s: c8, 64k ctx - SGLang - 599 tok/s: c8, 64k ctx - vLLM
显示更多