Yannick Monye(@keennay):nvidia/GLM-5.3-Flash-NVFP4 (DFlash2) Fits on 2x H200s. Pre-fill & decode throughput data were much closer to RTX Pro 6Ks than I expected given the lack of native FP4 2x H200 - 195 tok/s: c1, 512k ctx - SGLang - 150 tok/s: c1, 512k ctx - vLLM - 619 tok/s: c8, 64k ctx - SGLang - 621 tok/s: c8, 64k ctx - vLLM 4x RTX Pro 6000 - 167 tok/s: c1, 512k ctx - SGLang - 137 tok/s: c1, 512k ctx - vLLM - 620 tok/s: c8, 64k ctx - SGLang - 599 tok/s: c8, 64k ctx

2026.10.20 15:56

nvidia/GLM-5.3-Flash-NVFP4 (DFlash2) Fits on 2x H200s. Pre-fill & decode throughput data were much closer to RTX Pro 6Ks than I expected given the lack of native FP4 2x H200 - 195 tok/s: c1, 512k ctx - SGLang - 150 tok/s: c1, 512k ctx - vLLM - 619 tok/s: c8, 64k ctx - SGLang - 621 tok/s: c8, 64k ctx - vLLM 4x RTX Pro 6000 - 167 tok/s: c1, 512k ctx - SGLang - 137 tok/s: c1, 512k ctx - vLLM - 620 tok/s: c8, 64k ctx - SGLang - 599 tok/s: c8, 64k ctx - vLLM