nvidia/GLM-5.3-Flash-NVFP4 (DFlash2)
Fits on 2x H200s. Pre-fill & decode throughput data were much closer to RTX Pro 6Ks than I expected given the lack of native FP4
2x H200
- 195 tok/s: c1, 512k ctx - SGLang
- 150 tok/s: c1, 512k ctx - vLLM
- 619 tok/s: c8, 64k ctx - SGLang
- 621 tok/s: c8, 64k ctx - vLLM
4x RTX Pro 6000
- 167 tok/s: c1, 512k ctx - SGLang
- 137 tok/s: c1, 512k ctx - vLLM
- 620 tok/s: c8, 64k ctx - SGLang
- 599 tok/s: c8, 64k ctx - vLLM
Show more
so you're telling me tern also integrates with omp through ssh sessions?
yeah tern fucks
