Karim Mattar
@MattarARK
AI Research Associate @ARKInvest | AI, Cloud & Semis | Research x Automation | | Disclosure:
加入 September 2024
253 正在關注 7K 粉絲
Rubin has 3.5x the FP4 compute of GB200 but vLLM got 7.8x+ the throughput on an agent benchmark
2.4x the memory bandwidth is doing a lot of that work
Glide or cliff for GB200s?
vLLM now supports NVIDIA Vera Rubin. The early results show more than 7.8x the throughput of GB200 on MiniMax M3 on AgentX.
@inferact, @NVIDIAAI, @RedHat_AI, and the vLLM community have been bringing vLLM up on Rubin since it was announced. Here is where things stand.
🧵 1/5
顯示更多
