Something worth more discussion for anyone who compares AI chips on FLOP/s.
Epoch's new Inference Performance Index puts a GB300 at about 25x an H100's inference capacity per chip on agentic workloads. On 8-bit FLOP/s the gap is only 2.5x, so the spec sheet understates it by about 10x.
The difference is memory bandwidth and chip-to-chip communication. When an agent is holding a long context and streaming tokens, those can matter more than raw arithmetic.
Google's new TPU 8i Max is built around the same point. It doubles the HBM bandwidth of TPU 8i, because once dozens of agents work across a codebase at once, memory bandwidth becomes the bottleneck.
The newest racks add far more agent capacity than their FLOP/s headline suggests. If you're sizing the buildout for agents, count how many requests each rack can serve at once, and the case for the newest systems gets a lot stronger