Evidence-first AI semiconductor analysis: what new silicon claims prove, where bottlenecks move, and whether gains survive at system and economic scale.
NVLink's most important spec is not 1.8 TB/s. It is 72, the number of GPUs in one NVLink domain.
The card draws the rack and the toll bridge. Here is what it leaves out.
Earlier: NVLink inside the node, RDMA across nodes. That node was eight GPUs on one HGX baseboard. NVL72 stretches the island to a full rack: 72 Blackwell GPUs reach each other through switch trays, never through a NIC. Scale-out is InfiniBand or Ethernet NICs between racks.
Compare per direction. NVIDIA's 1.8 TB/s per GPU is two-way, so halve it: 900 GB/s each way against 50 GB/s for one 400G NIC per GPU. About 18x (teaching math, not a benchmark). The 36x you may see is double that; it sets the two-way 1.8 TB/s against a one-way line rate.
Why serving cares: tensor-parallel all-reduce fires in every layer, and MoE all-to-all fires in every MoE layer, so every decoded token waits on the fabric. A decode step is too short to hide that wait behind compute, so it lands in per-token latency. NVIDIA's wide-EP decode example for DeepSeek-R1 uses 64 GPUs in one domain. That fits inside NVL72. Across eight-GPU nodes it still runs, but most of the all-to-all crosses the bridge.
NVLink is not a faster cable. It is a bigger island.