Register and share your invite link to earn from video plays and referrals.

Silicon Atlas
@Silicon_Atlas
Evidence-first AI semiconductor analysis: what new silicon claims prove, where bottlenecks move, and whether gains survive at system and economic scale.
Joined March 2021
139 Following    2.9K Followers
NVLink's most important spec is not 1.8 TB/s. It is 72, the number of GPUs in one NVLink domain. The card draws the rack and the toll bridge. Here is what it leaves out. Earlier: NVLink inside the node, RDMA across nodes. That node was eight GPUs on one HGX baseboard. NVL72 stretches the island to a full rack: 72 Blackwell GPUs reach each other through switch trays, never through a NIC. Scale-out is InfiniBand or Ethernet NICs between racks. Compare per direction. NVIDIA's 1.8 TB/s per GPU is two-way, so halve it: 900 GB/s each way against 50 GB/s for one 400G NIC per GPU. About 18x (teaching math, not a benchmark). The 36x you may see is double that; it sets the two-way 1.8 TB/s against a one-way line rate. Why serving cares: tensor-parallel all-reduce fires in every layer, and MoE all-to-all fires in every MoE layer, so every decoded token waits on the fabric. A decode step is too short to hide that wait behind compute, so it lands in per-token latency. NVIDIA's wide-EP decode example for DeepSeek-R1 uses 64 GPUs in one domain. That fits inside NVL72. Across eight-GPU nodes it still runs, but most of the all-to-all crosses the bridge. NVLink is not a faster cable. It is a bigger island.
Show more