가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Silicon Atlas
@Silicon_Atlas
Evidence-first AI semiconductor analysis: what new silicon claims prove, where bottlenecks move, and whether gains survive at system and economic scale.
가입 March 2021
139 팔로잉 중    2.9K 팬
NVLink's most important spec is not 1.8 TB/s. It is 72, the number of GPUs in one NVLink domain. The card draws the rack and the toll bridge. Here is what it leaves out. Earlier: NVLink inside the node, RDMA across nodes. That node was eight GPUs on one HGX baseboard. NVL72 stretches the island to a full rack: 72 Blackwell GPUs reach each other through switch trays, never through a NIC. Scale-out is InfiniBand or Ethernet NICs between racks. Compare per direction. NVIDIA's 1.8 TB/s per GPU is two-way, so halve it: 900 GB/s each way against 50 GB/s for one 400G NIC per GPU. About 18x (teaching math, not a benchmark). The 36x you may see is double that; it sets the two-way 1.8 TB/s against a one-way line rate. Why serving cares: tensor-parallel all-reduce fires in every layer, and MoE all-to-all fires in every MoE layer, so every decoded token waits on the fabric. A decode step is too short to hide that wait behind compute, so it lands in per-token latency. NVIDIA's wide-EP decode example for DeepSeek-R1 uses 64 GPUs in one domain. That fits inside NVL72. Across eight-GPU nodes it still runs, but most of the all-to-all crosses the bridge. NVLink is not a faster cable. It is a bigger island.
더 보기