Heat is a known issue in HBM, which makes this slide interesting from @SamsungSemiUS as a potential way to handle the increased thermal challenges. I'm also intrigued on the notion that each cHBM base die will not be created equal and performance will likely vary.

The person who presented for Samsung is someone who moved from Samsung S.LSI to the memory side as an HBM “relief pitcher.” He came to present at Hot Chips today, but I had other schedules so I couldn’t meet him in person—I only touched with him yesterday in advance. He has cleared one hurdle and is now pushing zHBM as the next stage. Three-step evolutionary roadmap: Phase #1# (offloading & optimization): reclaims xPU silicon area by adopting compact D2D PHYs and offloading memory controllers to the B-die. Phase #2# (functional expansion): integrates value-added logic—such as telemetry sensors, external memory expansion, and PEs—directly into the B-die. Phase #3# (true 3D convergence): eliminates 2D/2.5D interfaces via zHBM, achieving direct vertical integration of xPU logic and DRAM stacks for maximum energy efficiency. Big thanks to Tae Kim (@firstadopter) for posting the photos live from the venue—those shots let us clearly see the actual slides. From the slides: - The industry is rapidly moving toward tightly coupled AI memory solutions. The key challenge for AGI and inference is maximizing tokens per second within strict power limits. zHBM is presented as the ultimate solution: true 3D vertical integration of the xPU and C-die stack that completely eliminates the 2.5D interposer. - Key differentiators of zHBM are distributed I/Os and a true 3D structure. Distributed I/Os minimize data travel distance inside the stack, the 3D structure removes conventional 2D interfaces (HBM PHY or D2D), and the design targets ultra-low power consumption. - The primary advantage is a dramatic reduction in I/O power. By removing SERDES (data align/DQ I/O) overhead, zHBM delivers about 70% lower power versus HBM5. At the system level (1× GPU + 4× memory stacks, assuming 1200 W GPU), this translates into roughly 230% higher DRAM bandwidth, a ~100 W saving, and an 8.3% improvement in overall power efficiency.

@IanCutress Posting FLOPS, GB, GB/s, and W is not a meme dump, it's what every single respectable chip maker does. Compare the recent Cerebras press to the Etched one. Etched posted a video where they put an empty wooden box in a truck and said don't worry our lead investor thinks it's good.