Subbu
@subbdue
I develop AI inference Chips/SoCs for a living. Read my work and subscribe to my newsletter at
52 Following 1.2K Followers
I'll be at OCP this week. Here are the sessions I'm attending in Memory/Storage, Networking, and Systems.
The ones I won't miss:
Mon 5:45pm · Ian Buck, NVIDIA keynote
Tue 11:10am · Partha Ranganathan, Google keynote
Tue 2:50pm · Samsung on CXL pooling and SSD-backed KV cache offload
Thu 8:20am · Marvell on custom HBM, co-designing the base die, D2D power, and package thermals
Full schedule in the images.
Show more
@StragglerLiu , you’re spot on. this is not a HBM4 replacement story yet. You have to think of this in terms of an SRAM accelerator’s roadmap (d-Matrix, Groq or Cerebras). We (d-Matrix) think this 3D DRAM technology is the right progression from our first generation SRAM based accelerator, called Corsair. We can extract SRAM like bandwidth (100 TB/s) but with much more capacity than a SRAM based accelerator.
Show more
Link to the d-Matrix ISCA ‘26 paper explaining how we use 3D DRAM in our accelerator and the issue we faced .. ask me any questions your have
Add on a few more months before improved kernels and meaningful MFU. I write about why this is the case in my deep dive:
The Uncomfortable Truth Behind Deploying the Latest NVIDIA GPUs: MFU, Silent Data Corruption -
Show more
IMPORTANT: it is important to understand that the CoreWeave & Microsoft photos are still Engineering/Quality Samples, and there is still some time before the software stack bring-up finishes & first production tokens are generated. The VR200 & MI455 rack metric to watch out for is time to first at-scale production token TTF-(ASP)-T. You can clearly see in the CW rack photos that none of the scale-out 800G OSFP cages are even populated.
Show more
Micron has seen 6 boom & bust cycles since 2000. In 2019, hyperscaler over-ordering crashed their revenue from $30B to $21B. In 2023, the pandemic unwind cut it in half again, from $30B to $15B.
Has the AI super-cycle finally broken this pattern? Or is a reckoning coming? — especially since it's the same set of buyers, buying for the same reasons.
I analyze Micron through three lenses: Engineering, Finance, and Strategy. The current tailwinds, and the possible headwinds.
Full Article here:
Show more
