🎙️ NEW EPISODE: @Microsoft Maia 300 with Saurabh Dighe Why Maia 300 makes memory bandwidth and capacity a first-order design principle. - Moving data to feed the compute engine can cost two orders of magnitude more energy than the compute itself - Maia 200 had six HBM3E stacks at 7 TB/s. Maia 300 has eight HBM4 stacks at 18 TB/s - Compute grows about 35% while rack-level memory bandwidth grows 5x - 7x token throughput per rack over Maia 200 at 2.5x perf per watt - Prefill and decode are disaggregated, and the KV cache moves over the scale-up fabric instead of the slower scale-out network - Memory interface chiplets "extend the shoreline" so the TSMC N2P compute die stays one efficient die This episode is presented by @CrusoeAI. Crusoe's Serverless Fine-Tuning trains an open model like Llama, DeepSeek, or GLM on your own data, and you pay per million tokens, not per GPU hour. About $2 buys roughly 5M training tokens on smaller models. Get $5 in credit to start: Chapters: 0:00 Intro: Saurabh Dighe 1:39 Microsoft's Silicon Philosophy 5:52 Workload Shift: Agentic AI 9:32 The Data Movement Problem 14:20 Crusoe Ad Read 17:47 Maia 300 Specs 24:07 Chiplets & HBM4 29:12 Prefill/Decode Disaggregation 30:33 Two-Tiered Network 35:45 NPO vs. CPO 36:46 Rack-Scale Power & Cooling 41:35 Agentic Kernel Optimization 49:08 What's Next: Maia 400 Get more of Austin and Vik daily, free! Sign up: Connect with Vik and Austin: Vik's Paid Substack: Austin's Paid Substack: @austinsemis @vikramskr