FEELING HOT HOT HOT!
🎙️ NEW EPISODE: OpenAI's Jalapeño!
Vik and Austin react to
@OpenAI's Jalapeño inference chip, announced at Hot Chips by
@rh00 @cdleary.
- "Dark silicon is cheaper than idle accelerators": one balanced chip beats a GPU + LPU pair?
- Nine months from RTL to tapeout, a Blackwell-class chip designed with AI for EDA
- NUMA-style local HBM slices per accelerator fix the "operands arrive late" problem
- Broadcom ESUN scale-up: 128 chips per rack at 600 GB/s, 2,048 chips across 16 racks at 200G (still scale up!)
- The "regret factor": opportunity cost of missing a future model beats the cost of generality
Plus the extra spicy questions: should OpenAI sell it, how much of the RTL was AI-written, and where is Anthropic's chip?
Chapters:
0:00 Hot Chips Reaction
2:32 Designing for User Experience
11:16 A Generalized Inference Chip
14:18 The Foundry-IDM Analogy
18:42 The 'Regret Factor'
21:02 The 9-Month Design Cycle
23:45 Challenging the Two-Chip Solution
35:08 Solving HBM Underutilization
36:46 The NUMA Architecture Solution
39:28 System-Level ESUN Networking
42:06 Dark Silicon vs. Idle Accelerators
49:08 A Wake-Up Call for the Industry
52:59 Where's Anthropic's Chip?
Get more of Austin and Vik daily, free!
Sign up:
Connect with Vik and Austin:
Vik's Paid Substack:
Austin's Paid Substack:
@austinsemis @vikramskr