People always ask: where's the next structural opportunity in AI chips? One of the structural shifts driven by the paradigm change over the past few months is heterogeneous AI inference — and SRAM-route startups led by Cerebras is right at the frontier of this new trend. Every year, Nvidia's GTC conference introduces paradigm-shifting concepts in technology, setting the benchmark for the entire industry. Everyone scrambles to rewrite their roadmaps and copy the homework after GTC. ------------------------ To understand where SRAM-route companies fit in the ecosystem, you first need to look at the workload characteristics of different stages of genAI inference. It breaks down into three parts: Prefill: extremely high compute intensity, low demand on memory bandwidth, moderate-to-high demand on memory size. Decode-stage attention: moderate compute intensity, extremely high demand on memory bandwidth (repeated reads/writes to KV cache), extremely high demand on memory size, because KV cache grows linearly as batch size increases. Decode-stage FFN: moderate compute intensity, extremely high demand on memory bandwidth (repeated reads of model weights), moderate-to-high demand on memory size (model weights). The characteristics of SRAM-route chips are equally clear: They push memory bandwidth to the absolute extreme, but everything else is a severe weakness. They fundamentally trade off compute intensity and the inability to scale memory size for the ultimate in memory bandwidth speed. --------------------- Now let's look at SRAM's suitability across the three stages of AI inference: Prefill: SRAM can't achieve high compute intensity because SRAM takes up too much die area, leaving limited space for compute units. So prefill is a weak point. Decode-stage attention: SRAM can meet the high memory bandwidth requirement, but its memory size is too small to handle the batch size demands. So SRAM only satisfies half the requirements for attention. Decode-stage FFN: SRAM can meet the high memory bandwidth requirement, and the memory size requirement is moderate. Through optimized interconnect communication, SRAM chips can barely solve the memory size problem — the cost is steep, but the ROI can still pencil out in certain scenarios. ---------- So the applicable scope of SRAM-route accelerators in heterogeneous AI inference is crystal clear: Prefill — forget about it. Garbage performance, garbage economics. Decode-stage FFN — with enough effort and added cost, it's within reach. Decode-stage attention — KV cache demands on memory size are too extreme, and the cost of batch processing is prohibitively high. Letting Cerebras's $2.3 million-per-chip, 45-chip, $100-million luxury system serve as an exclusive ultra-VIP service — sure, that technically works. Imagine this: one or two users running agent flow on a coding task with 1–2M context length, and it takes the entire 44GB of SRAM on a single $2.3M Cerebras chip just for KV cache, otherwise the speed tanks. What kind of extravagant service is that? ------------- So the conclusion couldn't be more obvious: if Cerebras tries to do full-stack AI inference on its own (prefill + decode ATTN + decode FFN), the economics simply don't work. There's no future in it. Because the cost of Cerebras is staggering. Even with their gross margins squeezed razor-thin, the implied rental rate of each CS-3 system is still $41.96/hour — roughly ten times the rental of a B200. And that's for a single CS-3. You need to chain many of them together for LLM inference, so multiply that rental by a lot more. This is precisely why the economics of the SRAM route are so poor — Nvidia already made this point crystal clear at GTC (see chart). People hyping up SRAM as the future replacement for HBM? That's a pipe dream. With SRAM scaling already hitting a wall, SRAM density per generation of chips is already nearly impossible to improve. On the memory size dimension, HBM's exponential growth will only widen the gap with SRAM further. Even on the memory bandwidth dimension, HBM is growing exponentially too, narrowing the gap with SRAM. So Nvidia's solution is elegant and clean: hand off the decode-stage FFN to the SRAM route, keep everything else on traditional HBM GPUs, and push the entire Pareto frontier significantly toward the upper right. Rubin + LPX breaks through 1000 tokens/s at peak speed while still maintaining enough overall throughput to generate real commercial value (this is critically important). Remember — on Blackwell, if you wanted to hit 400–500 tokens/s at high speed, you could only process a tiny handful of concurrent requests. That's a massive waste of GPU resources. But now, even at 1000 tokens/s, you can still maintain a meaningful batch size (throughput), and it can finally generate commercial value. The chart shows that at 400 tokens/s, Rubin + LPX delivers a 35x improvement in throughput — classic token economics. At that token speed, it represents a 35x improvement in commercial value over Blackwell. --------------------- After GTC revealed this standard answer — and even earlier, after the Groq LPU acquisition — everyone has already started copying this homework on the heterogeneous inference front. Google's TPU tapped Marvell for the SRAM component. Amazon AWS's Trainium tapped Cerebras for the SRAM component. ByteDance's AI ASIC tapped Qualcomm for the SRAM component. We will absolutely see more announcements like these in the future. And this is the best path to economic sustainability for Cerebras: stop trying to muscle through full-stack AI inference, focus on what you're good at, partner with mainstream AI ASICs on inference, and fight to embed your SRAM chips into other companies' decode FFN pipelines. This is also why the key to Cerebras's long-term trajectory hinges on how deeply it can integrate with AWS Trainium's disaggregated inference. If it's just what's been reported so far — Trainium handles prefill and Cerebras handles decode as a simple split — the technical implementation difficulty is much lower, but the economics still don't pencil out. It can only be a strategic positioning play — enough for a slice of the market, but not enough to generate real scaled competitive advantage. To follow the Nvidia playbook, they need deep integration of both companies' strengths — and that will take real time and technical effort, with non-trivial difficulty, but the payoff would be worth it. Solution one: Trainium handles prefill and decode attention, Cerebras handles decode FFN. Solution two: Cerebras runs the draft model, Trainium runs verification. Either solution delivers dramatically more market competitiveness. -------------------- Does this kind of partnership with mainstream AI ASICs shrink the TAM for SRAM-route companies? No. This is the only long-term sustainable path for SRAM-route companies to grow their market. Heterogeneous AI inference is unequivocally the future. Finding your piece of the puzzle early in this growing blueprint is the only way to grow alongside the market. The moment an SRAM-route company embeds itself into any mainstream AI ASIC's heterogeneous inference pipeline, its valuation will skyrocket — because the TAM in terms of unit shipments isn't even in the same order of magnitude. Otherwise, heterogeneous AI inference will relentlessly erode the speed advantage of the SRAM route on the token speed dimension (not throughput). Full-stack inference on the SRAM route degenerating into an expensive toy is an inevitable outcome.