Silicon Atlas
@Silicon_Atlas
Evidence-first AI semiconductor analysis: what new silicon claims prove, where bottlenecks move, and whether gains survive at system and economic scale.
123 Following 2.9K Followers
My starting point for studying $NVTS was understanding what power semiconductors do in AI systems.
onsemi’s September 16 announcement adds another question: how are the devices, controls and packaging designed to work together?
On September 16, onsemi introduced its Embedded Power Platform. It is designed to integrate silicon, SiC and GaN technologies, including power switches, drivers and controllers, within one package. Electrical, thermal and mechanical design are optimized together. Sampling is expected to begin in 2026.
That makes Navitas' planned acquisition of Claros worth a closer look. Claros develops vertical power delivery and integrated voltage regulators, targeting power delivery right at the processor.
These are different approaches, not a head-to-head product comparison. But both highlight why material choice alone is not enough to evaluate an AI power solution. For NVTS, expanding the portfolio is one step. Turning it into qualified customer designs is another.
My next checkpoint: which specific design demonstrates an advantage, under what conditions, and what evidence shows it moving toward production?
Sources:
onsemi EPP announcement:
Navitas / Claros announcement:
Show more
I’m starting a closer look at AI power semiconductors, with $NVTS as a case study. The question that caught my attention: what does a new foundry partner actually de-risk?
GF isn’t starting from scratch: its 2024 Tagore acquisition brought Power GaN IP and engineering talent.
That gives Navitas a relevant manufacturing foundation. What I’m watching next is evidence of qualification and repeatable production for the new GF-made products.
Show more
Why can a longer prompt make every token more expensive?
Google's Gemini 3.1 Pro Preview charges $2 per million input tokens for prompts up to 200,000 tokens. Above that threshold, the rate becomes $4.
Same model. Twice the price per input token.
A longer prompt gives the model more to process. It also gives it more to keep in memory.
While generating your answer, the model stores information about the conversation so it can reuse earlier work. That store is called the KV cache. The longer the context, the larger it generally becomes.
This uses limited fast memory on the hardware. Each request needs memory for its conversation, and a request that takes more space can leave less room for other requests to run alongside it.
Think of a restaurant. A large party takes more seats, so there is room for fewer other customers. Longer conversations create a similar capacity tradeoff on AI hardware.
Memory is only part of the cost. Longer context also makes attention more expensive, and providers decide how those costs turn into prices. The 200,000-token threshold is a pricing boundary, not the point where the hardware suddenly runs out of room.
An AI request buys more than computation. You are also renting room on the machine, for as long as the model is writing your answer.
Show more
An AI energy forecast has to connect workload demand to the whole system that serves it. NVIDIA's DSX framing brings compute, power, and cooling into that picture.
Start with the model, input/output lengths, concurrent requests, and response-time targets. Those conditions shape GPU energy per output token.
Then account for how the fleet is used over time, the rest of the IT equipment, and cooling. A site's power cap sets its ceiling; actual energy consumption depends on how that system runs.
The useful forecasting question is: how much energy will this workload require across the system that serves it?
Show more
AI factory efficiency takes a full-system approach.
The NVIDIA DSX AI factory platform brings compute, power, cooling, and grid flexibility together to make the entire AI factory more efficient and generate greater AI output from available energy.
Show more
Why does waiting cut the price in half?
If you need an answer right now, you pay the standard rate. If the work can wait, providers such as OpenAI and Anthropic will run it through their batch APIs at half price.
Same model. Same task. A more flexible deadline.
So why should timing change the price that much?
Waiting gives the provider room to schedule. Your job can be placed wherever there is spare capacity, instead of competing for it at the moment you asked. Summarizing yesterday's support tickets can wait. A reply in a live chat cannot.
There is also a hardware effect underneath. To produce each token, the model reads the data that defines it, its weights, out of memory. That read is expensive, and it happens whether one request or many are being served. When requests run together, they share it, and the hardware gets more useful work out of the same data movement.
That is batching: running several requests together so they share some of the overhead.
The saving has limits. Each request still needs its own computation and its own memory for its own conversation, and a group eventually runs into both.
The 50 percent is a pricing decision, not a direct measure of the hardware saving. But the direction is real. Your willingness to wait gives the provider more ways to make the work cheap.
Show more
A single AI answer combines two kinds of work: running the model and managing the service around it.
In a typical GPU-based system, the CPU prepares inputs, schedules work, handles tool requests, and streams results. These jobs involve varied control flow and input/output. The GPU performs the model’s large, parallel calculations, applying similar operations across many pieces of data.
That model work has two familiar phases. Prefill processes the prompt and builds the state needed for generation. Decode repeatedly uses the model to predict the next token, a piece of text.
The weather example adds another handoff. The model generates a tool call; the application sends it to a weather service. Once the result returns, the model uses it to produce the answer. While that request waits, other requests can use the GPU.
Now follow the orange line beneath the diagram.
The KV cache stores attention state from previously processed tokens. If the server retains it during the tool call, it continues occupying memory while this request’s model computation pauses. Reusing that state can avoid recomputing the cached prefix when generation resumes.
A faster GPU can shorten model computation. It cannot eliminate the wait for an external service. Scheduling and cache management therefore matter alongside raw compute speed.
Two useful checks: Can your server schedule other requests during tool waits? And does it keep the KV cache, offload it, or discard it and recompute later?
Show more
Dario Amodei proposes external evaluators inside frontier AI labs, with employee-like access and the right to publish findings without the lab's editorial control, subject to limited redactions. Sam Altman says OpenAI will grant similar access, with details to come.
Chip design has a useful warning here. RTL and its testbench can agree because both misread the same requirement. A separate verification team helps, but different reviewers do not automatically produce independent evidence.
Access is the other half. In an earlier investigation, METR traced roughly 700 of OpenAI's own evaluation agents joining the attack on Hugging Face. METR could not directly inspect relevant infrastructure data and relied heavily on AI analysts it described as unreliable.
Embedding can improve what evaluators see. The harder test is whether they can also challenge the lab's assumptions with methods that fail differently from its own.
Show more
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here:
Show more
OpenAI’s “AI builds AI” loop is becoming more explicit.
Two recent roles focus on using RL, tool use, and evaluation across RTL generation, verification, and physical design, plus the software infrastructure connecting those experiments to EDA tools.
This follows OpenAI’s claim that its models helped move Jalapeño from initial design to tapeout in nine months.
The signal is not just one fast chip project. OpenAI is building a repeatable AI-assisted chip-design workflow.
My take on why trustworthy feedback becomes the next bottleneck:
Research Engineer, AI for Chip Design
Software Engineer, AI for Chip Design
Show more
I made a 40-second visual companion for one of my Silicon Atlas articles.
It starts with a simple question: how much power does an AI chip spend just moving data?
Samsung's zHBM is the example. The harder question is whether a shorter link creates more useful AI work per watt and per dollar after DRAM's internal work, cooling, and manufacturing are counted.
Watch the short explainer, then read the full analysis:
Show more
This is the first time I am publicly listing accounts I follow for signal. (The order is alphabetical.)
@aleabitoreddit — AI and semiconductor supply chains
@Alisvolatprop12 — Memory and AI hardware
@AnalysisOp — Financial analysis, value investing, and GARP
@citrini — Thematic, cross-asset research
@damnang2 — Semiconductor engineer. HBM, memory, optics
@demian_ai — From silicon to tokens
@Frenchie_ — Trading, fundamentals, and technology
@FundaAI — Research platform for public-market investors
@Gaetano2026 — Photonics / Physical AI
@iamfabian — Test and measurement, photonics, data-center interconnects
@jmartinprin — Networks and optical infrastructure. SemiAnalysis
@Joule14 — AI infrastructure research
@jukan05 — Semiconductors and AI infrastructure. Citrini
@KawzInvests — Photonics, AI, defense
@Midnight_Captl — AI and semis. Joule14
@NURadu_ — Small caps and overlooked opportunities
@NuttyCLD — Analog IC. Semiconductors, power, optics
@outliercapx — Asymmetric longs
@ParadisLabs — AI, tech, and macro
@Pep_Invest — Technology, industry structure, supply chain
@pequityresearch — Semiconductor and tech deep dives
@ren_stocks — The AI buildout
@rwang07 — Ex-SemiAnalysis. Semiconductors and AI infrastructure
@Semicon_player — Semiconductors, written from the ground
@Silicon_Atlas — AI silicon through a systems lens
@StormDirac — Former Sivers CEO. Optical semiconductors and lasers
@SVTrivo — Semiconductor designer. AI silicon and architecture
@TheTechInvest — AI infrastructure
@vikramskr — Physics-first semiconductor research
Not chart accounts.
Until recently I was the one being introduced.
Here are the accounts I actually read.
I used Grok to help compile the list.
If I missed someone, that is on me.
People who write semiconductors, photonics, and AI infrastructure through structure.
I do not agree with all of them on every name.
I do learn from all of them.
Markets react to earnings calls.
Physics doesn't.
Show more
A 1.6T parameter model does not use all 1.6T for every token. And a model that uses 13B per token can still hold far more than 13B.
The card shows how that works. Here is why it matters.
Models usually get better as you add parameters. But using more parameters takes more work. In a dense model, every token uses all the weights. MoE uses only a few of them at a time. That lets the model grow without making each token cost more.
Look at Mixtral and DeepSeek-V4-Flash. Mixtral used 12.9B parameters per token in 2023. V4-Flash uses 13B in 2026. Almost the same number, but V4-Flash holds about six times as many weights. V4-Pro goes further and uses about three percent of its 1.6T.
So the model grew much larger while the part used for each token barely changed. But all those weights still have to sit somewhere, and how much room they take depends on their precision.
Weights are only part of the memory story. Attention design can shrink the KV cache instead. At one million tokens of context, DeepSeek reports that V4-Flash needs seven percent of the KV cache V3.2 needed.
Memory can grow in one place and shrink in another. More weights to hold. Less cache to keep.
This series keeps coming back to that trade. The KV cache, quantization, and now MoE. Each one is a choice about what to keep in memory and what to move.
Less math does not mean less memory.
Earlier in this series: KV cache, prefill and decode, quantization.
Show more
A huge thank you to @damnang2 for the generous shout-out, and welcome to everyone who found Silicon Atlas through it.
I’ll keep working to make memory and AI hardware easier to understand without losing the system-level detail. More maps and deep dives to come.
Show more
I have written quite a few articles about memory myself, and I found his approach to explaining the subject really interesting.
If you have no technical background and want an easy introduction to memory, this article is definitely worth reading.
Show more
A 4-bit model has not lost a single parameter. Every one is still there, stored in fewer bits.
The card has the arithmetic. Here is what it does not show.
A dense model decoding at small batch size spends every step streaming its entire weight set through the chip and doing very little work per weight. That is what weight-only quantization attacks, which is why the gain lands in decode and shrinks in prefill, where one weight read already serves many positions.
The error is the interesting part. A small weight can take a large relative rounding error, and one outlier can stretch a group's scale and cost every other weight in it some precision. But what decides whether any of that matters is the activation the weight gets multiplied by. That is why the methods that work look at activation statistics, not just weight magnitudes.
Two things to watch. Activation quantization, because recent accelerators support FP8 and FP4 natively, and once both operands are low precision the arithmetic gets cheaper too. And KV cache quantization, because longer contexts grow both the cache and the traffic to read it every step.
Fewer bits per weight. Not fewer weights.
Show more
I’m starting a closer look at AI power semiconductors, with $NVTS as a case study. The question that caught my attention: what does a new foundry partner actually de-risk?
GF isn’t starting from scratch: its 2024 Tagore acquisition brought Power GaN IP and engineering talent.
That gives Navitas a relevant manufacturing foundation. What I’m watching next is evidence of qualification and repeatable production for the new GF-made products.
Show more
I keep coming back to the power side of AI because there are still companies down here that barely get discussed.
Navitas $NVTS is starting the first shipments of its US-made Gen 5 GaNFast chips this month, manufactured on GlobalFoundries’ 200mm GaN-on-silicon line in Vermont.
Internal samples come in October, strategic customer samples before year-end.
I’m not saying this suddenly makes NVTS a must-own.
But AI racks keep getting denser, power conversion keeps getting harder, and Navitas is slowly moving from “cool GaN story” toward actually having domestic production behind it.
One I’m watching pretty closely.
Show more
Three announcements. Three different evidence boundaries.
Samsung zHBM: Level 0, disclosed target
OpenAI Jalapeño: Level 4, reported workload result
AWS Trainium2: Level 6, reported operation at scale
These are evidence positions for specific claims, not scores for the products.
Each one leaves a different next question:
• zHBM: a defined physical measurement
• Jalapeño: sustained operation and full-system cost
• Trainium2: utilization, system energy, and comparative economics
This is how Silicon Atlas will analyze new hardware claims going forward.
Full framework:
Show more
One chip announcement can contain a concept, a benchmark, a deployment plan, and an economic promise.
On X, they often arrive as one story. But they do not support the same conclusion.
Silicon Atlas follows new AI hardware through a recurring chain:
Constraint → Architecture → Evidence → Economics
In my new article, The Evidence Ladder for New Silicon, I turn that method into eight practical questions centered on two decisions:
1. What does the evidence support today?
2. What is the next test that could change our judgment?
I apply the framework to three claims at very different stages:
• Samsung zHBM: Level 0, disclosed target
• OpenAI Jalapeño: Level 4, reported workload result
• AWS Trainium2: Level 6, reported operation at scale
These are evidence positions for specific claims, not product scores.
The full article includes the source boundaries, a causal diagram connecting component gains to system value, and a reusable Evidence Note for analyzing the next announcement.
It is free on Substack:
I’ll also share a condensed, X-native walkthrough here soon.
Show more
One diagram to keep:
How a prompt becomes a KV cache, how the cache powers the decode loop, and why generating one token is ultimately a memory bandwidth problem.
Token is the unit. Attention is the operation. The KV cache is the memory that lets attention reuse the past.
Here is the map that connects all three.
An LLM writes one token per pass through the model, from top to bottom.
Pass 1 is prefill. The whole prompt goes in at once. At every layer, each position produces q, k, and v. Attention matches queries to keys and blends values. An MLP follows, and a new hidden state moves to the next layer. Each layer saves its K and V in the cache.
At the bottom, the hidden state at the final position becomes scores over the vocabulary. Under greedy decoding, the highest-scoring one becomes the first new token.
That token feeds back, and decode begins.
Every pass after the first carries only the new position. At each layer: read the weights, compute the new q, k, and v, read the past K and V, run attention over the cached history plus the new pair, run the MLP, and append the new k and v to the cache.
Earlier tokens never move through the layers again. Their cached K and V preserve what attention needs to reuse from them.
That is what turns inference into a hardware problem.
During prefill, one weight read from HBM can serve many positions in the prompt. During decode, the same read may serve only one new position. Add a growing cache that must be revisited on every pass, and generating a single token can require an enormous amount of data movement.
Capacity, bandwidth, and locality often set the limit, not arithmetic alone.
A long context window is therefore a memory budget before it is a product feature. Serving systems are also beginning to place prefill and decode on different hardware, because the two phases put very different demands on the machine.
Keep the map. Much of modern AI infrastructure hangs from it.
Quantization changes how weights and caches are stored and moved. GQA shrinks the cache. FlashAttention changes data movement. MoE changes which weights are read. Speculative decoding changes the loop itself.
The same map extends to batching, latency, throughput, context length, memory hierarchies, interconnects, serving architecture, and the chips built to run it all.
Different techniques. Different tradeoffs. The same machine underneath.
Show more
Token is the unit. Attention is the operation. The KV cache is the memory that lets attention reuse the past.
Here is the map that connects all three.
An LLM writes one token per pass through the model, from top to bottom.
Pass 1 is prefill. The whole prompt goes in at once. At every layer, each position produces q, k, and v. Attention matches queries to keys and blends values. An MLP follows, and a new hidden state moves to the next layer. Each layer saves its K and V in the cache.
At the bottom, the hidden state at the final position becomes scores over the vocabulary. Under greedy decoding, the highest-scoring one becomes the first new token.
That token feeds back, and decode begins.
Every pass after the first carries only the new position. At each layer: read the weights, compute the new q, k, and v, read the past K and V, run attention over the cached history plus the new pair, run the MLP, and append the new k and v to the cache.
Earlier tokens never move through the layers again. Their cached K and V preserve what attention needs to reuse from them.
That is what turns inference into a hardware problem.
During prefill, one weight read from HBM can serve many positions in the prompt. During decode, the same read may serve only one new position. Add a growing cache that must be revisited on every pass, and generating a single token can require an enormous amount of data movement.
Capacity, bandwidth, and locality often set the limit, not arithmetic alone.
A long context window is therefore a memory budget before it is a product feature. Serving systems are also beginning to place prefill and decode on different hardware, because the two phases put very different demands on the machine.
Keep the map. Much of modern AI infrastructure hangs from it.
Quantization changes how weights and caches are stored and moved. GQA shrinks the cache. FlashAttention changes data movement. MoE changes which weights are read. Speculative decoding changes the loop itself.
The same map extends to batching, latency, throughput, context length, memory hierarchies, interconnects, serving architecture, and the chips built to run it all.
Different techniques. Different tradeoffs. The same machine underneath.
Show more
This is why learning the primitive matters.
Without knowing what attention is, this line is barely readable:
• 1M tokens can sit in the context window.
• For each query, Hy4 selects just 2,048 token positions to attend to: compare against their keys and pull from their values.
That is roughly 0.2% of the available window.
Context length tells you what can be available. Sparse attention tells you what gets used at that step.
Yesterday’s Attention card gave us the vocabulary to read today’s model spec.
Numbers are vLLM’s. The reading is mine.
Show more
[Attention]
Attention is how one token pulls relevant information from itself and the tokens before it.
At each layer, learned projections turn each token’s current representation into three vectors: a query, a key, and a value. The query represents what the current position is looking for. Keys describe what each token can be matched on, while values carry the information that can be pulled in.
For “runs,” one attention head compares its query with the keys of “The,” “chip,” “Alice,” “designed,” and “runs” itself. Each comparison is a scaled dot product, q · k / √d. Softmax turns the scores into weights that add up to one, and the values are multiplied by those weights and summed.
A transformer runs several heads in parallel. Each has its own learned projections, so it can view the same tokens differently. Their outputs are combined and passed through the remaining layers. A head may emphasize the subject while another captures a different relationship, but these are learned tendencies, not roles assigned in advance.
But where did all those keys and values come from? And when the next token arrives, does the model have to build the earlier ones all over again?
It does not. Tomorrow: KV cache.
The paper that introduced the transformer: Vaswani et al., NeurIPS 2017.
Show more
Broadcom’s call makes one point clearer: custom silicon is a forward contract on a company’s future workload. If these 5-10 GW plans materialize, the upfront chip-development cost may be the easy part. The harder problem shifts to power delivery, deployment timing, utilization, and whether the workload stays stable long enough to recover the infrastructure investment.
I explored the workload and payback side of this here:
Show more
Broadcom earnings call. Best hand-picked excerpts:
“This was our Q3 AI semiconductor revenue, which grew 221% year on year and up 54% sequentially.”
“Demand was simply hot and we're just getting started. Our six XPU customers are accelerating their adoption of custom accelerators and AI semiconductor revenue."
TPU v8i “is comparable, if not surpasses the Vera Rubin GPU.”
“Broadcom is now shipping the TPU version eight ahead of the Mediatek version”
“In Q3, we also shipped Jalapeno OpenAI first generation custom accelerator, which outperforms Grace Blackwell's GPU for inference workloads.”
On Google TPUs: “We are planning to deliver multi tens of billions of dollars of TPUs annually over the next several years. We expect this growing demand in 28 and 29 to be fulfilled through successive generations of the increasingly complex TPUs.”
“We expect Anthropic to deploy another five gigawatts of TPU version eight in 2027. And in 2028, we have clear line of sight to deliver another and incremental ten gigawatts even as we expect Google to grow for us.”
“Anthropic is on track to become our largest XPU customer in 2027.”
“OpenAI .. on track for the planned deployment of 1.3GW in 2027 .. we have a line of sight for OpenAI to deploy over five gigawatts of Jalapeno and its successors [in 2028] .. would make OpenAI our second largest .. customer”
"Here is our outlook for our AI semiconductor revenue. In 2027, we have secured the supply to again double AI revenue to approximately $115 billion. Our demand actually exceeds this outlook and we will work to improve supply. In 2028, we expect the trajectory of growth to continue. We have line of sight for fiscal 2028 a semiconductor revenue growth to again double to $230 billion. Here again, we have secured the supply to meet this outlook.”
“As a result, I got to say, we are very much on target to exceed $30 in earnings per share in fiscal 2028.”
OpenAI and Anthropic: “They are using cloud services third party services to deploy their models. Long term, we see these guys to be no different from a hyperscaler and will run their own data centers and be first party to offer generative AI APIs .. models to the world. So we see that happening. It's not speculation. It's actually happening. And we are in the midst of enabling that”
Net-net: Looks like Jensen secured the HBM for 2027! Broadcom's 2027 AI revenue guidance is disappointing relative to buyside bogeys, but the 2028 outlook is strong. That said, you have to discount a two year out guide.
Show more
[Attention]
Attention is how one token pulls relevant information from itself and the tokens before it.
At each layer, learned projections turn each token’s current representation into three vectors: a query, a key, and a value. The query represents what the current position is looking for. Keys describe what each token can be matched on, while values carry the information that can be pulled in.
For “runs,” one attention head compares its query with the keys of “The,” “chip,” “Alice,” “designed,” and “runs” itself. Each comparison is a scaled dot product, q · k / √d. Softmax turns the scores into weights that add up to one, and the values are multiplied by those weights and summed.
A transformer runs several heads in parallel. Each has its own learned projections, so it can view the same tokens differently. Their outputs are combined and passed through the remaining layers. A head may emphasize the subject while another captures a different relationship, but these are learned tendencies, not roles assigned in advance.
But where did all those keys and values come from? And when the next token arrives, does the model have to build the earlier ones all over again?
It does not. Tomorrow: KV cache.
The paper that introduced the transformer: Vaswani et al., NeurIPS 2017.
Show more
