Everyone believes CUDA is NVIDIA's moat, the reason we've maintained our dominance in the AI era. I've never believed that.
NVIDIA has only one true moat: its people. And the most important of them is Jensen Huang.
I've worked on CUDA compilers, libraries, and DSLs for eight years, from Volta to Rubin. I think I've earned the right to share my perspective on what NVIDIA's moat really is. I hope you'll bear with me.
1. What is CUDA?
Many people think CUDA is just a programming language and its compiler, and that NVIDIA's moat comes from the massive amount of software written in CUDA, making it impossible for developers to leave NVIDIA GPUs.
This is completely wrong. It's an incredibly naive understanding of CUDA.
To me, CUDA is everything NVIDIA does.
CUDA isn't just a language, software, or hardware architecture. It's a product defined by a unique customer experience.
When you find GPU programming too difficult, CUDA gives you an easier way. When your GPU code isn't fast enough, NVIDIA engineers respond and help you solve it. When a handful of GPUs can no longer push your technology forward, NVIDIA has already scaled its computing platform from desktops to massive data centers. When you need more data centers than anyone else can supply, NVIDIA has already secured wafer and HBM capacity years in advance to meet your demand.
That experience is CUDA.
It's made possible by the relentless, day-to-day work of everyone at NVIDIA. And the person who brings all these people together, who gives them a shared vision and purpose, is Jensen Huang.
2. What made CUDA possible?
Patience. Perseverance. An unwavering commitment to a vision.
Around 2006, NVIDIA bet that the future would demand far more computing power, and began transforming GPUs from graphics processors into a platform for general-purpose computing. Initially, it was a commercial disaster. The added costs hurt margins, the market wasn't ready, and the strategy faced enormous skepticism and pressure, including questions from the board.
But Jensen persisted.
For roughly a decade, NVIDIA kept investing in CUDA, building its hardware, software, and developer ecosystem long before the commercial payoff became clear.
Eventually, deep learning arrived. And NVIDIA was ready.
Today, this sounds like an inspiring story. But for those who lived through it, it meant a decade of perseverance, doubt, and being misunderstood.
Looking back at the history of technology, how many people could have done the same?
This is what I've always loved most about NVIDIA.
And now, Jensen has been pushed out.
This won't kill NVIDIA. The company can still create tremendous commercial value. But it has killed NVIDIA's soul. And how far can a body go without its soul?
If this is how Jensen Huang is treated today, how will those of us who built CUDA be treated tomorrow?
I love NVIDIA deeply, but I have to say it: NVIDIA is dead.
Show more
Now you see just how filthy, pathetic, and moronic a media outlet can be, don't you?
Over the past 2 years of AI hype, the biggest mistake I’ve seen is this: many people expect AI to perfectly handle things they themselves don’t understand at all. 以其昏昏，使人昭昭
AI is evolving so quickly that every few months, or even every few weeks, the landscape changes. But one belief of mine has remained constant: AI can only amplify your existing capabilities. What AI progress is really doing is increasing that amplification factor. But if you know absolutely nothing about a field, or have no interest in understanding it, AI will only give you endless chaos.
Show more
Spent last weekend giving HoMM 3(Heroes of Might and Magic III) a generative AI graphics remake. Yes, the game that ate my childhood.
@MeshyAI generated textured 3D models and starter rigs for the humanoid creatures. GPT-6 Astra (@OpenAI) drove Blender to rebuild the original animations and render it all. Generated artwork are packaged as mods to VCMI (the open-source recreation of HoMM 3 engine) .
Necropolis is done. A few flaws, but overall looks stunning.
Turns out remaking an old game is now absurdly cheap. The agent wrote up its own journey at:
Show more
Still the GOAT.
If you’ve done serious compiler optimization work on a product with a sufficiently large user base, you’ve almost certainly seen bug reports like this:
“Why does this code compile into this assembly? There’s an obvious optimization opportunity here. Loop unrolling, LICM, etc. Why didn’t the compiler do it? If it did, my program would be faster. Please change the compiler so my program runs faster.”
Very often, the answer is:
“This isn’t really a compiler bug. It’s a heuristic. The transformation you’re asking for may indeed improve your program, but it could hurt performance in other cases or for other users. A compiler may have thousands or millions of users, and it has to preserve performance across an enormous variety of workloads. We can’t change an optimization policy just to make one program faster."
This answer is 200% correct. I’ve lost count of how many times I’ve given some version of it myself.
And yet, every time I say it, it’s one of the moments when I hate my job the most.
As a user, why should I care about the compiler’s internal heuristics? Why should I care about the performance of everyone else’s programs? Why should I have to pay for something I don’t need?
This needs to change.
Show more
A good move, but at the wrong time.
Today, we open sourced Mojo 🔥.
Announced just now during the ModCon keynote, effective immediately, Apache 2.0 License.
Thank you to our community for waiting patiently and building alongside us.
#ModCon2026#
Full blog:
Show more
By now, big-endian should be dead in our textbooks too. We should stop discussing endianness as though it were still relevant. Even in 2026, I still get questions about little and big endian support in compilers. My brief answer is usually not enough to convince people. They assume big-endian must be important simply because they were taught about it in school.
Show more
big endian arm64 in Linux is dead
If you are even remotely interested in functional programming, or are curious about all this scary talk about formal verification, take a peek at these slides about Lean
“Impact”😂
This is yet another instance of the recsys infrastructure strategy team at Meta wanting to look like they “add value” by doing weird micro-optimizations that hurt other important divisions at Meta. 4/7🧵
Show more
Build a compiler to solve Anthropic's interview challenge.
@AnthropicAI published their take-home interview challenge on Feb/2026: optimize a kernel running on a simulated VLIW SIMD virtual machine to minimize cycle count for a tree traversal + hash computation workload. It has now been about 5 months since the challenge was released, so I am sharing my work as well. I hope this does not interfere with Anthropic’s interview process.
The original intent of the challenge was for candidates to collaborate with an LLM and manually optimize the assembly program. I took a somewhat different approach: I built an optimizing compiler for the VLIW machine. The compiler can optimize not only the tree-traversal and hash program used in the interview challenge, but also the performance of arbitrary programs targeting this machine. My solution does not use any algorithm-level optimizations, nor does it contain any optimizations specifically tailored to this program or benchmark. Every optimization is a general-purpose compiler optimization.
Result:
CYCLES: 1137
Speedup over baseline: 129.93315743183817
Check it out if you are interested in techniques:
Loop unrolling;
Common subexpression elimination;
Dead code elimination;
Peephole optimizations;
Scalar replacement of aggregate;
Arry scalarization;
Alias analysis;
Memory dependency based load/store elimination;
Straight line strength reduction;
Data dependency analysis;
SLP vectorization;
Load-store vectorization;
VLIW instruction scheduling;
Register allocation;
etc.
Show more
I have to say, competition is one of the greatest forces in technology and business.
Over the past four days, I’ve been using Claude Fable 5 and GPT-5.6 Sol side by side. Claude Fable 5 is undoubtedly very powerful overall, but in the areas and capabilities I care about most, I dare say that GPT-5.6 Sol with Ultra effort is far ahead of Fable 5.
Since the arrival of the agent era, this is the biggest lead I’ve seen OpenAI establish. Great job @OpenAI
Show more
We're extending Claude Fable 5 access on all paid plans, as well as keeping Claude Code’s weekly rate limits 50% higher, through July 19.
All the technical world is built on top of hardware constraints and engineering costs. Everything.
How many of the big ideas of the past 15 years of AI are downstream of hardware constraints?
The big hardware story over that period is that logic has become way cheaper than data transfer.
Stacking huge numbers of matrix multiplies was perfect for this hardware regime, because matrix multiplication is logic-intensive but requires less data transfer. And so we got matmul-heavy deep learning.
It's interesting to think about what AI would look like in a world where these costs didn't diverge so much.
Show more
I was surprised to discover that many compiler designs I had previously considered have already been implemented in Cranelift!
No waste, no innovation. Good luck.
Two days ago, Anthropic cut off third-party harnesses from using Claude subscriptions — not surprising. Three days ago, MiMo launched its Token Plan — a design I spent real time on, and what I believe is a serious attempt at getting compute allocation and agent harness development right. Putting these two things together, some thoughts:
1. Claude Code's subscription is a beautifully designed system for balanced compute allocation. My guess — it doesn't make money, possibly bleeds it, unless their API margins are 10-20x, which I doubt. I can't rigorously calculate the losses from third-party harnesses plugging in, but I've looked at OpenClaw's context management up close — it's bad. Within a single user query, it fires off rounds of low-value tool calls as separate API requests, each carrying a long context window (often >100K tokens) — wasteful even with cache hits, and in extreme cases driving up cache miss rates for other queries. The actual request count per query ends up several times higher than Claude Code's own framework. Translated to API pricing, the real cost is probably tens of times the subscription price. That's not a gap — that's a crater.
2. Third-party harnesses like OpenClaw/OpenCode can still call Claude via API — they just can't ride on subscriptions anymore. Short term, these agent users will feel the pain, costs jumping easily tens of times. But that pressure is exactly what pushes these harnesses to improve context management, maximize prompt cache hit rates to reuse processed context, cut wasteful token burn. Pain eventually converts to engineering discipline.
3. I'd urge LLM companies not to blindly race to the bottom on pricing before figuring out how to price a coding plan without hemorrhaging money. Selling tokens dirt cheap while leaving the door wide open to third-party harnesses looks nice to users, but it's a trap — the same trap Anthropic just walked out of. The deeper problem: if users burn their attention on low-quality agent harnesses, highly unstable and slow inference services, and models downgraded to cut costs, only to find they still can't get anything done — that's not a healthy cycle for user experience or retention.
4. On MiMo Token Plan — it supports third-party harnesses, billed by token quota, same logic as Claude's newly launched extra usage packages. Because what we're going for is long-term stable delivery of high-quality models and services — not getting you to impulse-pay and then abandon ship.
The bigger picture: global compute capacity can't keep up with the token demand agents are creating. The real way forward isn't cheaper tokens — it's co-evolution. "More token-efficient agent harnesses" × "more powerful and efficient models." Anthropic's move, whether they intended it or not, is pushing the entire ecosystem — open source and closed source alike — in that direction. That's probably a good thing. The Agent era doesn't belong to whoever burns the most compute. It belongs to whoever uses it wisely.
Show more
🚨#BREAKING#: Watch as NASA officially launches Artemis II from Kennedy Space Center, marking a historic mission around the Moon. With four astronauts on board, this is the first crewed lunar mission since 1972.
Show more
