Neil Movva (@neilmovva) started his career at Nvidia, working on GPUs and kernels, and has an unusually deep understanding of inference, from software to chips to power. We spend a lot of time on each of those layers, how they connect, and where the important tradeoffs are. What makes this conversation special is how detailed it is (like a 401-level class), yet Neil makes it remarkably clear and easy to follow. Today he runs Sail Research, a company building infrastructure for agents to make tokens as cheap as possible. We discuss: - Latency versus throughput - Why there are no bad chips, only bad pricing - The end of kernel engineering - Buying chips and power no one else wants - New chip architectures - Nvidia lore + his contrarian view of the company - Open source and the frontier labs I learned a ton. Enjoy! TIMESTAMPS 0:00 Intro 0:38 Building a “Token Factory” 4:21 The Future of Background Agents 13:09 Nvidia and the GPU Stack 23:27 Chips, Memory, and Transformers 36:14 The Future of AI Training Data 44:32 Chip Scarcity and Compute Arbitrage 52:44 Reinventing the AI Data Center 59:01 Power and the “Scavenger Strategy” 1:10:10 Open vs. Closed AI

Neil Kinnock helped make Britain’s main centre-left party electable again, by purging hard-left extremists who had infiltrated it. His story is one the Democratic Party could usefully study

Neil Chilson on why the US can't ban open source: "I think it's pretty unlikely. It runs smack into some very clear First Amendment issues, some right-to-compute issues. There are constitutional reasons it would be very hard to do such a thing in the US, especially if you're trying to target open source models developed here." "Practically it's always been hard to shut down software. These weights are not giant files. They're transferable quite easily, and running them you still need a lot of compute, so maybe that'd be the bottleneck regulators would try to look at." "But moving the models around themselves, that's very easy to do, and it would be very hard for export-import controls to stop the flow of something like open weight models." "There is a big difference between using a Chinese model, just using the weights here on US infrastructure, and using a Chinese model on Chinese infrastructure. In one case you control the data, you're not sending it overseas. In the other case, if you subscribe to something like DeepSeek and you're using the app and it's based in China, you're sending your data there. There are some security risks to that." "There are other concerns from using models you don't fully understand, but most of those, because of the open-weight nature, are relatively easy to resolve. There is a huge appetite among startups, entrepreneurs, and researchers for open-weight models to do research on. Shutting down the open source AI ecosystem is not a way to continue US AI dominance." @neil_chilson @abundanceinst