Register and share your invite link to earn from video plays and referrals.

Interconnects AI
@interconnectsai
What you need to know about the latest models and AI research trends, from @natolambert
1 Following    9.5K Followers
How well can we measure open model inference share? Our first open model economy research note for paid subscribers.
New talk on the open model economy. I wanted to break down the case of where open model builders should steer the economics of the industry, if we want to avoid being swallowed by OpenAI and Anthropic in due time. This rests fully on being able to drive the cost of inference down dramatically on specialized, repetitive tasks, rather than trying to compete on cost with the frontier labs in any general agent. A core piece of it is how the open model economy is very fragmented, leaving an interesting puzzle we need to solve to get revenue from applications to feed back into the next model cycle. This is a very near-term problem as closed labs scale rapidly. This talk investigates: Are people using open models? How close are open models to the frontier? Mapping the open model economy How open models win This is a practice recording of my talk "How Open Models Win" for Fireworks Forge 2026 in San Francisco (3 November 2026), which will have improvements over this one.
Show more
Was I wrong about open models & cyber risks?
Artificial Analysis v5 and the benchmarker's dilemma A sort of review of the benchmarking industry and what's missing.
My first open model lecture, covering all the many pieces that build the ecosystem, and the open questions/debates determining where it goes next. How does distillation fit in? How do we manage risks? How do we see access to models changing in the future? I'm excited to revisit these questions about every 6 months, to give a state of the union on the field. Chapters 00:00 Introduction 01:36 What is an open model? 08:47 Why should we care about open models? 18:52 How competitive are the best open models today? 25:28 How are open models built? The role of distillation 34:52 Are people actually using open models? 41:47 What comes next with open models? 47:32 Conclusions What questions did I not answer that you have?
Show more
0
62
1.1K
129
Forward to community
Interconnects Podcast #20#: We got together to debate hard questions and messy realities after a crazy, somewhat intimidating, week in the AI world: - Why did Kimi K3.1 Pro have a mixed reaction with such incredible benchmarks? Will it be adopted? - Were we wrong about the risks of open-weight models and cybersecurity? - Why do cyber experts discuss recent events so differently than AI experts? - What comes next in the few months? What will change our minds? Chapters: 00:00 Kimi K3.1 Reaction 08:15 Kimi’s open-weight cyber principles 11:23 You can hack everything era 15:54 PHP2Shell, LiteLLM, and the new internet 24:00 Abliterated APIs, safeguards, and trusted access 31:41 Anthropic not hiring juniors employees 36:16 Florian’s Muse addiction 45:30 Wrap-up More of these soon!
Show more
Interconnects Podcast: Kimi K3.1 Pro, the new cyber reality, and Muse musings #20# YouTube: Transcript / Podcast:
Finally got to put together a complete deck on the state of open models. What they are, licenses, how competitive they are, signs people are using them, distillation's impact, and how we balance the risks and benefits of openness! Will record a video lecture of these soon, and hopefully re-do them a few times a year.
Show more
Meta shakes up the frontiers, open and closed US open weights are back!
Nathan Lambert of @trillium_labs says Chinese AI researchers' jaws drop to the floor when he tells them how much compute OpenAI has: "The Chinese labs are very technically excellent. They are more compute-starved." "Without a doubt, they have a comparable talent pool to draw on... To say that China is anything but equal on talent, you're gonna have a hard time because they just have so many people being trained at such a high level." "What they don't have is compute." "You talk to people there, and you're like, 'Oh, I heard an average OpenAI researcher has five thousand H100 equivalents that they can access at any time,' and like their jaw just falls to the floor. They're just like, 'I can't comprehend.'" @natolambert @lishali88
Show more
We've been monitoring a decline in HuggingFace downloads across the top open LLMs we track for interconnects. It looks like there was a slight change in counting of downloads, as we saw a ~30% download rate drop for many top models, but this is continuing over time. What else do people think it may be?
Show more
0
131
415
22
Forward to community
I expect rapid progress but not towards general superintelligence Musings on RSI after a week in SF.
The Cyber Risk Discourse is Broken Open-weights, ideology, and acknowledging trade-offs.
Very excited to see the next step in the Nathan Lambert extended universe. Lots more to do!
Today we're unveiling Trillium Labs @trillium_labs, a new non-profit to foster the open science of frontier AI. We're building open post-training recipes and will expand into open infra to study RSI, reward-hacking, multi-agent systems, and whatever comes next. We're built around the theory of change that you need more eyes to solve hard technical problems. We have faith in the scientific methods and communities that humanity has built, and worry that AI is becoming too closed to utilize them. Trilliums are wildflowers that bloom briefly in the spring, before the forest canopies fill out. Though they are small, they lay the foundation for the cycles of growth and nourishment through the rest of the year. At Trillium Labs, the recipes will be the slow nutrients for the seasons and the model releases will be the blooms. Building an institution dedicated to this is needed because, much as nature’s trilliums are slow to expand and grow, the open-ecosystem needs time and dedicated resources to catch up. I co-founded with with a long-time friend and collaborator Tom Zick (@thesezickbeats). We're hiring (full time + student collabs/interns), we're fundraising, and we're looking for compute. Please get in touch if you're interested in helping out. Offices based in the Bay Area and Cambridge MA, remote okay. I’m in the Bay Area until for The Curve and COLM to connect with people who are interested. We’re thankful to have initial support from Halcyon Futures and Schmidt Sciences with more funding en route to enable our ambitions of scaling. Our advisors @Thom_Wolf, @HannaHajishirzi, @gneubig and @ctnzr have been instrumental to building the ecosystem that exists today, and I’m stoked to get to keep working with them.
Show more
Hearing AI regulation is in the air, especially with a misguided tone of “open dangerous, closed safe” so it felt timely to re-up this one. There are ways to make the AI industry safer without kneecapping transparency, education, and competition.
Show more
The biggest disagreement JSD and I had in this podcast was on the impact of distillation. In our research for Interconnects, @xeophon and I agree that there's no hard evidence that distillation is a massive impact for the Chinese labs. At the same time, the gossip mill in SF has been doubling down on the Chinese labs getting massive gains from distilltion. The argument where distillation is a huge impact is something along the lines of distilled traces go into mid training and make RL work far more easily. My argument, that distillation is a 1-2 month pull ahead in capabilities closer to the frontier, is that Chinese labs already have sufficiently strong models, where this mid-training setup can be done on their own models, and most of the capabilities gains are from scaling RL environment training, which doesn't link cleanly to the distillation data pathway. We feel like the recent RL dashboard from @XiaomiMiMo supports this claim. A lot of it comes down to a gut call on if you can believe it that the Chinese labs are really great at building LLMs, maybe even better than focusing than OpenAI/Ant etc, as their current ambitions are a bit narrower (catching up), rather than transformative products/inventions
Show more
Debating RSI, the US-China Gap, and Jaggedness with JS Denain of Epoch AI Podcast #19# Substack: YouTube:
Check out our latest open model data -- which models are mentioned in every arXiv ML paper since ChatGPT.
Over the weekend I had Codex parse 500K arXiv AI/ML papers since ChatGPT to understand which open models are used for research. In 2024, ~30% of papers mentioned an American open model and only 10% a Chinese model. Today, ~40% of papers mention a Chinese (open) LLM, and only 25-30% an American one. Chinese models are the default for research. Chinese mentions are still growing while American open models are stagnating. When looking at this data it's important to remember that papers substantially lag model releases, as research takes a long time. Qwen's steady growth is reflective of this, but so is Llama's lasting power. Some more observations: 1. Qwen has been steadily growing, and today 1/3 of papers which mention any LLM mention qwen. OpenAI's closed models are the highest overall, at ~37%. 2. Llama peaked around April of 2025 at 30% of papers which mention any LLM (including ChatGPT etc). Llama 4 was released at about the same time, and Llama has been declining since. 3. Gemini and Claude are less common than the leading open models, mentioned in 10-15% of papers puts them behind all of Qwen, Llama, and DeepSeek. Open models should be and are the foundations of open research. The % of papers mentioning any LLM have been steadily climbing since 2023. | Year | January | April | July | October | | 2023 | 10.43% | 15.39% | 18.69% | 32.18% | | 2024 | 29.70% | 33.93% | 35.70% | 44.25% | | 2025 | 39.23% | 45.28% | 44.94% | 53.52% | | 2026 | 55.49% | 57.26% | 53.14% | TBD Now over 50% of AI papers, from 10% in 2023. Other notes: - Gemma and Mistral hover around 5-10%. - Our beloved fully-open Olmo models have been ~1% since the first release in Jan. 2024. - DeepSeek has a clear jump after R1 in Jan. 2025 - Data derived from the most popular ML arXiv categories: cs. AI, cs. CL, cs. CV, cs. LG, stat. ML Just like our downloads and derivative model data, this is updated daily on the Interconnects Open Model Dashboard.
Show more
Personal milestone: 1000 true fans of @interconnectsai ! Hitting a very long term goal feels great. I’m very happy to get to be an independent voice in AI. Cultivating a paid base helps me commit to that longer term, and scale Interconnects’ impact.
Show more
Teaching Everyone to Fish for Tokens Nvidia wants you building your own model, not buying from Anthropic/OpenAI.