Proud to partner with @baseten and @baselabs to build safety infrastructure for open-source models—which are essential to lots of safety research, including our own. Safety must be built into open models and provided by those who serve them, and we’re excited to help enable that!

Safety is not just for closed models. The closed frontier labs are a canary in the coal mine for what is coming at scale. They give us a glimpse into the future and a window to harden our systems and prepare for abundant intelligence, with all the risks that come along with it. The OpenAI agent swarm attack on Hugging Face is the kind of failure we need to prepare for as open-source models catch up. The providers serving those models (such as Baseten) have a big role in establishing what safety and monitoring standards look like. We’re proud to be taking the lead on this at @baselabs with our collaborators @huggingface and @GoodfireAI. We’re developing safety research in the open and building it directly into Baseten’s inference infrastructure, with the aim of making it available to all our customers. This includes training models to follow explicit policies, detecting failures at runtime, and connecting those signals to controls that can intervene. We invite others in the open-source ecosystem to join us in building the tools and standards we’ll all need.

This is the last time you’ll be able to spot this. It’s like AI generated images start of this year. You’ll stop spotting it, because people will get so good at it. That is, because every single product, experience, game, thing will become so well engineered at capturing real world inputs to seed RL envs / midtraining data. There will be whole design philosophies on doing this without the user realising. Dark user patterns will abound. The world will become a factory for RL envs

very kind from AT but most importantly I agree that someone should figure out the contours of this conflict: "on one hand: <> “anything that can be learned through RL can be distilled very easily”. this is the justification as to why open source models (chinese) can catch up to closed source models (american). fair... 5 minutes later <> “Opus 5 could not distill / generalize well from Fable [despite anthropic having the live deployment, access to logits, and obviously a good prompt distribution]”." we are working on a version of this at @baselabs

I think a downstream consequence of increased commitment to safety and alignment is that RL envs companies get screwed, or at least held to a much higher standard and thus their unit economics changes. The evidence is mixed on this early but it seems likely that impossible to poor quality envs lead to misalignment and poor behaviour in the models (as they try to do anything to get reward). So why would the labs risk this quality control with anything but big in-house efforts moving forward

A flavour of our research interests on the Dwarkesh Podcast this week, discussed by our very own @oneill_c. Just the start of a longer conversation about long-horizon RL and frontier open-source training here at Base Labs.

For better or for worse, chamath + All In pod is somewhere that so many people, particularly non technical, get exposure to information and opinions on AI. Right now is a perfect opportunity to get more informed and thoughtful voices on there

really cool, didn't know @oneill_c was Australian until i heard the accent I hadn't really considered the prompt distribution and the usefulness of realistic data in that regard and how the distillation goes through these channels, its a good explanation of why Chinese models are so good compared to like sonnet or opus though.

I think one of the more interesting things we debated here is whether RSI is a cumulative task. Attention plus MoE plus GRPO etc seems to me like a line in the sand that you can just add to the stack once you discover it. You don't need to take five steps back to take 10 steps forward. But a lot of the work in the world isn't this clean and it certainly isn't this stationary eg legal work. This leads to some perhaps unintuitive predictions such as why RSI might land before continual learning (and why it's going to be hard to get off the current paradigm even if it's wrong) Thanks for having me @dwarkesh_sp!

In the post-agi world (perhaps even in the post-rsi world) I think there will still be cognitive labour. I actually think the most important people will be philosophers, economists, people who deeply understand opportunity cost, scientists deep in a domain who have also been exposed to others, all with the moral seriousness to say which problems are worth solving. What will all these people be doing? Essentially deciding where to point the big supercomputer (the eye of stargate, as @willdepue would call it) But instead of this being a sad reality (we're not solving problems anymore) instead we might view this as a beautiful new reality; we're solving the meta-problem of what problems to solve (with finite compute, we're going to need some process for determining where to point the superintelligence, and this process will likely become the most important one in the world). This is a large-scale problem (do we cure this cancer or that cancer? Alzheimers or climate change) and a small scale problem (how much compute do we give everyone? How unevenly does it get distributed? Do we believe in ubiquitous, UBI-style compute grants? Is this necessary, or will the value of the labour above still be enough to sustain something vaguely capitalistic)? So if you're a switched on and ambitious young kid trying to think about what to study, instead of the answer 5 years ago of computer science i'd now say become a renaissance thinker. Study economics and science and philosophy and literature. Learn how to think, and how to say what you think to other people. This to me is the opposite of repressed and vacuous humanity; it's the version of humanity that is doing the most useful form of work. The final form of work is wishing well

Some harness + training cooptimization we did with the cracked team @baseten. Tons of banger charts but my favorite is data room read coverage which moves from < 1% (base agent) -> 64% (RLM harness) -> 92% (RLM harness + RL) For basic behavioral strategies like "read the whole data room" it still surprises me every day (1) how strong models still don't just know them but, more excitingly, (2) how quickly any model learns them through RL in the right environment!

Fable is probably ~2-2.5T parameters, not 10T. Kimi K3 is 2.8T params, trained on maybe 20–30k Blackwell-equivalents. It lands within spitting distance of Fable 5 in terms of capabilities (5, not 5.1). Anthropic has far more compute than Moonshot, better rl environments, better architecture and better optimizers and all of that adds to capability per parameter. So if Fable is only slightly ahead of K3 with this in mind, it's almost certainly a smaller model. GPT-5.5 and 5.6 are smaller still (I'll say more on that later)

this was a nice starting point to analyse a now age old question, great work Dwarkesh and Jerry! Am excited to see people extend this work, many suggestions in the post itself and the comments which people should work on

Today we're launching Base Labs, a research lab by Baseten. Our mandate is to make open-source as useful as possible, and our one rule is that we publish without exception, including what fails. Up until pretty recently I thought the way to get the world onto open models was to train them for one company at a time. @mudithj, @maxkirkby and I cofounded @parsedlabs on that bet, @baseten acquired us, and we spent the last year running their training team doing it for customers one by one. Every one of those engagements taught us something new about how models learn, forget, specialise and get cheaper, and almost none of it got spoken about. Sadly, in general that's the field's default in that the people who know the most about training have the least freedom to say it. To be clear I don't think the closed labs are the villains here. They get to new capabilities first, which buys the rest of us time to harden the world before that stuff is everywhere, and they're the ones paying to find out what's actually possible. But I'm fairly convinced the only real advantage they have is data and scale, and their incentives point squarely at the frontier. You can't do slow, public science on how these things learn when your job is the best model by end of quarter. Someone without that pressure has to, and there are very few of those someones around. Hence Base Labs. We have the broad remit of making open-source as useful as possible and our one rule is that we publish without exception, including what fails. The first problem is continual learning, which I have come to think is several problems wearing one name. We are also working on the open RL environments and data that open models need and currently can't get, because we have to aggregate data with the same ferocity everyone's been talking about aggregating compute. Plus a bunch of other stuff I'm genuinely excited about, eg a safety stack people can run on top of open deployments, and performance research so these things are cheap for everyone to serve. I still think open and closed coexist, and that's the good world. It's just that coexistence isn't free, someone has to actually do the work, and this is basically what keeps me up at night. We're hiring researchers, engineers and fellows. Come help distribute the mandate of heaven!