It has been easy to back the open source narrative recently with so much exciting progress, but the cyclical swings in energy we have seen seem inevitable. Inspired by the great scientists (including LeBron) pictured in video, @baselabs is a very direct effort to pursue a long-term research agenda that aims to support the WHOLE ecosystem (that we see being a mix between closed and open source models) for the long run. It is backed by @baseten ripping (which has a little more $$ than academia), but has a clear mandate to publish without exception and encourage skepticism. Our manifesto mentions 'Science done behind closed doors is worse science. It cuts off the one mechanism that has ever made knowledge reliable: correction from the outside'. Here's to democratising intelligence!

Incredibly excited to finally announce Base Labs, a new industry lab in Baseten that is fully open source and publishing, with the goal of advancing and democratizing engineered intelligence. Grateful to be leading this with @maxkirkby @oneill_c and @mudithj, and all of the talented researchers already at Base Labs. Engineered intelligence should be built in the open. Much more to come soon.

Today we're launching Base Labs, a research lab by Baseten. Our mandate is to make open-source as useful as possible, and our one rule is that we publish without exception, including what fails. Up until pretty recently I thought the way to get the world onto open models was to train them for one company at a time. @mudithj, @maxkirkby and I cofounded @parsedlabs on that bet, @baseten acquired us, and we spent the last year running their training team doing it for customers one by one. Every one of those engagements taught us something new about how models learn, forget, specialise and get cheaper, and almost none of it got spoken about. Sadly, in general that's the field's default in that the people who know the most about training have the least freedom to say it. To be clear I don't think the closed labs are the villains here. They get to new capabilities first, which buys the rest of us time to harden the world before that stuff is everywhere, and they're the ones paying to find out what's actually possible. But I'm fairly convinced the only real advantage they have is data and scale, and their incentives point squarely at the frontier. You can't do slow, public science on how these things learn when your job is the best model by end of quarter. Someone without that pressure has to, and there are very few of those someones around. Hence Base Labs. We have the broad remit of making open-source as useful as possible and our one rule is that we publish without exception, including what fails. The first problem is continual learning, which I have come to think is several problems wearing one name. We are also working on the open RL environments and data that open models need and currently can't get, because we have to aggregate data with the same ferocity everyone's been talking about aggregating compute. Plus a bunch of other stuff I'm genuinely excited about, eg a safety stack people can run on top of open deployments, and performance research so these things are cheap for everyone to serve. I still think open and closed coexist, and that's the good world. It's just that coexistence isn't free, someone has to actually do the work, and this is basically what keeps me up at night. We're hiring researchers, engineers and fellows. Come help distribute the mandate of heaven!

Today we launch @baselabs. It is time to take a very very big swing. Over the past few years, I’ve been astonished that seemingly anything the AI community has pursued, in the big or small labs, has worked. I think all scientists are when their experiments actually work and their theories are validated. But like come on? Backprop through a trillion parameters (yes I know only the active ones), add some correction to a memory block, let a sigmoid function decide what to forget? How does this all actually work? And yet “we can only see a short distance ahead, but we can see plenty there that needs to be done” (Turing, 1950). Base Labs is built on the premise that we all deserve to have a say in advancing the science of intelligence. That yes, we have made many big strides, but that the fragments of our success in training and inference have been scattered and only pulled together through the sheer willpower of a few incredible researchers and engineers. And, at the same time there are many fragments we cannot see, and a science that is being built and organised away from our eyes. Today we announce our research mission to advance and democratize open-source intelligence. Base Labs is our bet that there is much work to be done to train and serve SOTA models, and that this work should be organised (like a science) and for everyone to access and contribute to. We will work across model specialization, learning, memory, reasoning, and serving, and in doing so publish all our results (positive or negative), experiments and data for everyone to learn from and work on. Join us!

If you want to RL (or sft/opd) glm-5.3-flash, DM me to get access to it on @baseten loops. We're very excited about this model as a great base for further RL for your task. Being smaller than kimi k3, dsv4 pro, glm 5.2 etc means you can really get a lot of RL out of it because inference is much cheaper. And the model has been redesigned around inference efficiency at long context lengths

for my final post as @waterloo_intern, i'd like to ‘outroduce’ myself. it takes exactly 4 years and 8 months to make a waterloo intern. today is the last day of mine. the full life cycle, in four stages, is as follows: 1- interview hazing 2- cali or bust 3- inflection point 4- convergence stage 1: interview hazing a cohort of high school grads are ingested by waterloo. they are put through 4 months of academic bootcamp; the pre-requisites of the academic program they’re in. waterloo filters out the students who could not pass. they are put through another 4 month bootcamp, while simultaneously FORCED to find an internship for the following 4 months. this leads to students with ZERO experience applying to every job that is semi-adjacent to their field of study (a strategy, known in the literature, as spray-and-pray). it immediately backfires with a sea of interviews leading in a 1:1 fashion directly to infinite rejections. eventually, the student finds something…anything. this stage lays the foundation. rejection becomes noise so repetitively played it no longer registers. failure in an interview doesn’t matter. failure itself doesn't matter. for me, this unfolded as internships with blackberry and ford, summer of 2023 and winter of 2024, respectively. stage 2: cali or bust the next step by the program is to inject absolute dissatisfaction into its students (now in their second/third year). a CACOPHONY of immense fomo materialized as the university’s official unofficial motto: “cali or bust”. you must find your next internship in san francisco. your peers are doing it. all your smart friends have done it. you must do it too. failure to cali means your experience is BUST. you don't need to have a plan. you don't need to have anything. just go to san francisco. find an internship in sf. immediately. right now. for me, this was a cold email to a director at tesla and an internship with them in cali, fall of 2024. stage 3: inflection the reason behind step 2 appears here. somewhere, along the way, during your time in sf, you find your purpose. it could be as a direct result of your internship, but, more often than not, it is because of the people that you are surrounded by in cali. this is the inflection. for me, while at tesla, i was mentored by @pavanjayasinha , at the time interning at @Modular. he got me interested in gpu kernels & referred me there for my following internship. it was here that i had my second inflection point (due to the mentorship of fabio & hengjie, and the supervision of absolute field giants like @clattner_llvm), and decided to work beyond kernels and across the entire inference, serving, and model performance stack. stage 4: convergence having discovered the field you want to work in, you do your (usually last) internship in one of the companies within said field. here, you are at a disadvantage, being both a new-comer needing to onboard and inexperienced. but... you are also able to work on interesting projects / side quests that most full-timers are all but too busy to dive into. your time is also protected from a majority of the meetings others have to attend (in part because you are not important enough to have your presence expected), allowing you to make faster progress on assigned projects, and jump from one project (or team) to another. your title (intern) allows you to ask stupid questions of those around you, and so you can brain rake across the stack with no regard to who sits where in the company’s hierarchy... usually leading your questions with something to the effect of ‘can you help a new little intern onboard to X’, to which most people respond quite positively. in my case, this last stage took place over the past 8 months with @baseten, and i genuinely cannot imagine it being any better. i rejoin baseten full-time, but i wish i could have stayed an intern forever...

I spent 48 hours with the Kimi K3 modeling code. It took: - 650 mg of caffeine (mandatory) - 40 cans of LaCroix (optional... world record (?)) - 8 papers - 6 months off my lifespan Finally grokked the entire lineage of Kimi K3 and how we got here... every single step, since 2019 GPT-2

When @mudithj and I met @gabepereyra, we were expecting just another vanilla intro call and instead had the best yarn about research, the state of LLMs, and where intelligence is actually heading. It's rare to meet a founder this deep in the weeds who's also building for one of the most important verticals in this new age of intelligence So it was awesome to sit down with Gabe for an extended discussion on what it take to build agents that can reliably complete work over hours, days, or even longer? We talked about why agents today struggle with search and long context windows and how techniques like KV-cache compaction, synthetic data, and continual learning could help. 0:00 Introduction 0:36 Getting legal agents to review the whole data room 2:08 Data rooms larger than any context window 5:28 How far open-source models can go 7:58 Where specialist models fit in legal AI 10:59 Training legal models when client data is off-limits 13:06 Teaching a model how a law firm works 13:59 What belongs in context vs. model weights 15:36 From firm-wide AI to a model for every lawyer 18:37 What training adds beyond retrieving the right cases 20:26 Why context windows have plateaued 24:01 How models could learn continuously on the job 26:12 Can AI recursively improve AI research? 27:07 Research agents can run experiments but not choose them 30:00 Why open-ended research is hard to train 33:47 Why deployment, not intelligence, is the bottleneck 35:08 The cost of frontier intelligence 36:59 Different neolabs, different paths to intelligence 39:26 Using open datasets to compare research methods 41:13 Conclusion