Register and share your invite link to earn from video plays and referrals.

Philip Kiely
@philipkiely
Author of Inference Engineering | Building @baseten | Not an LLM (yet)
988 Following    16.6K Followers
Owned intelligence, by the token.
Baseten Loops is now GA, and serverless. Teams at @AmbienceAI, @datadoghq, and @LangChain use the Loops SDK for SFT and RL training to build durable intelligence. Serverless Loops includes: - Always-warm infra that eliminates cold starts - Pay-per-token pricing for training and sampling - One-click deployment from your trained checkpoint to production inference - Model-specific numerical validation, with a focus on reducing mismatch between training and sampling You bring the data, reward functions, and training logic. We handle the infrastructure. You always own the weights. Learn more:
Show more
New work on kernel-level optimization relevant for inference engineers.
For years, I said inference is everything. I was wrong. Agents need a suite of infrastructure: - Gateways and routers to manage model heterogeneity - Sandboxes to execute tool calls and run code - Post-training to turn traces into specialized intelligence So we built it.
Show more
Introducing the world’s best decision model, with free tokens for a week + open weights.
Today we release mammoth-1 and elephant-1! We post-trained mammoth-1, the #1# ranked best decision model in the world, and elephant-1, which is the best model under 15B (and beats Jev). We're open-sourcing all model weights and provide a technical report detailing all our training and data methods.
Show more
Scaling past a trillion parameters requires inference engineers to solve 3 issues: - Width: Routing between hundreds of sparse experts - Depth: Mixing global and local linear attention - Length: Processing 1M-token sequences I asked Ali and Bryce to draw it up. They delivered.
Show more
0
41
2.1K
211
Forward to community
An exceedingly clear introduction to harness selection and optimization. Highly recommended.
harnesses are everything. and they help smooth out the jagged edges of intelligence. here's a primer on them. a few months ago, I wrote a series documenting learnings and useful patterns around building and working with harnesses with my friend @zakkeener42 at @baseten. last week, we compiled them and turned it into a small book. it covers how to optimize existing harnesses, how to choose the best one for you, how they are co-evolving with models. as a bonus, we added some thoughts on what the future might look like. link in description:
Show more
Live in person is the highest-bandwidth interconnect.
Progress that moves the frontier is almost never made alone. We’re thrilled to announce All-Gather, our first-ever research conference that will bring together the brightest minds in research, founders, and engineers to join us for a day of debate and exchange. Come hear Noam Brown, Sarah Guo, Horace He, Bryan Catanzaro, Jerry Tworek, Sabri Eyuboglu, Tuhin Srivastava, and many more debate pre-training, post-training, infrastructure, model performance, and emerging fields. Join us on November 18th in San Francisco. Research has been part of our DNA from our early days of optimizing open-source models through our recent launch of Base Labs. We’ve grown with the open ecosystem, and we’re excited to keep building alongside it. Space is limited, but we’d love you to join us. Apply here:
Show more
Enterprise teams can now use open models like GLM-5.3 Flash and Kimi K3 natively in Codex and count spend against their OpenAI commit.
An open frontier is a safe frontier.
We believe openness to be an advantage for AI safety. Openness provides more visibility into the behavior of models and, most importantly, greater means of turning safety research into actionable and transparent controls than closed-source. This is why Baseten and Base Labs are building a stronger safety and security standard for open models, with the launch of our safety infrastructure. Base Labs will develop and publish methods for training and monitoring open models, and Baseten will integrate that work into its deployment infrastructure, live at runtime, and offer this work as a managed service. This will be a standard that is transparent and built into how our models are trained and deployed. We invite the open-source community to contribute, and are proud to partner with @huggingface and @GoodfireAI to bring this vision to fruition. Together, we are building an ecosystem of open models that are safe and accessible to all.
Show more
My agent tells me that inference engineering job openings have doubled in the past 6 months. I think that's an underestimate. Demand for inference has inflected, again. So too has demand for engineers who can operate AI models in production.
Show more
0
61
1.3K
69
Forward to community
It's the summer of Flash models. First GLM, now DeepSeek, these midsize models push the Pareto Frontier with remarkable intelligence at low prices. And, they outperform full-size last-gen models w/ new efficient architectures. Excited to see these recipes scale to 2T+ params.
Show more
🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6
Show more
The Blaxel team is full of wonderful people. Together, they have built a set of powerful infrastructure primitives that extend the inference layer to support agents from end to end. I'm excited to work with the entire Blaxel team in the coming months to ship incredible stuff.
Show more
Even with 1M-token context windows, a single M&A task can be 80X larger than fits in context. Great writeup about addressing this and other model limitations in a very sophisticated applied setting.
Show more
Just finished talking GLM-5.3 with Declan from Artificial Analysis. We covered: - Frontier price-performance offered by GLM-5.3 - Fighting saturation with the new AA Intelligence Index - GLM-5.3 Flash architecture and native vision I'll be around to answer questions!
Show more
Tomorrow at 11 AM PT, I'll be live on air with Declan Jackson of Artificial Analysis to discuss GLM-5.3 and GLM-5.3 Flash. We'll talk about benchmarks, performance, cost, and everything else you need to know before you switch.
Show more
An open frontier requires an open stack and open exchange. The folks working on this are some of the smartest researchers I have ever met and I am excited to see how far they can push the frontier.
Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.
Show more
Imagine how cool it would be to watch NFL with this tech -- first person view as the quarterback or any of the 22 guys on the field.
Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.
Show more
Hanging out next Thursday 9/10 with my friends from NVIDIA Dynamo and SGLang. We'll have tech talks on inference optimization at every layer of the stack. Join us!
Teach your agents inference. The full text of Inference Engineering is now available to LLMs everywhere. No more parsing PDFs. 3,000 tokens | links to each chapter: 70,000 tokens | full text:
Show more
Faster kernel != faster model. Great breakdown of systems thinking around kernel optimization.