Interesting approach. Models can't output freeform text but can choose from a set of predefined values. Could be useful for data labeling and tasks with a fixed list of possible actions. Unclear how reasoning would work though, but it's super cheap (output tokens are free!)

what's a good model tiering system? i'm sick of telling my agent orchestrator things like: "for Claude, use model X, for OpenAI, use model Y, etc." i want to be able to tell my orchestrator: "use models tier ..." for this kind of task

Congrats to @AlecRad @Luke_Metz and @soumithchintala! It's really cool to see work produced by a group of 20-somethings without a single PhD between them winning this award. I hope to see these three collaborate again one day :)

Excited to release PostTrainBench v1.0! This benchmark evaluates the ability of frontier AI agents to post-train language models in a simplified setting. We believe this is a first step toward tracking progress in recursive self-improvement 🧵:

Super impressed by the projects at the Agentic Hackathon last weekend! Many teams work on really hard/important problems: * Long running tasks: memory management, recovering from mid-task failures, and maintaining consistency across steps and sub-agents * Adaptive retrieval from multiple sources: databases, search indices, and websites * Agents that work with voice, video, and even 3D environments If you are in SF, come check out the finalist demos tomorrow! There will be talks by Douglas Eck, who is doing amazing work with Veo and Imagen and many other awesome folks. Thanks @MongoDB and @cerebral_valley for hosting and for letting me serve as a judge for these fantastic projects.

After years of following @lennysan's wonderful takes on product, I finally had the opportunity to chat with him about AI products! 1. Many AI product problems aren’t because of AI. It’s usually because of user experience, data quality, or organizational structure. A chatbot failed to get traction because their targeted users simply couldn’t type (because their hands were usually busy -- taking care of kids or driving), so showing pre-populated questions and adding a voice option significantly improved traction. Another team told me their lead scoring model was broken. It turns out that it’s because the marketing team wasn’t asking the right questions to get data. The biggest product improvements still come from understanding your users, preparing your data, and investing in your team! 2. Senior engineers see the most productivity improvement with AI coding because they have more experience with writing design docs and API specs, which help them write better instructions. However, they’re also more resistant to using AI for coding. Senior folks are often more opinionated and get frustrated easily when AI doesn’t do what they want. 3. Many teams spend a lot of time debating which tool to use, which can be counter-productive. When teams ask me which of the 2 tools to use, I usually ask 2 questions: “How much performance improvement will the optional tool give over the less optimal one?” --> If the improvement is small, then spend less time debating. “How hard is it to change from one tool to another once you’ve adopted it?” --> If the tool is new and not yet battle tested, I’d think twice about adopting something that I can’t get out later. 4. Many people know that the most effective way to learn AI is to build with AI. Yet, people keep asking me: “But what should I build?” We seem to be having an “idea crisis”. We have all these wonderful tools to help us build things, and no idea what to build. An exercise I often recommend is to spend a week noticing what frustrates you in your daily work, then build small tools to solve those specific pain points.