ARC-AGI-4 will be a benchmark for autonomous open-ended innovation. It will continue our commitment to open-source, giving the research community a shared target for progress that benefits all of humanity. Despite rapid model progress, humans still significantly outperform AI at open-ended invention. This is the meta-skill that unlocks progress across every field of technology. Advanced AI capable of scientific innovation will lead to tremendous new technology, knowledge, and understanding. This is a positive-sum future. We are deeply committed to advancing it. Open source is the foundation for that progress. The knowledge behind frontier AI, not just the technology itself, should be broadly distributed among researchers, academics, and organizations. Any coordinated effort by the AI industry to reduce openness or concentrate access to frontier AI would undermine that positive-sum future. We are committed to advancing a future where everyone can contribute to and benefit from AI progress.

I give two books away more than any other - The Lion Tracker's Guide to Life by Boyd Varty - Why Greatness Cannot be Planned by @kenneth0stanley Both changed how I think about curiosity, which projects to pursue, not forcing outcomes...Relevant for building, investing, parenting... It was an extreme privilege sitting down w Ken at Festival. Hope you enjoy the beautiful conversation and his wonderful insights. Thank you Ken!!👇

friends are regularly surprised when i say i don’t think math is (yet) “solved,” so in the wake of the ns result i figured i’d explain this take a bit more widely. first, this is massively impressive and clearly an example of AI being, in some respects, far more powerful than the human mind. the team deserves huge praise for attempting and succeeding at this. i’d love to read a technical report on the project (one can always hope :) but second, here again we ended up with a counterexample rather than a full proof: option C won the NS problem by proving the conjecture false (which was one valid way to solve the problem for sure) notice a pattern in many of the recent frontier results in ai for math? a striking number involve counterexamples or finding a needle in a haystack. now don’t get me wrong: this is extraordinarily hard and commendable. but it is only one aspect of mathematicians’ work. Math is also about: - finding deep, general mathematical understanding and explanations within proofs - revisiting proven results to find more « elegant » proofs - proposing new conjectures and hypotheses that might open « fruitful » directions - taking the leap of faith of proposing entirely new, « exciting » research programs that may take decades to bear fruit you may say these are simply further increments on the same intelligence scale, and perhaps point 2 above is already within the reach of current models. possibly but you could also see this string of results as an extraordinarily powerful, massively parallel extension of search: explore the haystack, find the needle i.e. the counterexample that breaks the conjecture. that would already be remarkable. but it would still leave open whether models understand the words I emphasized above: « elegant », « fruitful », « exciting ». as tristan put it in his brief report: “the first llm generated proof levent sent me was the most horrendous i have ever read.” despite their formidable technical abilities, models still seem to lack something mathematicians rely on constantly: a type of mathematical taste. the ability to distinguish a beautiful argument from a merely valid one, a fertile idea from a sterile one. i’d be very curious to ask gpt or claude what they think of the result they just proved. do they find it elegant? if they ignore all the human noise about it on the web and history of math, do they appreciate the result and the path to it more than other proofs? my suspicion is that, outside of the human knowledge that this is an important millennium prize problem, it may not 100% register it as fundamentally different from thousands of other proofs. or, more intriguingly, perhaps the beauty they find in mathematics is actually alien to our own sense of mathematical beauty, some « horrendous » proof to us might be deeply satisfying to them… all this to say that we might have significant progress to make in ai for math before declaring the field “solved,” (as too many are posting). in the meantime, ai will be an extraordinary collaborator for mathematicians. i just hope access to these tools be as widely shared as possible (so that you don’t need a Leven or Sebastien working in a big lab to help you), and that companies stop using mathematics primarily as a demonstration of prowess in their private race.

honored to be named to @techreview 's Innovators under 35 list! the article below highlights the work we're doing @LilaSciences to build scientific superintelligence with @GVMaltzahn , @AndrewLBeam and an amazing team of too many folks to mention we are working on scaled verification of AI hypotheses in the real world and are going full stack from post-training language models to running automated labs

GPT-6 Astra is the new SOTA on ARC-AGI-3 It's a qualitatively large leap towards AGI and the pace of progress is frankly surprising. That said, we lack evidence to call this AGI yet. While we are still studying the human capability gaps, we believe open-ended invention is unsolved, and this will form the new basis for ARC-AGI-4. Reminder that ARC v3 tests whether models can figure out how to efficiently make sense of unfamiliar environments and operate autonomously toward goals within them. Summary: properly equipped Astra can now do this. Astra direct model scored 66% at ~$500/game. Up from Sol 8%. This is apples-to-apples with all other ARC v3 scores we've verified to date. This score gives the best look at the general intelligence of base Astra. Using an OpenAI-specific adapter (which is open source) we verified the first ~100% at $300/game. If you're using Astra via API I would prioritize migrating to /conversations endpoint with compaction enabled. Direct model testing continues to carry important scientific interest towards AGI. But provider adapters are a better estimate of what you should expect day-to-day when using these models inside of products. The action efficiency of Astra is also pretty incredible. With the provider adapter, Astra used 50% fewer actions on average than our human baseline (who both score 100%). A surprise for us is that across all models, action efficiency on v3 is very bi-modal. Models either get the game or not. Weaker agent models aren't able to "brute force" their way to an inefficient understanding. Tactically, our results support always using Astra's higher reasoning tiers when deploying agents into environments. It not only makes Astra better but is cheaper too (as a result of needing less turns). The key technique Astra seems to leverage is on-the-fly symbolic world modeling. It develops a custom algebra and notation for the environment through interaction and uses this understanding to plan (all in token space). This approach is similar to earlier harnesses we saw which created symbolic models through side-car executable programs to model the environment and plan. Based on Astra results I now see clear paths to recursive self improvement via horizontal data scaling and continual learning where the learned state is information stored and managed outside of the model weights (text, programs, databases, etc). I expect more and more "learning" to happen outside the model weights and this will be of growing interest to both researcher and AI product builders. One remaining area of important capability we haven't seen demonstrated yet by any model is open-ended invention and discovery. Humans clearly posses this capability and we've so far seen zero evidence that frontier AI -- even as powerful as Astra -- is capable of this feat. Consider AI research 10 years ago vs today. Humans invented so much: transformer, self-supervised LLM pretraining, diffusion, RLHF, test time adaptation, ... Society needs AI systems that not only can autonomously operate towards defined goals. We need AI systems help craft great futures that are inherently undefined. I believe we in at a very tenuous spot. We now have extremely powerful automation system that capable of causing a lot of disruption. Without the counterbalancing force of demonstrated invention by AI, I fear society will turn more and more against open frontier AI research and development -- trapping us in a very bad middle ground. We must move as swiftly and with focus towards this important AI capability. And this has become the primary focus of our work on ARC-AGI.