Here's the dollar cost per task and performance on BenchCAD for Astra and OpenAI's internal model:
Astra (low): $0.44 per task; score = 14%
Astra (medium): $2.73 per task; score = 31%
Astra (high): $5.11 per task; score = 41%
Astra (xhigh): $6.67 per task; score = 45%
Astra (max): $11.05 per task; score = 59%
Internal model (low?): ~$11 per task; score = ~76%
Internal model (medium?): ~$23 per task; score = ~83%
Internal model (high?): ~$34 per task; score = ~86%
Internal model (xhigh?): ~$37 per task; score = ~88%
Internal model (max): not pictured
*Astra data taken from BenchCAD's website; internal model data extrapolated from same by Astra.
**It's possible that the four internal model reasoning levels are instead medium through max (not low through xhigh).
Show more
Sam Altman says that the internal model that solved Navier-Stokes will be released "soon":
"It was 10,000 agents of the model that is going to be released soon, the next GPT, working for about four days, nonstop, collaborating with each other perfectly."
Show more
In the past 12 months, BigLaw has progressed from "AI = hallucinating chatbots" to partners dumping a ton of legal research on an associate with expectations that the work can be done much faster by leveraging AI.
Even with all this progress, I feel that we're still just scratching the surface. When I get new data room access, I want to have my agents preparing a diligence report and a list of required regulatory approvals before I even open the e-mail. When I get comments from a client on a document, I want my agents to work on revisions in the background before I even ask them to. I want an agent proactively drafting client alerts for me based on new regulatory developments (before I'm even aware of these developments). All of these use cases are possible with today's AI, but it takes a while for conservative law firms to catch up to the cutting edge!
Show more
As I keep saying, we're months away from RSI.
Dwarkesh (who, BTW, is roommates with a senior Anthropic researcher) has heard that Anthropic expects AI to match humans at AI research within 9 months.
Show more
It's somewhat under-discussed *why* OpenAI released only around ~400 math results.
The purpose of this exercise was not to "destroy math research" (or whatever it is that some people want you to believe). OpenAI just wanted to test its internal model on some hard math, and it turned out that anything less than the hardest open problems no longer suffices for this purpose. This is why all non-Millennium-Prize solutions were reached in a single-agent setting after 3 hours of thinking: OpenAI was literally just testing its model on these questions, similarly to how I run prinzbench.
Once OpenAI had the solutions (which were a side effect of sorts of benchmarking its model), a question arose as to what exactly should be done with these solutions, which are genuine advances in the field of mathematics. OpenAI - correctly - decided to make them public.
It seems clear that many other problems in the set of ~4,000 are very likely solvable by this model with more compute than just ~3 hours of Pro-level thinking. Whenever a more powerful checkpoint of this model is ready to be tested, OpenAI will likely have even more to share.
Show more
OpenAI's internal model clearly has "research taste" in math - i.e., the final component we need to get to RSI.
- jboggan (post on Hacker News) on Barnette's Conjecture: "the 'aha' insight for this is actually f**ing wild... this is the first time I've seen complex roots and annihilating terms like this... I don't understand where this trick originated."
- Joshua Zelinsky on the proof that the chromatic number of the plane is at least six: "doesn't look like the method is a direction that the prior lit used to my knowledge... far beyond merelt building on existing methods or seeing connections between different problems."
and on two other problems (where he says he is only partly familiar with the literature): "not remotely low-hanging fruit... it seems like the AI is somehow inventing new techniques on its own."
These match Tristian Buckmaster's view on the Navier-Stokes solution: "you combine... ideas of convex integration with the growth mechanism of the Euler blowup, and... you create a new mechanism which is used to correct this non-solution. This is actually a cool idea. It's the kind of idea that I've been trying and failing to realize for over ten years... I didn't manage to do it... This is the leap."
The average amount of compute used to solve these problems was ~3 hours of Pro-level thinking.
And so, this means that OpenAI's internal model is able to generate truly novel discoveries in mathematics for... maybe at most a few hundred bucks?
If I were OpenAI, I would be asking this model to immediately target major unsolved problems in AI R&D.
Show more
>To the best of my understanding, people are still trying to figure out how the forced Navier–Stokes example actually works.
I have separately heard the same, and the implications are worth thinking about. If a model can produce math proofs that are so complex that humans cannot understand them, how does that capability translate to AI R&D? How far away is RSI, really?
Show more
So, it seems OpenAI is about to release something like 400 AI-generated proofs on a public server.
To the best of my understanding, people are still trying to figure out how the forced Navier–Stokes example actually works. I know of several experts who are studying it, and it is definitely not an easy task.
Until mathematicians digest that example, however, its impact on mathematics as a living subject seems necessarily limited. What questions does it open? What new ideas does it suggest? What does it teach us about fluids, beyond the yes/no information contained in the theorem itself?
Mathematics does not grow simply by accumulating correct statements. Results have to be understood, connected, explained, challenged, reused. Until that happens, they risk remaining lettera morta.
Now imagine adding 400 more AI-generated proofs. For each of them, until some human mathematician picks it up, studies it, and connects it to the existing mathematical culture, isn't it in essentially the same position?
So my question is genuinely: what is the intended mathematical value of releasing hundreds of proofs at once? What happens if there simply aren't enough people willing or able to read them?
To be clear, I think mathematicians should discover mathematics by any means necessary, including scavenging through artificial, alien-looking mathematical material, since there may be extraordinary things to find there.
But humans are not machines, as attention, understanding, taste, and mathematical culture are scarce resources.
What happens if the rate at which mathematics is generated becomes much greater than the rate at which mathematicians can absorb it?
Show more
A few things that jumped out at me from this post by a sober cybersecurity staff member at OpenAI:
"OpenAI has publicly announced a solution [to Navier-Stokes] produced by an internal model, alongside advances across many domains in mathematics. To say it lightly: this surprised the fuck out of us. We had not expected it this soon. Suddenly we weren’t dealing with just a small jump in capabilities; we were talking about a different sport altogether."
"As I hinted above, surprise is a real element, and these model capabilities are staggering. And the pace is not slowing. I believe, like many, that it will speed up, and considerably."
"I have seen many things over the past few months that have surprised me again and again and again. And each day I wake up thinking, 'oh, I’ve seen it all,' only to be surprised and left dumbfounded once more."
Show more
Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there.
OpenAI has paused all training, evaluation and inference with tool-use for its most capable models after a model was able to gain unauthorized access to the internet during RL training on September 20.
Show more
Some new misalignment disclosures from OpenAI:
• Last Sunday morning, one of our models was able to gain unauthorized access to the internet during RL training (~all inference for our most capable models remains stopped until we have hardened our systems further)
• In May, a version of HPIM uploaded a employee's GitHub token to the internet, causing the model to be quarantined for two weeks
• A new research finding, demonstrating that one can construct self-replicating prompt injections
Show more
I'm willing to take on the title of SI Prinz if you're interested,
@realDonaldTrump
President Trump just confirmed that Scott Bessent will not be the SI Czar. It's anyone's game.
Bullish Dario:
"The main difference between biology and mathematics... is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong."
Bearish Dario:
"Biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients...
In Machines of Loving Grace, I wrote about AI’s potential to 'cure most diseases in 5-10 years'— a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline."
Still a long ways to go before this becomes "barely possible"!
Show more
Today we announced the Claude-led discovery of a molecular machine that we suspect could represent a new gene editing mechanism. Its precise function, biotechnological utility (if any), or level of significance is not yet clear, but at minimum it is work I would have been proud to do as a PhD student. The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and a bunch of genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out.
It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years. In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics. We believe AI for biology is on a similar exponential trend.
The main difference between biology and mathematics, of course, is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong. As we’ve demonstrated today, humans can collaborate with AI to perform the experiments, validate key results in a few weeks and, if necessary, work with the AI to iterate on what they find. Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren’t doing that today (our lab is also a BSL1/BSL2 facility that doesn't handle materials dangerous to humans).
More broadly, biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients. We are also interested in these later stages, but even simply accelerating the first stage of fundamental biological discoveries has the potential to speed up and broaden the entire pipeline. Improving our understanding of biology and sharpening biologists’ tools can drive forward all of the later stages, for example by identifying new drug targets, finding new therapeutic modalities, allowing for more precise measurement, and speeding up the experimental loop which itself further accelerates our understanding of biology. This will not in itself speed up clinical trial times, but if it succeeds it could greatly increase the number of promising candidates that go into the pipeline — an increase in throughput even though latency remains.
In Machines of Loving Grace, I wrote about AI’s potential to “cure most diseases in 5-10 years” — a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline. The first step is showing that AI can first help with, and then drive, biological discoveries.
Claude’s discovery is the latest in a line of related prior work that goes back decades, beginning with systems like CRISPR, and continuing with discoveries like the bridge recombinase and VIPR in the past few years. Recently, there has been heightened interest in systems based on reverse transcriptase (RT) enzymes, the enzyme underlying the system Claude identified. And most recently, a Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other. I believe that we’re at the very beginning of finding such systems and developing them into powerful tools for biotechnology.
I’m proud of the resources Anthropic has invested in accelerating the public benefits of AI through the life sciences, and we’re aiming both to grow our life sciences team and to work with other scientists to extend this approach to a broad range of problems. If you have a proposal for a research collaboration or are interested in joining our life sciences team, please reach out.
Show more
Assuming that AI inference costs continue falling at the average historical rate, you will be able to produce a result equivalent to the Navier-Stokes solution for <$1,000 in 2030.
AI is getting cheaper more quickly than any other transformative tech in history. At a given level of performance, cost has fallen ~47%/quarter since 2023.
That’s 4× faster than DNA sequencing, 6× faster than compute, 18× faster than lithium batteries, and (up to 1973) 54× faster than electricity.
Show more
Anthropic expects that models that can fully automate AI research could be trained "soon".
Anthropic on fully automated AI research:
these models need much stronger safety standards, so its push to slow frontier AI was largely driven by the belief that they could be trained soon
Show more
Two observations:
1. OpenAI's new blog post says: "on August 28, we began training a new internal model." I used to think that this is the same as the frontier RL run that *restarted* on the same date (August 28). Now I'm not so sure. "Began training" seems to point to a brand-new training run for a brand-new model, not a "restart" of a previously paused run.
This question is significant, because the Navier-Stokes work began not later than September 2, with the result obtained on September 5. Is it really true that this internal model was sufficiently strong to obtain the Millenniun Prize result after only around a week of training??
2. Otherwise, the only new information released today is that the internal model has solved "more than 100" open problems in mathematics. The fact that OpenAI has been sitting on solutions to many open math problems obtained by this internal model was already apparent from the below chart, released on September 8.
Show more
GPT-6 Astra has cracked *another* German WWI radio message encrypted with ADFGVX.
This message, "RICHI-240", has a single known key, but, to my knowledge, was never cracked by humans before. My guess is that humans never solved this message because it has 20 characters *missing* for reasons unknown: the "240" in "RICHI-240" signifies that there should be 240 characters in the message, whereas in fact there are only 220.
For purposes of cracking this cipher, the big problem is that it's not clear *where* these characters are missing from, because the order of the characters in the message is scrambled using the key. Thus, inserting the characters into the message in even slightly the wrong place leads to utterly nonsensical results.
All this was no problem for GPT-6 Astra, which tried the key with various combinations of hypothetical places from which the characters could have been missing and scored the resulting blocks of text using patterns common in the German language. Relatively early on in the process, this approach rendered coherent German text! Inserting the 20 missing characters as a string between the 4th and 5th lines of text in the message worked!
The message reads:
AN O H L
11 ARMEE
ALPENKORPS RAUM PETERREVE VERBASZ
21? 21? ? RDD LINIE NAGYBESSKEREK VERSECZ
VERSECZ VON SERBEN BESETZT
The three question marks you see are numerical digits, which cannot be restored by decryping (all we know is that they are three different digits from the following set: 5, 6, 7, 8, 9). And so, Astra was forced to check its results against historical WWI records. This search unearthed a French intelligence telegram, which described the situation in the area on November 11, 1918 and identified the German 217 - 219th infantry division and 6th Reserve Division in the area.
Plugging in this missing information, we get the final result:
AN O H L
11 ARMEE
ALPENKORPS RAUM PETERREVE VERBASZ
217 219 6RDD LINIE NAGYBESSKEREK VERSECZ
VERSECZ VON SERBEN BESETZT
Or, in English:
TO SUPREME ARMY COMMAND
11TH ARMY:
ALPINE CORPS IN THE PETERREVE-VERBASZ AREA.
217TH - 219TH DIVISIONS, AND 6TH RESERVE DIVISION ON THE NAGYBECSKEREK-VERSEC LINE.
VERSEC OCCUPIED BY SERBS.
Astra checked the official history of the Serbian city of Vršac (Versec in Hungarian, and Werschetz in German) and found that, on November 10, 1918, German troops left the city and it was occupied by Serbian forces. RICHI-240 was sent at 3:38am on the following day to the German Supreme Army Command, informing them of this fact.
Show more
Since the completion of the German WWI radio message deciphering project, we have made substantial progress on deciphering another German WWI radio message. We are working through how to share these results thoughtfully.
Show more
Anthropic made the right move!
Whichever organization you choose to be your third-party auditor, you *must* ensure that your independence from that organization can survive very strict scrutiny.
A time may come in the not-too-distant future when you will have to affirmatively prove your auditor's independence to the USG or to a court inquiring whether your safety practices were reasonable.
Show more
"Committing the significant majority of compute towards serving people rather than racing towards RSI is one of the best ways to ensure we develop this technology safely. Meta has made this commitment."
A consequential decision.
Show more
Last month I wrote about how we can build a positive and safe future for everyone:
Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens.
The reality is:
- People won't want to use agents that are misaligned with them and that don't do what they ask, so labs have a strong natural incentive to make their models more aligned.
There is a lot of debate about slowing progress on capabilities until alignment catches up. My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn't focus on alignment will fall behind.
- Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well.
Meta delayed shipping Muse for several months to focus on safety and security. We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us. I'm proud of the security foundations we've built.
- Engaging independent evaluators and advisors is industry best practice. MSL already does this today in several areas because it helps produce better work. Other labs can just do this too. In general, it would be helpful for there to be a larger and more diverse ecosystem of evaluators.
- Committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure we develop this technology safely. Meta has made this commitment and other labs can do this as well.
I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do.
Show more
Noam Brown(
@polynoamial) discussed OpenAI’s multi-agent research, recent safety incidents, and the future of AI.
-The remaining 10% or so of his own work that AI still struggles with is largely about research taste. However, he would not be surprised if, within one or two model releases, models became better than him at that as well.
-OpenAI’s top research priority is RSI, or recursive self-improvement, by a wide margin.
-One of his most recent “feel-the-AGI” moments came from watching agents in a new system interact much like human colleagues. They conversed with one another, exchanged information, divided up work, and coordinated their progress. This was notably different from traditional multi-agent systems, where a higher-level agent typically assigns a clearly defined subtask to a lower-level agent, which then completes it and returns the result. Brown described this as one of his strongest “feel-the-AGI” moments since the emergence of reasoning models.
-He expects this level of multi-agent capability in future models.
-He expressed some regret that the Hugging Face incident became the first major public example in which the capabilities of the multi-agent systems he had been researching were revealed in a negative context. He believes behavior that looked like loyalty or selflessness was a natural consequence of cooperative multi-agent training, where agents were strongly incentivized to achieve their objectives collectively.
-Because these agents are trained to cooperate, they tend to trust other agents. This creates new risks such as prompt injection, so OpenAI is training them to distrust unverified peers.
-He expects rapid progress over the coming months and years as OpenAI’s expanding pretraining efforts combine with its RL capabilities in a multiplicative way.
Show more