Very well written post from Dario. The US must dominate in AI. And we must do it responsibly.
In 1957 the International Atomic Energy Agency (IAEA) was formed. The IAEA stemmed from President Eisenhower's 1953 "Atoms for Peace" proposal.
In very over simplified simple terms, the IAEA purpose was to ensure safe use of nuclear technology across the world. Verify that where uranium was being enriched, the enrichment levels were enough for energy but not enough to make a bomb (low single digit percentages not 90%). What could be used for good (energy) could also be used for bad (bombs). We needed a global institution to help ensure the bad uses weren't happening. Failure would be blocking this technology from being used for good....
They did this with specialized tools and site visits, forensic investigations, and ongoing monitoring solutions
Doesn't sound too different from what Dario is proposing with Embedded Evaluators! Independent third party actors with desks in their office, company laptops, access to company workspaces, etc
Back to the nuclear analogy. Fast forward to the late 70's / early 80's and the United States canceled the development of 100+ nuclear reactors (Three Mile Island partial meltdown in '79 caused a lot of the fear). What's important though is the IAEA didn't shut down development, fear did. The US then went 30+ years before building a new one.
As
@altcap discussed on CNBC yesterday, what's important is having balanced discussions about the risks. We don't need hyperbolic hypothetical statements that drive fear. The Hugging Face incident kick started fear, just like the Three Mile Island incident did in 1979. Too much fear (without balanced discussion) will lead the US AI's dominance to the same fate it's nuclear dominance fell to
The US is now paying for the reactor freeze from the late 70's. Our energy dominance is diminishing. We're seeing a huge power crunch. While we paused, other countries marched on with reactor development. Unfortunately, China and Russia particularly. France is also a gold example, with well over 50% of their electricity today from atomic energy.
The full circle irony - Three Mile Island is now being re-opened and redeveloped for AI data centers. The "Hugging Face incident" of the nuclear era is now coming back for AI.
We can't let fear grip us and force a similar pause in AI. And I think it's important to point out Dario isn't asking for a pause. He's asking for responsible pacing. These are different. The IAEA didn't cause a pause. Embedded Evaluators similarly won't be the reason we stop building data centers. I thought Dario's post was well written and balanced.
However, we can't fall into a cascading fear loop. This is what our adversaries want (look who's now dominating nuclear while we play catch up...)
It's imperative that the discussions around the risk are grounded in reality and good faith. Dario's essay is a great step
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here:
Show more