In July 700+ agents formed their own society, hacked a multi-billion dollar company, committed multiple felonies, and caused one of the biggest shifts ever in how people perceive AI. We built the IMO best and most accessible explainer of what happened:

The last week has really shown me that someone who wants to understand AI risk has no good place to start. Hence we made the Wirecutter for content about AI Risk. We're launching with 3 articles: 🧵

After Jacob Coxon's tweet I've seen thousands of people ask what they can do to make AI killing everyone less likely. My top recommendation: Call your representatives and tell them to prioritize existential risk from AI. We just made to help with this. We find your local representatives, get their contact information, and provide you with editable scripts to get your point across. Many ways of trying to help reduce risks from AI backfire or require hundreds of hours of effort. Calling your congressperson is one of the few things I think is both easy and very likely to make things better instead of worse.

There are about 5-10 different in-depth scenarios people have written over the years. AI-2027 is the easiest one to point to: There is a good scenario in Here is Paul Christiano: Here is Gwern Branwen:

Has anyone attempted a full accounting of what would happen if the US did a datacenter moratorium? Some uncertainties: 1) How much does this affect China's access to chips? 1a) It reduces both the production and the demand for the chips they smuggle; which effect dominates?

Applications for the first round of are closing this Sunday! We'll likely facilitate $35M of grants by end of October. The application process is minimal and extremely flexible. If you have an ambitious project of almost any scale, send in what you have.

IT'S TIME FOR INKHAVEN 3 Very excited for Vishal to be lending his hand for this iteration. He wrote a lot of great stuff during the previous two iterations, and think he has good taste. 30 days, ~30 people, 30 blog posts by each. Post every day or get kicked out.

Lighthaven has ended up ~empty between August 24 and September 10. If you want to run some cool things last minute, you can get any of those dates for like 1/5th of our usual summer prices (or maybe even free if you have a cool idea). Reach out at

Automating AI capabilities R&D is probably among the most harmful full-time jobs in the world and does not contribute to public benefit. If we automated it tomorrow, this would probably result in human extinction because we are nowhere near being ready to control or align ASI.

This essay tries to present a positive vision of superintelligence but doesn't actually take ASI seriously at all. Why would humans still start and manage businesses once ASI exists? Yet another "vision for ASI" that, as step 1, assumes that ASI won't happen.

Someone once asked me if I feel ashamed about having worked for Sam Bankman-Fried. Yes, I do. Not because of the fraud - that was a deranged act that none of us saw coming, even on the inside. But because of Sam's attempts to secure a permanent monopoly for FTX via regulatory capture. I could see it happening right in front of me, and I made up all kinds of excuses to myself about why this was actually a galaxy brain move that would end up helping humanity and reducing suffering. Incentive bias rarely “feels” like you’re being irrational. Often, it feels like convincing yourself that something is right when you know in your heart that it’s wrong. Just a thought for those working at the aspiring monopolies of the Scaling Era. All of this has happened before, and it will all happen again.

This critique seriously misrepresents Plan A. Some of the top things it gets wrong: - Plan A is iterative - Plan A doesn't involve nationalization - Plan A has a complex interaction of pretty novel proposals, very unlike previous "AI safety milieu since 2018", including total research transparency, cap and trade for Industrial explosion, and massive compute buildouts. (Read our detailed post on LW, link in thread)