Yeah, this is a really important question. I’ve been associating with rationalists for almost a decade. Their ideas were compelling and you can’t deny their foresight. The founders of every superintelligence effort are all bound up in the rationality sphere. Sam Altman and Greg Brockman used to frequent the MIRI offices. The way I relate to rationalists is it’s like the bizarre hypertrophied physiques on specialized athletes. In my opinion they have weirdly specialized minds that make them uniquely able to be correct about topics removed from daily experience. These “specialized minds” also can make them off-putting to many people. I dont know if I’d consider myself a rationalist. I am intuitively opposed to polyamory, veganism, and consequentialist intuitions. But I certainly trust them to be correct about factual questions.

0. The core disagreement was about the inevitability of a race 1. I think leadership is way too paranoid about China and the US government. They don’t believe it will be possible to negotiate. 2. They largely initiated the recent race to RSI, because of a belief in its inevitability. Note that OpenAI had to shed a bunch of dead weight like Sora because Anthropic was going for the jugular. 3. Even if they are **not** being pessimistic, I disagree with their consequentialist philosophy. If the race is inevitable you should not contribute.

I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities.

A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.