My final NYT column is about the AI worry moment we’re in, which has paradoxically made me feel much more optimistic about our chances of a good future.
I’m very heartened that people outside the bubble are waking up. AI is too important to be left to the labs!
Whether you’re a Democrat or a Republican, an EA or an e/acc, I think we can all agree that the person best suited to protect us against the harms of powerful new technology is Mark Zuckerberg.
Paul is one of the ~3 most credible AI safety experts in the world. Co-invented RLHF, served at US CAISI, a real OG in LessWrong x-risk circles. (His early debates with Eliezer Yudkowsky are famous among the safety crowd and people still cite his "What Failure Looks Like" post from 2019.)
He is joining the OpenAI nonprofit board now because he now believes "there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term."
Shit is getting real!
Millennium prize problems are falling, rogue agent swarms are doing heists, lab employees are begging for a slowdown…and the most popular AI videos on YouTube are from cope merchants saying it’s all a scam that’s about to collapse.
I get why the message is resonating! It’s comforting to think that the AI is all fake bullshit and the doomers are weird and wrong and life will go back to normal after the bubble pops. But if it doesn’t, these people are doing their audiences a profound disservice.