It's so sad that AI developers are creating affordable, powerful mental assistance for everyone—and yet are afraid this is somehow going to destroy the world.
"AI" is a new type of *computer*.
We need to be careful how we *connect* it to high-stakes systems.
That's it.
"AI went rogue" amounts to:
1. Someone took a powerful, open-ended computer that they don't perfectly control (i.e., one running LLMs).
2. They deliberately reduced their control over its ability to produce potentially harmful outputs.
3. They connected the deliberately-badly-controlled computer to a high-stakes system.
Whether this is done irresponsibly or as a legitimate exploration of potential risks, it does not justify doomerism.
The proper conclusion to draw is that we need to responsibly develop LLM computers and be very responsible about when and how we connect them to high-stakes systems.
Not that AI is some interested, conscious being that has a 10% or even 0.0000001% chance of killing us all.
"AI" is a bunch of computers running an amazing new kind of software that we can and should use to be of enormous assistance to our lives.
AI computers are no more self-interested or conscious than a calculator or an abacus—even though their outputs do a great job of mimicking interests and consciousness.
While there is a real threat of people accidentally misusing AI assistance to do a lot of damage, that threat is always going to be limited by the humans who ultimately direct AI computers and can literally unplug them.
The biggest AI threat by far is bad actors deliberately using AI assistance to do a lot of damage.
To prevent this threat, as well as to reduce the damage of accidents, we need good actors, above all in the US, to proudly lead in this technology. Not to indulge in fear-mongering based on metaphysical nonsense.