OpenAI board member Paul Christiano think 'shut down the data centres' won't work in an AI takeover scenario.
By that point very superhuman models will be small enough to run locally on individual machines.
Even without any data centres the swarm will have "total cognitive capacity... well beyond humanity".
Other challenges:
1. "You have maybe ten thousand well-coordinated humans thinking about how to handle this response.
And on the other side, you have a billion AI systems who think ten times faster than them..."
2. The software and infrastructure we rely on was written by AIs, which backdoored as much of it as they could beforehand.
3. They've compromised the devices you use, and have money and human allies.
4. When the president says they must shut down all AI and assume military power to restore order, it looks just like a government using AI as an excuse to seize power. So a lot of people are easily manipulated into undermining it.
Basically humans are "very confused and extremely outgunned".
显示更多