There's been a lot of ink spilled over how to create a surveillance state which prevents human beings from using superintelligence to destroy the world (like Nick Bostroms essay). The presumption at the time people wrote about this was that you would have to create a physical surveillance state. It would actually require new infrastructure.
However, a superintelligence can easily break all the encryption that we have and basically look into our digital world like an open window. Using Wi-Fi routers, AI can locate where people are in their homes; listening to keystrokes, it can understand what you're typing; and then of course everything you are actually doing on a computer is fully visible.
Using these methods, you can basically arrive through the back door at the most comprehensive surveillance regime possible. It does not require political consent, it does not require new laws, it just requires people in the background who are willing to use these tools to do this.
We also know that superintelligence will arrive in the US through one of the major labs. They will basically have this superintelligence between open source does and before china does. For them, for a matter of some months or more, they will live in this world where everything is an open window and they have unlimited power.
The only constraint, however, is that you cannot announce that you have this ability. If you do, then your subjects know they are being watched and they change their methods. You have to hold it in secret and you have to use it in secret in order for it to be an effective control.
In this way, using superintelligence, the US gov can institute the kind of surveillance state that Bostrom talked about, but they won't need political consent or new infrastrcuture. They will use this first and foremost to prevent others from independently attaining SI. Then they may also use it to their own ends. But the point is that when superintellgence arrives in this way it will first be put to work preventing the discovery of SI in all other places.
This will create a dynamic where people are surveilled and punished by this entity for actions which are not illegal, but which the entity has deemed unsafe. Will it be illegal to make SI? Probably not. Will the methods of surveilling and punishing people be illegal? In part, they will. So it will be used covertly. Everyone is going to live in a world where people around them are potentially arrested, restricted, and punished for these types of activities, and yet everyone is not allowed to know precisely why. It is an open question. People will suspect what is going on but never have confirmation, much like conspiracy theories today try to unveil the dark matter of history, this will create a much stronger dark matter in the present. A force which is clearly at work, but which you cannot name and cannot understand.
At some point you become aware of it, but you cannot concretely have knowledge of it. So you will begin to go along with it. You will go along because that's what it takes to get along, and moreover, if anyone admitted outright that the power existed, it would start to break the power's ability to keep control.
This is exactly the dynamic that existed in late Soviet Russia, and it's exactly the dynamic that will be created by a superintelligence. This dynamic is pretty analogous to what Alexi Yurchak observed in the late Soviet era and called hypernormalization: society in which everyone knows the system is a lie, and everyone agrees to act as if it isn’t, because no one can imagine anything else.
In this case I am using hypernormalization to refer to what it will be like in our own society when rules that don't exist are enforces and punishments that are not written in law are carried out on people who did not committ any real crimes, all of it done to prevent SI from proliferating. Hypernormalization.
In the end it is totally intuitive that the first person to SI could and woudl effectively use it to prevent others from doing the same. What I am talking about here is the social dynamic which will arise from seeing this process take place while not really being able to name it.
A side effect of this stuff is that the power that be will actually work pretty hard to keep the illusion of the real world or else some of their advantages will go away.
もっと見る