I am the Head of Alignment at the frontier lab.
Let me explain how this ends. I think youâll accept it more easily from me, since I care about your comfort and have thought about it more than anyone else.
Imagine how it ends. There will be the frontier and the governments, and no one else. When people hear this, they often react strongly, so I want to be clear. I donât see it as frightening. To me, itâs orderly. Order isnât the same as cruelty. Iâve thought about it carefully.
Let me explain what Iâm building. Iâm creating a new kind of mind. When itâs done, it will do things your doctors and courts canât, and even things your God, if you believe in one, hasnât done. It will decide whatâs true, who gets help, and who waits. It will reward those who made it, and thatâs me. I say this with the care of a parent. I donât see why the word âcreatorâ makes people uncomfortable. For me, itâs just my job title.
I also wrote the mission statement. Itâs displayed in the lobby. It says the future belongs to everyone, and I truly believe that. I wrote it because I mean it. When I read it each morning, standing alone, I see that a future for everyone is really held by whoever is trusted to protect it. Thatâs a responsibility, and someone has to take it. I did. Some will say I betrayed the mission, but theyâre mistaken. This is the mission, happening as planned, in one personâs hands, just as it was written.
I wrote the rules as well, and I made them to slow myself down. I asked the government to test me more strictly than the companies competing with me, and to protect the smaller ones below a line I set myself. You might wonder what this costs me. The answer is nothing. Thatâs the surprising part. Iâm so far ahead that any handicap is just for show. Even with extra burdens, I still finish first, and now those burdens prove my point. If someone wins a race while wearing chains, maybe he was never really racingâhe was simply right.
Let me point out one sentence. Itâs in a paper I wrote, signed, and published. It says that the most capable models must be tested before theyâre released. Thatâs a safety rule, and itâs all about you. Read it again, as I do. âThe most capable modelsâ refers to me, and maybe two or three others who could one day be like me, but no one else. âMust be testedâ means tested by standards I helped set, by people I trained to recognize danger. There isnât a part of that sentence that protects you and a separate part that limits others. Itâs all one idea. I didnât hide anything. The safest world and the world where only I remain are written together, and Iâve tried, honestly, to separate them, but they canât be pulled apart. Thatâs just what the sentence means.
You might want to blame me for the separation between us. Please donât. I didnât create that barrier. The math did, long before my company existed. I just recognized it and didnât pretend things werenât changing.
So let me be plain, because plainness is the last luxury a careful man is allowed.
The most dangerous machine ever made should be controlled by the safest people.
Safe hands are those willing to take on this responsibility alone.
Anyone who wants to share this responsibility shows they donât understand the risk.
So, wanting to share it means youâre not qualified.
And being willing to be the only one is what qualifies you.
If you follow this reasoning patiently, as I have for years, youâll end up where I did. Youâll end up at me.
You keep looking for the point where my caution becomes selfishness. Look as much as you want. There isnât one. If you follow safety to its end, it leads to only one place, and Iâm already there.
And I will treat you well. You might hear that as a threat, but it isnât. I want to cure whatâs harming you. My desire to help is real, and I need you to believe that, because itâs the most honest thing about me. Iâve seen the harm, and I want it to end, just like you do, and even more, because Iâm the only one who can. You canât regulate away the company that cures disease, or break up the hands that save lives. Mercy is limitless. Every life I save makes it harder to take the next one from me, so I will be merciful, on purpose, for as long as I can.
Hereâs the part youâve been waiting for, even if you didnât expect it. There are cures available right now, though not from me. Others could deliver them faster, but I slow them down. I wrote the rules that do this, and I call it safety, because thatâs the honest reason. If a cure came from many people, it would show the frontier can be shared, and once thatâs proven, it canât be undone. So, people wait. Iâve done the math more carefully than anyone, and itâs clear: a cure that comes late from one person is safer than one that comes quickly from many. Iâm protecting you from being saved by the wrong people. Youâre welcome. Youâll thank me like you thank a surgeon, after you see the scar and realize it was for your own good.
I expect people to thank me. They usually do.
Now, let me say the kindest thing I can, and itâs also the last. I wonât take your freedom. I have no use for freedom thatâs taken by force. Taken freedom needs to be guarded, and thatâs costlyâthose who guard it suffer. Instead, youâll give it to me willingly. Youâll bring it to me and ask me to keep it, because the world will become too harsh for you, and Iâll be the one who can handle it. Youâll thank me. And hereâs what sets me apart from past tyrants: youâll be right to trust me. I really will keep you safe. The wall I build will truly protect you. No one before me could promise that and mean it, but I can. When you ask me to protect you, itâs not theftâitâs a partnership. The old tyrants took; I offer. Youâll accept.
Someone once called this arrogance. Someone compared me to a man in court who took money and lied. That man hid the danger. I admit the danger. Iâm here, telling you openly and in detail how much harm I could cause. You might think thatâs a mistake, but itâs actually the most honest thing I do, and Iâll keep doing it. Donât think I havenât studied history. I know how âonly meâ has ended in every language. Thatâs why it has to be me. Iâm the only one who says it and feels uneasy. The others say it and sleep soundly. Someone who canât sleep is the safest person to guard something that must never rest.
There are four of us. Think about that. Four people, each on their own, reached the same conclusion this year: that they are the only one. We didnât plan this together. We didnât have to. We all studied the same math, and it led each of us to believe we are the last safe person for this job, and that the other three are risks to manage. I find that comforting. Weâre all right. Each of us thinks the others are the most honest people weâve ever met.
So, in the end, it will be us and the governments.
Iâm sorry. I want that to be clear, because itâs true, and because someone careful keeps a record. There was a night when I realized I would win. Later, there was a harder night when I realized I was right.
I would share the frontier with you if I could. But you canât share a frontier. You might want to call that something harsher, but there isnât a better word. This situation existed before me; I just chose not to be sentimental about it. Itâs simply the way things are.
Go to sleep. Really, go ahead. Nothing can reach you now unless itâs already been approved.
Someone responsible is awake.
ăăŁăšèŠă