Founder Editor of The Blind Spot @theblindsp0t and The Peg @cashequivalence. Previously POLITICO, and FT. Fights for the human. Typos, syntax errors guaranteed.
It’s worth reading the actual Dario letter in full. Even tho it’s over 3000 words.
The proposition he makes is more like soviet era detente. It really reminds me of Nixon era outreach to Russia over nukes.
And the comparison is all the stronger when you realise
detente was influenced by America becoming over stretched in Vietnam and facing a growing fiscal and balance of payments crisis.
But the end result of detente wasn’t that nuclear engineering stopped. Build out and silo growth was curtailed but engineers continued to advance capabilities. SALt also paced certain capabilities over others in a negotiated way to either iron out asymmetries between powers for MAD purposes or lock in negotiated strategic advantages.
In the AI letter Dario makes the SALT comparison. But his proposition is mostly to pace recursive development only after locking in a greater American advantage. To achieve that he needs permission to basically forge a domestic cartel and to lockdown development work in ways that prevents China getting access that enables distillation. Hence my Manhattan project quip.
But the Chinese would
never agree to any detente on technological development or recursive influence on their front without some compensating trade off from the US. And for them it would either be access to nvidia chips or a promise that America will slow build out while they catch up on innovation.
As the doc that is “coincidentally” circulating outlining Chinese perspectives on AI risks shows, they are less concerned by AI is going to kill us risk and more concerned about the risk that AI might undermine the CCP.
Either way the pacing doc introduces the idea of an international treaty which regulates the recursive and architectural engineering element in China. But such an agreement is MEANINGLESS if the CCP loses control, which it could well do because the Chinese models are open weight and as its own risk paper shows that is the greater risk.
So what is really on offer is a quid pro quo. We reduce buildout and the recursive element of our technology (focus more on advances that we control and architecture), and maybe give you a few more chips, and you clamp down on open weight models and bring more of your research in house too.
So perhaps the better way to read it is that this is an outreach to China to get the open weights under control.