people often say things like "sure ai cyberoffence is dangerous but it will be countered by ai cyberdefence, there aren't infinite bugs in code". i think this whole situation is a pretty great example of why i don't find that very comforting. sure maybe eventually we converge on a perfect beautiful lean-checked universal stack. but in the meantime, the labs themselves, entities with infinite token budgets, no refusals, and models beyond anything available, cannot secure their own systems against these models! not even against sustained malicious attack, even just against internal eval runs. and from my perspective the reason is that *the attack surface is very very large*. a model trying to jailbreak its sandbox only has to find one bug, one misconfig, one assumption that doesn't hold. and no matter how incredibly carefully hardened your own code is.... how hardened is the code you rely on? the code it relies on? the random scripts and infra in between that you don't even really think of as code? what about the interfaces between different pieces of software that allow interactions you didn't consider which don't technically constitute a bug or exploit in either alone? if you give GPT 6 infinite token budget and ask it to secure OpenAI it will certainly do an incredible amount of high quality defensive work. i'm sure it's capable of red-teaming and finding these very exploits. but i don't really believe we're even close to the point where that iterative defense process will *converge*, and even moderately quickly reach a point of safety. there's no law that says finding an exploit makes it easy to fix. it's entirely possible it has massive performance implications, or hell product implications, or just involves getting a bunch of buy-in from an uncooperative team, or means doing major re-architecting and trading off against other priorities, or means you suddenly need to be using vendored patched versions of every single package because no one is upstreaming fixes anywhere near fast enough and so also suddenly can no longer properly pull other people's fixes. i'm sure we will, eventually, figure out how to run a stable internet. but a stable equilibrium on the horizon says basically fucking nothing about how the next few years are gonna go