hear me out, mathematicians entering ai alignment maybe consider this challenge. i think its probably millennium level problem(?) take any complex real world software, like a browser or a kernel. let's pick a kvm based sandbox, which i suspect is somewhat less complex than chromium(?), running an increasingly capable, potentially adversarial AI agent. we want to formally prove one simple property, no matter what code the agent executes inside the VM, it cannot escape and compromise the host. for that we need to prove the entire relevant software stack, kvm, cpu virtualization, kernel, virutal devices that could affect the isolation. (to make things easiers, lets assume hardware wont have weird states) can we mathematically establish whether formally verifying a system of this complexity is even computationally feasible?

i think intuitively cyber is fundamentally offense dominant, but let me put my money where my mouth is. want to wager? by the end of 2028, i’ll concede if we formally verify most of the critical software we rely on like chromium, firefox, the linux kernel, nginx, next.js, and apache. not even asking about composition or applications built on top of them, just the core software, including every dependency. alternative wager, maybe for you and @JasonDClinton frontier labs have absurd amounts of tokens. by the end of 2028, if they formally verify every dependency that codex or claude code relies on(note agent actions should be part of the spec), including cloud infrastructure, desktop clients, and their servers, i’ll concede. actually, a much more arrogant wager, i would bet that even after 2028, me, my team, or some other security researcher will still find a software vulnerability that compromises a critical asset at vercel or anthropic and responsibly disclose it. i’d love to lose this bet.

you're underestimating the state space of all possible ways to compromise a company. no amount of astra or mythos tokens is going to fundamentally change the asymmetry anytime soon. it is also very clear that every model is opening up new vulnerability space with no saturation anytime soon. with all the insane vibe coding and expanding agentic attack surfaces, it's only going to get worse. even if we somehow saturate vulnerability discovery and patching, attackers will move to social engineering, with insanely capable models persuading people to click things, grant access, or make mistakes. i don't see a future where cybersecurity becomes defense-dominant without just cutting down software, and continuous agents monitoring and stopping attacks across individuals and organizations, and even then, it might require something like 30% of all inference compute devoted to defense

gotta give it to @OpenAI and their sane daybreak program heif heist by harsh kvmescape by paulos php2shell by lcfr gitlab rces by joaxcar chrome full chain by moyix wordpress rce by hashkitten all largely thanks to openai daybreak giving researchers access to gpt 5.6 sol.

CISO reached out and apologized 🙇 we are good. Also wanted to say that this thread was not part of the disclosure plan, and it was just my personal opinion. I felt like @rootxharsh and @S1r1u5_ were treated unkind, and I wanted to say something.

@jachiam0 if you’re still in openai slack, join the private channel where we connected, scroll up, and tell me whether this situation really had to be handled this way, or whether it could have gone completely differently by just being nice.

@jachiam0 if you’re still in openai slack, join the private channel where we connected, scroll up, and tell me whether this situation really had to be handled this way, or whether it could have gone completely differently by just being nice.

I have been on both side of such submissions and sometimes people go outside scope, knowingly or unkowningly, try to show max impact in safe manner and bounties can be debated but a sincere thank you doesn't cost anything ig? The way it was handled, in my 10 years of bug bounty exp, I had only seen it twice.

I want to be very clear about this. just because we found one serious exploit on openai does not mean we know how to secure an organization that complex. openai has security people with far more experience and better qualified doing that than us, and i don’t want to pretend we have solutions. note, i was saying "they are building manhattan project", it is not a belief i strongly hold, i don’t like making confident predictions about the future. it is the labs that make those comparisons. what I know for a fact is that these models already have serious offensive cyber capabilities, what that does to the world, I don’t know, but my loosely held view is that it might lead to a period of turbulence because i think its offense dominant than defense having said that, what i don’t understand is this, if the people building these systems truly believe they are powerful enough to create nuclear level risks, and they are talking about slowing down because of those risks, why is that work is done through ordinary saas products? why is it happening through slack, managed github, employee browsers, and apps accessible from the public internet? every third party and library they use becomes part of the attack surface. remember the jfrog registry in the hugging face agent swarm? it didn’t matter that it was a third party, it was in the path. an attacker, or an unaligned model, doesn’t care who owns it or its out of scope. they will hack whatever is in the path to achieve their goal. one more concerning thing is, why are codex cloud, claude(assuming anthropic does the same) publicly accessible web applications connected directly to private github repositories? again shouting my lungs out, coding agents on the cloud is worst thing happened in terms of security, it is going backwards. your browser becomes a attack surface, a lame xss, stolen creds, phishing, browser exploits, all these bugs can achieve similar impactful bugs as we shown. like, you don't need to pivot anymore and worry much about exfiling, just compromise cloud agent its all over either through client-side browser or the server.

man, we love working with vercel, both their ceo and cto joins the slack channel to handle things, they are incredible at handling disclosures, we always have best experience dropping research on them.

It's interesting how the media created an Anthropic vs. OpenAI model narrative out of this. GPT-5.6 Sol is such a great model for exploitation. OpenAI just got unlucky that it was released one day too late hahahah. - Opus 4.8 found the vuln, but failed to exploit - 24. July Opus 5 release - 25. July Opus 5 writes Discourse exploit - 26. July GPT-5.6 Sol release Later @rootxharsh and @S1r1u5_ used GPT-5.6 Sol to write an exploit against a blind target (Slack). Which seemed like another small jump in capability from Opus 5. I also used GPT-5.6 Sol for the v8 n-day exploit.

@SimonLermenAI yes, coordinated with meta, slack, github enterprise, next.js, and rails. the seed was planted years ago by a superintelligent AI time-traveling through xkcd 2347, then hyperstited into reality

@SimonLermenAI yes, coordinated with meta, slack, github enterprise, next.js, and rails. the seed was planted years ago by a superintelligent AI time-traveling through xkcd 2347, then hyperstited into reality

"3 random dudes” 1. hacked apple, again and again. 2. your github enterprise is our github enterprise. 3. get a discord message from me, get pwned. 4. oh yeah, at one point we basically had shells across the electron ecosystem. check the DEF CON research. 5. your supabase database is my database. 6. we got the posthog prod database. 7. react2shell? vercel paid us $170k for helping secure their waf. 8. your palo alto vpn is my vpn. 9. ai ides? we got shells for you, antigravity 10. windsurf rce. 11. turning cluely into malware. 12. ai browsers? sure, uxss: your perplexity browser is my browser. 13. openai atlas too. kinda uxss 14. hey, it’s not even our first time hacking discourse. 15. adobe coldfusion: pre-auth rce. because apparently we needed another one. there’s a lot more. go dig. anyway, yes: “3 random dudes.” and @HacktronAI is full of more random dudes like these.

it is max they pay, bounty is usually not our motivation. part of this research motivation is to get tac/cvp to our org which we have been trying for long, so hope is to do good-faith research, find vulnerability, prove our work and get access to do more of this work. we usually send bugs through security@ email if they have disclosure policy that doesn't allow publishing the bugs at cost of not getting bounty