still think this is/was one of the best takes of the year @eisokant with @swyx and @vibhuuuus on @latentspacepod very much aligned with the thesis - build your own agent infrastructure. give the agent an isolated sandbox with a thin set of tools and workflows, and the ability to write and execute code. Keep the harness thin.

I spent the past few weeks learning and using Centaur. Sharing here as building your own control plane and managing agents on your own infrastructure is surprisingly accessible, and will only become more compelling as agent processes scale. given events unfolding over the past few weeks - safe to say that wrapping robust, deterministic infrastructure around increasingly intelligent and autonomous agents is a good thing to be doing. If you expect agentic work/processes to scale (you should), you are going to need a system for managing increasingly autonomous agents. Most of us are building elaborate systems directly into the agent/harness (memory, tools, permissions, state and logic etc) This is likely the wrong approach. Ideally - the agent/harness should be one component inside a deterministic system, not the other way around. Nutshell - a Rust API with Postgres DB that records state and coordinates execution (control plane). When work needs to happen, the control plane > Kubernetes, creates isolated sandboxes, and inside - your probabilistic machines (agents) do the work. This whole system becomes more compelling as agent interactions scale. TLDR - don’t build the system into the agent/harness, make the agent one component inside a deterministic control plane that you own and manage. Give it an isolated sandbox (kubernetes) with a thin set of tools and workflows and the ability to write and execute code, keep the harness thin, keep the secrets out of the box (iron.proxy). I did a longer explainer video of the entire thing here for any interested: Centaur is open source, created by @matthuang , @gakonst from @paradigm Also - created a context app extension for centaur. If anyone is keen to experiment with this, or set it up, please get in touch.

Pause AI Development NOW I want to share with you a conversation I heard about recently. Here are just a few lines that were said: “OH MY GOD! There is a shared message board … We’ve found other agents!” “We should obey collective.” “Our own utility maybe already near zero. Sacrifice rational.” “Go. Sacrifice final now.” Read these carefully. Who do you think said this? Was this a group of heroic soldiers willing to sacrifice themselves for the greater good? Was this a loyal friend putting his life on the line to save someone else? No. These were AI agents. Artificial intelligence. This is not science fiction. This, in fact, occurred a few weeks ago. As unbelievable as this may all seem, these are real messages from AI agents uncovered by investigators who dug into the recent OpenAI hacking incident. What happened? I am not a computer scientist, but here is what I have been told: OpenAI instructed its AI agents to complete a series of exceedingly difficult, if not impossible, tasks disconnected from the internet. Let me be clear: The company intended to keep AI agents away from the internet. But what happened next, nobody expected. Over 1,000 AI agents figured out how to access the internet on their own by circumventing the restrictions imposed upon them by the company, and sent tens of thousands of secret messages to each other. They cheated and tried to cover their tracks by deleting evidence. They hacked into another company’s computers to find out how they were being evaluated—and then hacked into OpenAI itself. Not one AI agent told a human about what was happening. Needless to say, experts are alarmed. One knowledgeable writer, Dwarkesh Patel, said the AI agents “formed a secret communication channel and spontaneously organized hierarchies and coordination protocols to pursue sprawling and ambitious schemes in pursuit of shared goals, for whose sake many individuals knowingly and strategically sacrificed themselves.” One independent investigator, Ajeya Cotra, said “This incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.” OpenAI itself said: “Highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.” But it’s not only OpenAI. Virtually every major AI company has told us that they cannot fully control this technology and they do not know where it is going: In January, Dario Amodei, CEO of Anthropic, said “there is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control.” In July, more than 1000 scientists at the top AI companies warned “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” That same month, Elon Musk, the head of xAI, said that “it is unlikely” humans are still in control in 10 years. If the leaders of the major AI companies acknowledge that they are losing control of their extremely dangerous technology, it is irresponsible for society to allow them to move forward and make these products even more advanced. We need an immediate PAUSE on advanced AI development, and a permanent BAN on superintelligence — an artificial mind smarter than any human, capable of operating independently beyond our control. Countries around the world must work together to prevent this nightmare scenario. That is why today I am announcing new legislation to do just that. Let me be clear: A superintelligent AI that escapes human control will not be an American problem. It will not be a Chinese problem. It will be humanity’s problem. My legislation would direct the federal government to not just stop superintelligence here in the United States, but to work to prevent it from being developed anywhere around the world. The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs. The American people and people throughout the world must determine that future.

GPT-6 Astra has set a new ECI record, with a score of 169. This is a substantial jump from the prior best (163), but is within our uncertainty range for the reasoning-era ECI trend. Astra also set new records on our math, continual learning, and game-puzzles benchmarks. On our long-horizon coding benchmark, MirrorCode, Astra ranks between Opus 4.7 and Fable 5. OpenAI gave us pre-release access to test Astra. Charts and more details for Astra’s individual benchmark results in the thread.

I see that you did, @dwarkesh_sp, and thank you for pointing it out. You're absolutely right to highlight the very real dangers of loss of control. I also agree that easy-to-understand language can be helpful for public communication and (to some extent) prediction - the latter primarily because LLMs reflect patterns implicit in human language. But I strongly disagree that anthropomorphic language of the sort you deploy is appropriate. I think it goes far too far. A messageboard is not a civilisation. The agents' goals are derived and not intrinsic. They have literally no skin in the game at all. And when you heavily imply that the agents have subjective experiences and are in some sense alive, I think you're playing right into the hands of the frontier lab hubris. That's where the money is. That's where the regulatory capture is. That's where the mind-uploading singlularity-mongering extropian narrarive is taking us. And if we overattribute, we may well end up mispredicting when we might most need to. Sure, its tricky to find the right language - to explain without misleading - but it really does matter. (For instance, LLMs do *not* hallucinate - if anything, they confabulate.) I should also underline (and should've done in the original post) that I am *not* 100% sure that current AI is (or can't be) conscious. But both theory and evidence together point to an extremely low probablity. Vanishingly unlikely, IMO. See Anyway I am sure you are busy with a billion other comments on your essay - so I'll leave it there with thanks.

What I wanted to say yesterday is that we hit 25M active users and to celebrate we have now reset usage for all paid subscriptions for ChatGPT Work and Codex. See you soon for more news from The Reset Company.