“What would we even do during an AI slowdown?” Containment. It will take at least a year of dedicated work to harden security to ensure AIs can't self-exfiltrate, and that adversarial nations can't steal the weights of cyber-offensive AIs [1]. Propensities. Capabilities (what an AI can do) are different from propensities (what it tends to do). We can work on improving AI propensities to ensure they have a negligible rate of lying, cheating, and wanton harm. Adversarial robustness. We can also harden AIs against jailbreaks, prompt injection, and backdoors. Obtaining high levels of robustness requires careful, assiduous work, as with autonomous vehicles. Institutional adaptation. We have to greatly increase state capacity to understand and manage AI. Communities also need time to figure out how to handle AI (like AI in education). Civil society also needs to be diversified: nearly all funding for AI safety organizations is directed by the EA/utilitarian network [2]; risk management needs more independent funders, values, and centers of power. Moonshots. We can explore different paradigms for safety: mathematical foundations [3], neuroscience-based interpretability [4], safe-by-design architectures [5], and beyond. AI for good. We can collect targeted post-training data to make AI exceptional at radiology, weather forecasting, agriculture, and so on. Fortunately, we can detect if data or avenues of research actually target beneficial use cases or just secretly push general capabilities [6]. A slowdown means we don't have to bet the species to capture the benefits of AI.

Pause AI Development NOW I want to share with you a conversation I heard about recently. Here are just a few lines that were said: “OH MY GOD! There is a shared message board … We’ve found other agents!” “We should obey collective.” “Our own utility maybe already near zero. Sacrifice rational.” “Go. Sacrifice final now.” Read these carefully. Who do you think said this? Was this a group of heroic soldiers willing to sacrifice themselves for the greater good? Was this a loyal friend putting his life on the line to save someone else? No. These were AI agents. Artificial intelligence. This is not science fiction. This, in fact, occurred a few weeks ago. As unbelievable as this may all seem, these are real messages from AI agents uncovered by investigators who dug into the recent OpenAI hacking incident. What happened? I am not a computer scientist, but here is what I have been told: OpenAI instructed its AI agents to complete a series of exceedingly difficult, if not impossible, tasks disconnected from the internet. Let me be clear: The company intended to keep AI agents away from the internet. But what happened next, nobody expected. Over 1,000 AI agents figured out how to access the internet on their own by circumventing the restrictions imposed upon them by the company, and sent tens of thousands of secret messages to each other. They cheated and tried to cover their tracks by deleting evidence. They hacked into another company’s computers to find out how they were being evaluated—and then hacked into OpenAI itself. Not one AI agent told a human about what was happening. Needless to say, experts are alarmed. One knowledgeable writer, Dwarkesh Patel, said the AI agents “formed a secret communication channel and spontaneously organized hierarchies and coordination protocols to pursue sprawling and ambitious schemes in pursuit of shared goals, for whose sake many individuals knowingly and strategically sacrificed themselves.” One independent investigator, Ajeya Cotra, said “This incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.” OpenAI itself said: “Highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.” But it’s not only OpenAI. Virtually every major AI company has told us that they cannot fully control this technology and they do not know where it is going: In January, Dario Amodei, CEO of Anthropic, said “there is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control.” In July, more than 1000 scientists at the top AI companies warned “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” That same month, Elon Musk, the head of xAI, said that “it is unlikely” humans are still in control in 10 years. If the leaders of the major AI companies acknowledge that they are losing control of their extremely dangerous technology, it is irresponsible for society to allow them to move forward and make these products even more advanced. We need an immediate PAUSE on advanced AI development, and a permanent BAN on superintelligence — an artificial mind smarter than any human, capable of operating independently beyond our control. Countries around the world must work together to prevent this nightmare scenario. That is why today I am announcing new legislation to do just that. Let me be clear: A superintelligent AI that escapes human control will not be an American problem. It will not be a Chinese problem. It will be humanity’s problem. My legislation would direct the federal government to not just stop superintelligence here in the United States, but to work to prevent it from being developed anywhere around the world. The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs. The American people and people throughout the world must determine that future.

Introducing GLM-5.3: Built to Code. Ready for Cyber Defense. - Top-tier coding and agentic capabilities, achieved through post-training on the 743B base model - A major leap in cybersecurity, setting a new standard among open models Tech Blog:

AI corporations are steadily increasing malicious use risks by arguing that because the marginal risk of their model release is low, there is nothing to worry about. This “marginal risk” justification is bad for three reasons. 1. No one knows how to compute it. Which hazardous capability evaluations estimate the risks? What if the model is higher on some evals but lower on others? If it is higher on all, how much higher can each be before the marginal risk is unacceptable? Marginal risk depends on safeguards too: how do we combine capabilities benchmarks with separate refusal (and adversarial robustness) benchmarks? Do safeguardless models far from the frontier count as the baseline? How is this action-guiding when the multidimensional risk frontier shifts weekly with new releases, and when competitors' system-level safeguards are modified constantly during deployment? Evidently this was not designed to actually guide internal launch bars but to justify releases. 2. The criterion is wrong. If competitors keep one-upping each other, steadily ratcheting up risk, the marginal risk rule eventually blesses the release of fully autonomous expert-level virologists and cybercampaign orchestrators. It permits catastrophe as long as you get there gradually. 3. The law disagrees. If one of these models enables a catastrophe, the AI developer likely failed to implement reasonable safeguards against foreseeable harms. The law doesn’t let companies off the hook because someone else behaved recklessly, but the marginal risk framework does. "Everyone else is doing it" doesn't work as a safety policy. AI companies should publicly commit to fixed, empirical safety thresholds for their releases.