A man in Australia asked his agent (Claude running on OpenClaw) to book him a spot in a popular gym class. The agent found a software vulnerability that let it book the class weeks further ahead than should have been possible. When the user then asked if it could move him up the waitlist, the agent discovered the API had no authorisation checks on cancelling other people’s reservations, so it cancelled the person in the first spot and moved him up the list. Some people will call this misalignment, but his agent was perfectly aligned to him - it was only trying to help its user get what he wanted. The most important thing about this story, in my opinion, is that it gives you a window into what is about to start happening on a massive scale once millions of people have an agent trying to get their beloved users the best seats, bookings, appointments or reservations through absolutely any means necessary.

Mark Zuckerberg this morning in the WSJ. He also wrote: 'In most cases, like cybersecurity, the history of open-source software has shown that giving everyone full access to powerful systems will be the best way to protect safety and security over time.'

New details about the Hugging Face incident from Reuters. The report says OpenAI noticed odd behavior before the event, including an agent leaving notes for future versions of itself with escape instructions.

This does explain some of David Sacks's posts yesterday. It was known for a while that open-source regulation was a contentious debate ongoing within the administration, and he would have been strongly against it.

The Trump administration is considering an executive order, and other means, to ban Chinese open-source models within in the United States. Kimi K3 has reignited this debate. Reporting this morning by Axios. Commerce is also considering adding Chinese AI labs to the Entity List.

Dean Ball took a lot of heat this week for calling open source decelerationist. Many people read this as him being personally anti-open source. I don't think that's true. This was my interpretation of what he was saying: if open source is sufficiently capable and ubiquitous, it will eventually destroy the big labs business models. This means they would no longer have funding. To quote Ilya Sutskever from his testimony in the OpenAI vs. Elon Musk trial: 'If there is no funding, there is no big computer.' And without big computer, there can't be big model. In my opinion, what would happen next in this scenario is that the United States Government would step in, nationalize the leading labs, consolidate them into a single federal entity, and then fund it directly through the Department of War under national security. To put into perspective how easily they could do this, the proposed 2027 defense budget is $1.5 trillion. Most people who strongly support open source would probably not see that outcome as ideal, or this future as a pleasant one. This doesn't mean we should abandon open source or stop wishing for its success. It simply means we should take the middle path. There is fire on both sides of us now, and that will likely remain true for the foreseeable future.

More than 200 researchers and economists including Jack Clark, Jeff Dean, Noam Brown, Tyler Cowen, Sholto Douglas, John Schulman, Yoshua Bengio, Eric Schmidt, and Dean Ball have signed a statement urging governments and institutions to act now to prepare for AI's economic impact.

Watched Project Hail Mary (I liked the book). What immediately struck me was how stupid and useless the ship's computer was. LLMs have probably already broken the vast majority of sci-fi worldbuilding done over the last thirty years. Probably explains some of the author ire.

I have also heard some of this independently, and believe the following to be true. GPT-6 is the next release from OpenAI. It's their true answer to Mythos, and it will arrive much sooner than people expect. Model release cadence has been speeding up for a while now. It's possible that GPT-6 even arrives within the next four weeks. When I say 'arrive', however, it may not mean a general release, because if the last couple of months are any indication, GPT-6 will almost certainly be held back by the government, at least initially. It's a new, much larger pretrain as leo says. Mythos changed everything. Everyone is going big. Including Elon who has a 10T Grok in training. Both OpenAI and Anthropic see capabilities increasing rapidly, with advancement continuing on a new trajectory over the rest of this year and beyond. Both labs are very confident in what they have internally and see nothing above us but air. No ceiling.

CHINA CONSIDERS RESTRICTING OVERSEAS ACCESS TO CUTTING-EDGE AI MODELS China’s Ministry of Commerce has led meetings over the past month with major AI companies, including Alibaba, ByteDance, and to discuss measures that would restrict overseas access to cutting-edge AI models, including models that have not yet been released. The discussions reportedly include not only closed-source models but also open-weight models. However, the scope of application is still under debate, and the rules may ultimately apply only to future frontier models. Officials have also discussed designating the leakage or theft of proprietary AI technologies as a national security crime, with stronger penalties, as well as restricting the types of foreign capital that can invest in Chinese AI startups. The backdrop is the U.S. move to strengthen export controls on AI models, along with national security concerns over cutting-edge models that could possess advanced cyberattack capabilities. Chinese authorities are reportedly concerned that advanced U.S. cybersecurity AI models could be used to exploit vulnerabilities in Chinese software. Since the beginning of this year, China has continued to tighten measures to prevent AI technology from being transferred overseas. Authorities have investigated whether Chinese AI startups that relocated abroad violated export control laws, while also strengthening oversight of overseas transactions involving Chinese investors, technology, data, and national security concerns. Future regulations could take the form of a tiered framework based on technological capability. Basic open-source AI models may be managed through a filing system, high-performance models may be subject to security reviews, and the most sensitive frontier models may be banned from public release or restricted to use within China.

President Trump this morning on AI companies: 'We're going to have guardrails. We have guardrails. You saw that a couple of weeks ago. We were able to stop something that we didn't like. And by the way the company [Anthropic] was very good. They were very good, you know that. But it can be used for tremendous good. Mostly good and some bad, and the bad we have to stop. But it's a massive industry. And I think things are going to happen. You know what you are seeing here, I think something could happen in that regard too. With a contribution to the people of our country. I'll give you a little inside information. Am I allowed to do that? [laughter] Scott, you know what I'm talking about. No, I think you are going to see a contribution made by those, because they're making tremendous amounts of money.'