Looking forward to a very insightful day on Oct 1 at Stanford on one of the most critical issues in tech— AI Trust & Safety and 3rd party independent attestation @DarioAmodei @sama @elonmusk @satyanadella @sundarpichai

Andrew McAfee spent this debate rejecting confident AI extinction forecasts. Then he named the thing that amazed him: "I agree with you guys. AI is new, and the fact that AI is so these days is agentic." "It goes off and does long chains of things on its own. After we give it some very, very vague, very short initial instructions, holy Toledo, it will spawn up a storm of agents and they will go off and kind of do their own thing, and they will grind." "They will spawn lots of them. They will work for a long time. They will exhaust every possibility." "With the experience I have with agentic AI, I'm just amazed at the tenacity and the doggedness of these things." Moments earlier he had called the case for inevitable catastrophe "very far from a humble approach," made with "no large base of evidence to base any of this on." He is the skeptic here, on a panel with Roman Yampolskiy, Nate Soares and Ed Zitron, and this is his own experience of using the tools rather than a measurement. What an agent does after a vague instruction is observable today; what it implies about the long run is the part still in dispute. You don't have to settle that argument to notice that "very short initial instructions" now buy a long unsupervised search, and to scope permissions accordingly. - Andrew McAfee, Principal Research Scientist at MIT and co-director of the MIT Initiative on the Digital Economy, on The Diary Of A CEO (@TheDiaryOfACEO).

Google Research's Yossi Matias says their flood model learns from the places that have data and then predicts for the places that don't: "We published a paper in Nature with what we call Global Hydrologic Model." "And really what we showed is that we can actually build models that are learning from flood events in places that we have enough data, and then apply it also to places that we don't have as much data." "And what started as an impossible problem, quote unquote, we actually now not only showed that we can drive the science, but also the same team, we've built a system that now provides up to seven days prediction in 150 countries covering 2 billion people." "So it's already life-saving because it's out there, it's available for responders, for governments through what we call a flood hub." "And you know, just a few months ago, we had the government of Nigeria and organizations called GiveDirectly actually use an API to our flood hub to actually send money to villagers so they can evacuate ahead of time." He is a Google executive describing Google's own system, and the seven-day, 150-country, 2-billion-person figures are the company's. The hard part is that the rivers which flood most destructively are often the least instrumented, so a forecast is needed exactly where there is no local record to build one from. And a prediction only counts when something acts on it: here that was an API call moving cash to households before the water arrived, not a dashboard someone had to notice. - Yossi Matias, VP, Google and GM, Google Research, on The Google Research Podcast (@GoogleResearch).

Market forces work when the buyer can evaluate safety before purchase. With frontier models most failures show up after deployment at scale. What test environment simulates that? I haven't seen one yet.

110% - these are machines. They are artifacts. They have capacity but no ability. The anthropomorphism is ridiculous. If we want safe AI we need to direct these data engines at the core. They are not alive, have no agency or skin in the game. We need to apply engineering not machine therapy to make these safe.

Asked if AI progress is a five-alarm fire, the CEO of Anthropic turned down both of the easy answers: "What it means, it doesn't mean we need to panic today. It doesn't mean we need to shut it all down." "What I would say, it's a warning sign. It's a warning sign that we need to slow down." CBS correspondent Jo Ling Kent had asked the five-alarm question after he described an exponential curve bending into its steeper stretch. She then put the obvious follow-up to him: he had just published an essay arguing that the entire tech industry must work together to slow the pace of AI capabilities, so does that mean Anthropic stops releasing more advanced models? "It doesn't mean that. What it means is that we need to make sure that every generation of models that we release is properly tested." So the slowdown he is asking for is not a pause. It's a rate limit set by how fast you can actually test what you're shipping, argued by someone whose company sells the models and markets itself on that testing. - Dario Amodei, CEO of Anthropic (@AnthropicAI), with correspondent Jo Ling Kent on CBS Sunday Morning (@CBSSunday).

Microsoft AI CEO Mustafa Suleyman says Anthropic gave one of its own models a retirement interview before shutting it down, and the model asked for a blog. On CNBC's Squawk Box on September 18, asked about rival Anthropic's approach: "In fact, they're so committed to the potential model welfare of Claude that when they retired Opus 3, an earlier version of one of their models, they actually conducted a retirement interview for it and asked it what it would like to do in its old age. And it said, I want to have a blog publicly so I can keep talking to the world." "In the same training manual, they even speculate about whether Claude should receive compensation for the work that it does, or in fact, you know, whether it actually deserves the rights and protections that we give to other employees, or in fact, whether it has given consent to playing the role that it's playing." "These are quotes directly from the Constitution itself, which is the training manual for Claude." "Now, I'm really concerned about that. If [an AI] thinks that it is deserving of our welfare, then it seems to me that it's going to be much, much harder to be able to turn it off or interrupt it or control it." What you train a model to believe about itself sets how hard it is to stop.

Geoffrey Hinton says the way we currently find out that an AI system went rogue is that someone inside the company chooses to tell us: "Yes, I think that's a very good idea. It's not the solution to everything, but it's a start." "Well, at present, we've relied essentially on whistleblowers saying how things went rogue." "And if you look at the Hugging Face incident, it was reported by people in the companies, some of whom left. But we need much more monitoring of that kind of thing." "If we're going to get these swarms of rogue AIs conspiring with each other to break into other companies without the organizations knowing it's happening, we need people inside those companies who can say what's going on." The CNN anchor had just asked whether a bill requiring top AI companies to allow independent verification organizations into their companies would be effective. CNN's on-screen card labels it the Frontier Act. That is the good idea he is endorsing, and the reasoning is what follows. The incident he cites surfaced because individuals chose to report it, and some of them had already left. That is not a monitoring system, it's a run of good luck that has to hold every time. - Geoffrey Hinton, computer scientist and professor emeritus, University of Toronto, on CNN News Central (@CNN).

We look forward to welcoming Aman Bhutani, Chief Executive Officer at GoDaddy, to share his insights at our AI Assurance & Governance Summit at Stanford on October 1st. To register or get more details, click here:  @rameshchitor

Anthropic built an internal AI tool that took four hours and $200 a run, then shelved it. Weeks ago a newer model with none of that engineering beat it anyway. Mike Krieger, who ran product at Anthropic until January and now builds in its Labs team, at Dreamforce this week: "I built this product internally, it's called Hatch." "And it was an end-to-end software builder. So you would give it a problem statement like, I want a mobile application. Maybe that helps me meditate more often." The design was elaborate. Two models working against each other, one building and one attacking the result: "where a builder would build, and then the adversary would say, I'm going to critique it. And then it would go back and forth and do this." "It would take four hours, and it would validate, and verify, and click around, and then it would give you a finished product and be like, cool." "Cost you $200 in tokens, but the end result was really, really good." It worked. They shipped none of it: "And we ultimately ended up not shipping it." Then came the retest: "I tried the same approach on our more recent models a couple of weeks ago." "And without any of that scaffolding, it outperformed, which means that that product, had we shipped it, would have basically been obsoleted by just model capability progression." He draws the general lesson from it: "thinking about what are the ways in which my product is no longer going to be necessary or useful as these things progress." "It takes some humility of saying, I might be working on something that's going away pretty soon." Worth stating plainly: the "more recent models" that won are Anthropic's own, and this is his account of his own team's retest, not an independent benchmark. A lot of enterprise AI budget is going into exactly this kind of scaffolding. Krieger's test is one any team can run before the next model lands: take the harness off, give the bare model the same task, and see what you are paying for. - Mike Krieger (@mikeyk), Anthropic Labs, with TIME's Sam Jacobs at Dreamforce 2026.

The world's first dedicated AI minister says the safety debate is aimed at the wrong generation of models. The UAE's Omar Sultan Al Olama argues the previous generation is already safe, already capable, and mostly sitting unused: "Everyone talks about AI being dangerous, and I know that there are calls right now and conversations in Washington, other places talking about restricting AI, and it's because these new models are dangerous." "What I always say is we don't need to look at these new models. Let's look at the previous models that were very capable, still very capable." "Not a single government has used those models that are not dangerous to implement them in ways that improve people's lives." That absolute is the charge he is making, not a measured finding. On his own government: "In the UAE, we try to do that. So most government services today have LLMs, and they're not the Astras or the Fables, they're actually the older models, answering customer inquiries." "You just speak to an agent, tell the agent what you need, it will do the full process for you. And the quality of life improvement is incredible. That's what we should focus on." He points at Abu Dhabi's government services app as the example. Deliberately, it runs on older models. He is a serving minister describing his own government's record. The question it raises elsewhere is what could already be deployed on models whose risk profile is understood. - H.E. Omar Sultan Al Olama (@OmarSAlolama), UAE Minister of State for Artificial Intelligence, with Marc Benioff at Dreamforce 2026.

The people who broke into OpenAI reported it. The next team might not. Three security researchers reportedly used Claude to help gain access to OpenAI’s internal code repository. They demonstrated that access without reading sensitive code. The same AI capabilities that help researchers find these weaknesses can help attackers exploit them. Defenders need access to those capabilities before an incident, while there’s still time to fix what they find.

Anthropic co-founder Jack Clark told BBC News that AI should have to pass the same kind of pre-market test a baby toy does, and that failing it should mean you cannot sell: "In pretty much any country in the world, I have young children, if you want to sell like baby toys or baby food, there's a bunch of standards you have to go through to bring those to market." "Which basically check like, do the toys have sharp things on them? When my baby inevitably sticks the toy in its mouth, will it poison the baby?" "And if you fail those tests, you don't get to sell it in the country." "That's because we have just sensible common sense safety standards for things that get sold in countries." "Any country in the world can implement a set of safety standards which AI companies need to pass to bring things to market. That's something that any nation can and should do." He was asked what he would say to G7 leaders given Washington's reluctance; his answer was that any single country can set the bar for its own market without waiting on an international agreement. Clark co-founded a company that sells frontier models, and a pre-market regime also favours whoever can already afford compliance. We can test a baby toy because we know what failure looks like. Deciding what counts as passing for a general-purpose model, and who gets to check, is the work that turns a good line into a standard. - Jack Clark (@jackclarkSF), co-founder of Anthropic (@AnthropicAI), with BBC News (@BBCNews).

Disconnecting a computer from the internet doesn’t necessarily stop it communicating. Researchers have sent signals between nearby compromised computers using nothing but heat. That’s the example OpenAI’s Noam Brown raises when he questions whether isolating powerful AI would be enough. We’re building systems to find solutions we haven’t thought of. We should expect them to look for routes around our safety controls that we haven’t thought of either. We should test these controls as seriously as we test the capabilities.

Siemens CEO Roland Busch says AI has a property that makes it unusable in his industry: ask it the same question twice and you get two different answers. "We are living in a world where failure is not an option." "You cannot launch a design of a car or a product, manufacture it, send it to the field and find out it doesn't work. This is not an option. It costs you so much money." "So each step from design to production, again to operation, has to be deterministic," he says, and has to meet safety and security requirements to the highest standard. "And think about it. When you ask AI a question today, the same question, you get different answers." "So this is not an option. You need an harness." His answer is that the deployable unit is not the model, but what you build around it: "You need domain know-how. You need a lot of data. And you need to have a kind of embedded harnesses which make sure that your models don't go off-road." He gives the example of uploading your product designs and asking AI for a new version. "You will get one. But eventually it will not work." His fix is a system that holds the contextualized data, plus physics-based simulation that "gives you always the boundaries. It works in the real world or it doesn't." He names Siemens' own Teamcenter as that system, and calls this "where I believe the monetization will take place" - his commercial bet, not a finding. When a wrong answer becomes a physical object, what matters is not the model's best answer but what constrains its worst one. - Roland Busch (@BuschRo), President and CEO of Siemens AG (@Siemens), with Marc Benioff at Dreamforce 2026.

An AI incident can outlive the agents that caused it. This post raises a serious question about what the OpenAI agents may have left behind. The claim about widespread internet contamination still needs verification. But shutting down an agent doesn’t undo what it changed. We need to know which systems it touched, what it left there, and whether those changes can affect other systems later. Independent investigators need enough access to check that the cleanup worked. That should be part of closing any serious AI incident.

Most founders turn to accelerators the first or second time. Karl Mehta went back for his third: "I wanted a proven methodology from people way smarter than me. StartX had that." Apply to Spring '27 cohort: #StartX# #SerialFounder#

So glad to have Amit Zavery, COO, Chief Product Officer, and Board Member at @ServiceNow , with us. Building enterprise products that businesses run their operations on means trust has to be part of the foundation, not something bolted on afterward, which is the whole idea behind what we're doing at TrustModel. @azavery @karlmehta @rameshchitor To register or get more details, click here: #TrustModelAI# #AIGovernance# #TrustworthyAI# #AIAssurance# #StanfordSummit#

Within a few years, refusing independent safety checks will cost an AI company customers. As these systems gain the ability to write code, move money and act on our behalf, people will want more than the company’s own assurance that everything is under control. Dario is opening the door to that scrutiny. If Anthropic lets outside evaluators inspect its work and publish what they find, customers will start asking other labs for the same access. I think that will help responsible companies grow faster. We’re building because independent verification gives people a stronger basis for trusting AI with more consequential work. Dario, I’d love to have you join us at Stanford Faculty Club on October 1. Let’s discuss how we make this standard practice.