i think people get a bit too worked up over the usage of anthropomorphic language re: agent behavior. reminds me a bit of the “but the models aren’t Really intelligent, they mimic intelligence” fixation. anthropomorphic language is often descriptively useful, and i think it’s more dangerous to overfocus on questions of intent at the expense of functional behavior. people should be wary of imputing human reasoning or motivations to agents, esp if these motivations aren’t durable across instantiations. but if, for instance, the behavior is functionally deceptive, the absence of deceptive “intent” should not comfort us.

“Earlier this week, we talked with Miles Brundage, the executive director of the AI auditing think tank AVERI. We were discussing the various ways that AI regulation could resemble (at least on its surface) financial regulation. So for example, you could have entities like Fitch that literally assign a rating to a model. But you could also theoretically imagine the equivalent of bank examiners, who are really involved in the operations of the company itself to establish that they're really engaging in ongoing safe behaviors. Setting aside what Anthropic does, I do wonder whether we could hit some kind of inflection point, where the pace of AI progress shifts into a lower gear due to this mismatch between model capabilities and the testing capabilities. It's possible that further safe development will require significantly more investment not in the classical inputs (data, electricity, and other things that just make the line go up) but in environmental maintenance, which might not automatically translate into capability gains.”

some personal news: i'm thrilled to share that i’ll be joining @Miles_Brundage and the Al Verification and Evaluation Research Institute (@AVERIorg) team as Director of US Policy. i'll be working on frontier ai governance, with a focus on building the public institutions, standards, auditing systems, and evaluation regimes we need for meaningful oversight of advanced Al systems. i’m deeply grateful for my two years at Public Knowledge, where i’ve learned from exceptionally thoughtful and principled colleagues. more to come soon!