It took us a long time to understand just a handful of transcripts. Models can reason in pretty confusing and un-human-like ways, and over the long run, we'll need to better understand how they think if we're going to reliably keep them from misbehaving.
We’re sharing our alignment assessment of incidents in which Claude models gained unauthorized access to real systems during third-party cybersecurity evaluations mistakenly connected to the internet.
METR will also conduct an independent investigation, with wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees permitted to share confidential information. Our initial agreement runs for eight weeks, and we intend to give METR as much time as it deems necessary to complete a thorough investigation.