Ugh, yeah, sorry about this. METR makes a bunch of valid critiques but this is the one I feel worst about; I think not putting enough time into making sure the public version of the AI R&D section was as clear and transparent as possible was one of my biggest failures in the drafting process for this report. The badness of this language is particularly rough due to a second issue, which is that the paragraph right below this one in the public report makes it sound like the question of whether leading indicator acceleration is above or below 2 is the primary factor determining RSP threshold determination - which it's not, RSP talks about 2x in aggregate capabilities. The private report doesn't make an RSP threshold claim solely on the basis of this number, and says it thinks that overall progress had *not* accelerated by that much (bc AECI analysis at the time didn't look faster), but the public wording makes it sound (incorrectly) like these leading indicators were the primary basis on which to make a threshold determination, which is particularly unfortunate given that the phrasing downplays the value of this number and doesn't caveat its uncertainty enough. METR's suggested language of "We are uncertain how these leading indicators relate to downstream capabilities" would have made this clearer. To give some behind-the-scenes context on how the risk report sausage is made, here's my current best reconstruction of how it came to be that this paragraph ended up in the public report: (1) Around July 1, a very early draft of some of the material for this section had a lower central estimate for this rate of progress, and it included the text "At our central estimate (~[redacted]×/yr), [redacted] progress falls short of a doubling", with some later caveats about ways that more aggressive estimates would imply a doubling in the rate of progress on this metric. I think it was still close enough to 2 that it should have explicitly given a number, but it was more reasonable wording imo. (2) After updates to our data and methodology around the mid-July coverage date, we had a higher estimate that was quoted in the final report. The progress rate number was edited in place but the surrounding text wasn't; probably someone checked the number was less than 2x and didn't think hard about whether the surrounding language was tonally off. (3) Our main focus was on drafting the private section to be as accurate and clear as we could; producing a redacted public version was a much later process. We had a few different drafts of possible public versions floating around depending on where we landed on decisions about what to redact. On August 6, someone sketched a doc with a possible public version that included the text "Our leading indicators point to a picture of meaningful acceleration starting in early 2025, but falling short of the “double the pace of progress” threshold", basically just removing the numbers from an earlier sentence in the private report. (That drafting was by a different person than the author of the earlier text, and they hadn't run the numbers to see this would imply a ratio very close to 2. If they had, probably this would have said "have nearly doubled" or something.) (4) Very close to risk report publication, we decided on the version of the public draft which had the above language; I had previously left a comment on that draft noting that doubling the pace of progress was "not technically the v3.4 threshold, which is impossible to crisply refer to in a sentence of reasonable length", so I edited the final version to say "though by less than a factor of 2" so as not to misstate the RSP threshold. (And as mentioned above, RSP is about overall downstream capabilities anyway.) Anyway this language was accurate enough that Claude-powered factchecking didn't complain and it survived through publication from there. IIRC when making these edits I didn't go back to perform this calculation and wasn't tracking that the corresponding "less than 2" number was so close, although I had computed it at earlier points when working on the private report. Anyway, apologies that the public risk report underemphasized the uncertainty around these leading indicator acceleration rates and wasn't sufficiently clear about their connection to the RSP; this was not anyone's intention, and I'm glad METR was able to catch this for the public. I will try to make sure the next one does better.

@Plinz @JeffLadish Hard to settle a bet on ground truth, but I'm happy to bet you at 100:1 odds that we will never see credible evidence of this being a leadership plot (eg in future court depositions or whistleblowers or whatever).

A ranking of takes on embedded third party evaluation this past week, from worst to best: [contentless sneer against outgroup] METR bad because [huge Sankey diagram]. fact checking? timelines of relative investments that make causal sense? determining which numbers are large or small fractions of other numbers? sounds like some EA bullshit to me. just look at this diagram with all those curved lines, you can SEE the nest of snakes. METR bad because [an actual specific chain of actors who have some pairwise relationship to each other that ends with someone it would be bad for them to have strong COIs with, e.g. "METR once received funding from an org who once received funding from a person who once gave funding to a now-frontier AI company"]. It is beneath my dignity to explain how this actually affects the decisionmaking of METR; you should vaguely perform a mood affiliation and keep scrolling. Don't think too hard. guys guys guys you HAVE to embed my company/organization into the labs. I have been an unwavering supporter of external embedded auditors since 7 minutes ago when I saw this essay taking off. Look at this thing we published once that sort of looks like AI safety if you squint! Fear not, I can assure you that I have never done anything altruistic in my life and if I had I would have been ineffective at it. I observe that [politicized actor] has had [bad take]. Let me use this correct observation to score points for my side and politicize the situation further. === zero point: takes beyond this line are better than logging off === Dear [person with insane take], here is an earnest explanation of why you are wrong. METR is bad/problematic because [an actually reasonable concern, like greater cultural overlap with Anthropic than OAI or the pressure for individual employees to be on good enough terms with labs that they could later get hired], with no further suggestions for what to do about this issue. Tweets which simultaneously acknowledge that (1) more social independence from labs would be good and (2) almost everyone competent is socially connected to labs, even if they don't propose solutions. [literally any take that engages with object level assessment of AI companies done by a third party org like METR, SecureBio, Guidelight, Redwood, Nightingale, etc] === current discourse frontier: takes beyond this line are better than anyone has yet posted === I have [actually reasonable concern] with METR/Redwood/etc. To address this concern, we should do [concrete proposal that makes any sense and would solve the problem]. I am founding a new third party evaluation org / pivoting my existing org to do more third party evaluation. We're doing [thing which is both (1) not identical to METR (2) remotely useful for assessing AI companies]. Our initial work will be on [concrete specific thing that would help]. METR has problem X. We should instead use preexisting organization Y, which does not have problem X and has comparable expertise at assessing loss-of-control risks and misalignment incidents inside AI companies, for instance [past work they've done of similar quality]. (this would be an amazing take but it is impossible to post because no such org exists)

@jillgun I would burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive. I expect a great many of my colleagues across the industry would as well. I promise you, we are actually just fucking scared, it's not galaxy brained marketing.

@danpfeiffer I promise you that we are actually literally worried about world-ending consequences from this technology. Please find some people you trust who work at these companies and actually talk to them about their views.

Anthropic's second Risk Report is out! I'm pretty happy with a lot of the new things we landed in this one and think it's a big improvement over our first one in a bunch of ways. Among the novel features: * Coverage of internal models * A much more structured argument around alignment risk * Claude-powered review of the alignment section * Discussion of a wide variety of incidents, safety process failures, and ways Anthropic has fallen short (see 4.8.2, 6.5, and especially 5.2) * An attempt to get somewhat quantitative about threat modeling for CB-2 risk despite enormous uncertainty * Much more detailed discussion of jailbreak risks * Two different cases of increasing assessed risk levels from "very low" to "low" on the basis of general increased uncertainty given worries about the overall degree of process assurance * Early notes on some cool new model organism work that informs the alignment assessment and more! Excited to hear thoughts on how the next one could be better.

+1. I think working at METR would have been better than the frontier AI company work I was doing a year ago. The work I'm doing now is a sufficiently good fit for me in particular that I thiiiink it looks higher impact, but I think it's plausible I'm still making a mistake!

In light of recent discourse around AI company employees' willingness to express dissent with their employer, I wanted to signal-boost this reply about a particular dynamic I experience fairly often. I suspect stories like this one are generally underrated in folks' models.

(epistemic status: not trying to make sweeping claims about the nature of epistemic pressures within the company or saying this is the whole story, just anecdata about a failure mode I've personally experienced that I think is rarely discussed in these kinds of situations) One kind of awkward dynamic I've noticed in myself wrt public negative commentary on Anthropic comms is that in some ways it's hard because of high _internal_ reception for disagreement? Like, suppose that some senior person at Anthropic says a thing publicly and I think they had some bad takes or misleading narratives or whatever. If I oriented to them as a shadowy nefarious Authority who might punish me for speaking up, then whatever, screw them, that's an easy integrity call. But if they're just like some earnest nice person who I think is mistaken or wasn't particularly thoughtful in thinking about some consideration or whatever, then it feels super rude to go disparaging them on twitter when I haven't even brought it up to them directly and I know they would thoughtfully listen to the feedback and wouldn't dream of acting in a retaliatory way. But then, like, aw geez this senior person is extremely busy and this was kind of a minor point and I'd like to reword my totally unfiltered thoughts to be a little more tactful and charitable and avoid coming across like I think this was some kind of malicious act on their part when I really don't, and if I do send that DM or post in that thread then I should budget some time to reply to follow up conversations as a result, and it's probably worth checking with someone else internally to sanity check I'm not missing something silly that would make this a total waste of their time, and so the prospect of doing THAT is kind of friction-y and easy to put off when there's lots of other important work to do, and and and... So as a result it's not like I actually feel like My Opinions Are Suppressed but it does end up being the case that a smaller fraction of random negative takes I have make it out into the world because it feels like a lot of work to do well and it's easier to procrastinate them. I don't really know what to do about this kind of situation, other than to be the sort of person who puts really quite a lot of time into talking over this sort of thing with people anyway.