Ugh, yeah, sorry about this. METR makes a bunch of valid critiques but this is the one I feel worst about; I think not putting enough time into making sure the public version of the AI R&D section was as clear and transparent as possible was one of my biggest failures in the drafting process for this report. The badness of this language is particularly rough due to a second issue, which is that the paragraph right below this one in the public report makes it sound like the question of whether leading indicator acceleration is above or below 2 is the primary factor determining RSP threshold determination - which it's not, RSP talks about 2x in aggregate capabilities. The private report doesn't make an RSP threshold claim solely on the basis of this number, and says it thinks that overall progress had *not* accelerated by that much (bc AECI analysis at the time didn't look faster), but the public wording makes it sound (incorrectly) like these leading indicators were the primary basis on which to make a threshold determination, which is particularly unfortunate given that the phrasing downplays the value of this number and doesn't caveat its uncertainty enough. METR's suggested language of "We are uncertain how these leading indicators relate to downstream capabilities" would have made this clearer. To give some behind-the-scenes context on how the risk report sausage is made, here's my current best reconstruction of how it came to be that this paragraph ended up in the public report: (1) Around July 1, a very early draft of some of the material for this section had a lower central estimate for this rate of progress, and it included the text "At our central estimate (~[redacted]×/yr), [redacted] progress falls short of a doubling", with some later caveats about ways that more aggressive estimates would imply a doubling in the rate of progress on this metric. I think it was still close enough to 2 that it should have explicitly given a number, but it was more reasonable wording imo. (2) After updates to our data and methodology around the mid-July coverage date, we had a higher estimate that was quoted in the final report. The progress rate number was edited in place but the surrounding text wasn't; probably someone checked the number was less than 2x and didn't think hard about whether the surrounding language was tonally off. (3) Our main focus was on drafting the private section to be as accurate and clear as we could; producing a redacted public version was a much later process. We had a few different drafts of possible public versions floating around depending on where we landed on decisions about what to redact. On August 6, someone sketched a doc with a possible public version that included the text "Our leading indicators point to a picture of meaningful acceleration starting in early 2025, but falling short of the “double the pace of progress” threshold", basically just removing the numbers from an earlier sentence in the private report. (That drafting was by a different person than the author of the earlier text, and they hadn't run the numbers to see this would imply a ratio very close to 2. If they had, probably this would have said "have nearly doubled" or something.) (4) Very close to risk report publication, we decided on the version of the public draft which had the above language; I had previously left a comment on that draft noting that doubling the pace of progress was "not technically the v3.4 threshold, which is impossible to crisply refer to in a sentence of reasonable length", so I edited the final version to say "though by less than a factor of 2" so as not to misstate the RSP threshold. (And as mentioned above, RSP is about overall downstream capabilities anyway.) Anyway this language was accurate enough that Claude-powered factchecking didn't complain and it survived through publication from there. IIRC when making these edits I didn't go back to perform this calculation and wasn't tracking that the corresponding "less than 2" number was so close, although I had computed it at earlier points when working on the private report. Anyway, apologies that the public risk report underemphasized the uncertainty around these leading indicator acceleration rates and wasn't sufficiently clear about their connection to the RSP; this was not anyone's intention, and I'm glad METR was able to catch this for the public. I will try to make sure the next one does better.