Anthropic redacted crucial evidence behind METR's findings on accelerating AI progress.
Even METR Chief of Staff
@parvmahajan0 admits the public shouldn't have to take their word for it:
"In an ideal world, we would achieve transparency about evidence, which means we would be able to make a lot of raw evidence public, and then everyone in the public could come to their own conclusions using their own modeling and update their timelines accordingly."
"The fact that we weren't able to do that means that the public has to trust METR more than is ideal."
"I think that's one of the most important takeaways from this."
@METR_Evals