I'm just speculating like everyone else (no juicy secrets about this shared with me) but here's my guess about what is going on right now with OpenAI and the three fired employees:
The negotiations between OpenAI and METR are ongoing. OpenAI has said they want embedded third parties, METR has proven their worth with the previous engagements, so METR is an obvious choice. But my guess is that METR is asking for more this time; probably they want more access, more time on site, less narrow scope, the right to comment on redactions, etc.
(Compare with the terms of their previous engagements: In this one a third of their report was redacted and they weren’t allowed to say why, and in the Hugging Face investigation they only had 6 days and were only allowed to investigate the precursor swarms nor the subsequent more severe compromise of OpenAI infrastructure.
Various researchers at OpenAI probably support METR in this (including, I’d imagine, the fired three employees? Tomek was the point person on the hugging face investigation), but also various higher-ups at OpenAI and legal types probably oppose and want more restricted access and scope – basically they want to work with an org that’ll go easy on them and not give them trouble.
OpenAI holds most of the cards here, since this whole thing is voluntary and not legally required, and they can threaten to go with Accenture or HuggingFace or whatever instead. Anyhow to improve their bargaining position and/or minimize the blowback if they decide to say no to METR and go with someone more pliable, it helps if they can discredit METR.
Hence (I suspect) why they are going after my friend Ryan Greenblatt (who works at METR) now. They don't need to present actual evidence, they just need to cast vague aspersions, and many employees will give them the benefit of the doubt. If and when they say "We chose to work with a different third party" they'll face lower risk of employee discontent than if METR's reputation was unsullied.
This whole mess, by the way, was predictable. *Of course* there's a big political fight about who should be trusted to do this and who has relevant expertise and independence etc. and *of course* the companies are trying to make things shake out in a way that favors them.
It's part of why I've been advocating for *actually pacing the frontier* instead of just doing third party assessments. "We allocate at most 20% of our compute to AI R&D" is a lot easier to verify than "we are being safe and responsible" and depends a lot less on subjective judgment and technical alignment expertise -- AND, if you do it, people will be able to see the slowdown in the trends of agentic coding performance in the frontier companies and see laggard companies start to catch up. Harder for it to end up just being safetywashing / industry capture.
Show more