The Hugging Face hack happened at the lab with the best logging and, relative to other labs, solid monitors and action-gating.
This goes to show:
1) In absolute terms, safety & security at all labs is inadequate
2) "Small details" like monitoring all models during training really matter. In some ways you're only as safe as your weakest link.
3) It's hard for scorecards like this to capture fuzzy things like "do you have a safety culture / if your AIs hacked into your lab, would you look at the logs or just shrug and keep training?"
Our team spent hundreds of hours reading documents so you don’t have to, all to answer: How good are AI companies’ safety practices?
I’m really proud of what we’ve built: It’s Guidelight’s first scorecard, on whether companies can control their AIs, and it's launching today.