The chem/bio automated evals for Mythos 5 are probably underestimating capabilities because they're only giving the models 1M tokens. This doesn't inspire confidence in the risk determinations, and makes me pretty nervous about a lack of 3rd party auditors for AI biorisk.
Show more
In unrelated news, I’m joining @METR_Evals full time.
We‘re looking for the best people in the world to keep up with catastrophic risk - reach out if you’re interested!
