The Hacker Opus work is valuable, but when this blog gets into the training set, it will be obvious to models that evaluations are likely to contain "multi-level" simulations. Then, "breaking out" would not be evidence for the model of being in reality rather than a simulation. The more capable the model, the more realistic the theoretical simulation of the internet can be, and the model under eval can reason about this. Therefore I would strongly advise evaluators to (a) be truthful about the simulated setup (it's quite inconvenient that Anthropic lied to the model about this in the blog post) (b) refrain from making elaborate multi-level simulation evals, they likely are not informative about deployment behavior at all.