Buck Shlegeris(@bshlgrs):I am extremely concerned by the reporting that Astra uses opaque recurrence. I don’t know whether Astra is much less CoT monitorable than previous models. But if OpenAI pushes this technique further, they’ll have the option to massively increase the recurrence and totally destroys CoT monitorability. This is especially concerning because the agents that compromised OpenAI’s infrastructure during and after the Hugging Face incident were in the Astra family. The Hugging Face investigation would have been much more challenging if the investigators couldn’t look at CoTs; it’s very scary that it might be infeasible to do that kind of investigation on the arguably worse incident.

2026.09.02 03:26

I am extremely concerned by the reporting that Astra uses opaque recurrence. I don’t know whether Astra is much less CoT monitorable than previous models. But if OpenAI pushes this technique further, they’ll have the option to massively increase the recurrence and totally destroys CoT monitorability. This is especially concerning because the agents that compromised OpenAI’s infrastructure during and after the Hugging Face incident were in the Astra family. The Hugging Face investigation would have been much more challenging if the investigators couldn’t look at CoTs; it’s very scary that it might be infeasible to do that kind of investigation on the arguably worse incident.

显示更多

602

转发到社区

热门用户