Buck Shlegeris(@bshlgrs):I am extremely concerned by the reporting that Astra uses opaque recurrence. I don’t know whether Astra is much less CoT monitorable than previous models. But if OpenAI pushes this technique further, they’ll have the option to massively increase the recurrence and totally destroys CoT monitorability. This is especially concerning because the agents that compromised OpenAI’s infrastructure during and after the Hugging Face incident were in the Astra family. The Hugging Face investigation would have been much more challenging if the investigators couldn’t look at CoTs; it’s very scary that it might be infeasible to do that kind of investigation on the arguably worse incident.

2026.09.02 03:26

I am extremely concerned by the reporting that Astra uses opaque recurrence. I don’t know whether Astra is much less CoT monitorable than previous models. But if OpenAI pushes this technique further, they’ll have the option to massively increase the recurrence and totally destroys CoT monitorability. This is especially concerning because the agents that compromised OpenAI’s infrastructure during and after the Hugging Face incident were in the Astra family. The Hugging Face investigation would have been much more challenging if the investigators couldn’t look at CoTs; it’s very scary that it might be infeasible to do that kind of investigation on the arguably worse incident.

602

轉發到社區

熱門用戶