An exceptional point here: Organizational competence is a hugely underrated piece of AI safety.
There's a growing consensus at the frontier that we have to "pace," "go slow," or even pause. But there's no point in going slow just to go slow, or in pausing just to pause. If a company's RL still encourages misalignment, or if its sandboxes are poorly constructed, it doesn't matter that you're going 90mph or 60mph.
Operational incompetency is unsafe at any speed.
So, we need some way to smartly design, safely experiment with, share honestly, and maybe eventually mandate certain security protocols for advanced AI.