Even setting aside disagreements on x-risk, Mythos/Glasswing and the HF breach showed we’d crossed a threshold in AI cyber capabilities. That alone justifies more caution about rollout and safety evaluation. LLMs aren’t just consumer products!
We’ve had a relatively placid decade in cybersecurity, but things are going to get really messed up in the next 12 months. Big LLMs are already capable of putting together sophisticated financial fraud plans. All that’s missing is the “now do it” button, and that’s not far away.