SITUATION EXPLAINED: Why one global AGI project would be more dangerous, not less.
• MAGIC was proposed in 2023 by Jason Hausenloy, Andrea Miotti, and Claire Dennis. One institution permitted to build advanced AI, enforced by a global moratorium on everyone else
• If you get the alignment wrong, there's no second attempt and nobody else to catch it
• If one institution builds frontier AI, the rest of society can't prepare, because nobody else has seen what it can do
• And if one body builds the only frontier AI, the question becomes how you align the people running it
• A global pause after GPT-4 in 2023 would have cost trillions in market value and left society less able to defend itself
@theojaffee: "AI risk comes from the differential between the most powerful models and society's defense mechanisms. If you have lots of organizations building AIs and releasing them iteratively, people can use the last generation of models to build defense mechanisms and make society more robust."