With all due respect, @gmiller, but this reminds me of the "Courtier's Reply" from the New Atheism era, when atheists were told that they had no standing to question theism until they'd mastered the ontological argument, the Kalam cosmological argument, skeptical theism, the Five Ways, and whatever the latest iteration of Sophisticated Theology happened to be. Back then, unless you had read your Richard Swinburne, your Alvin Plantinga and your John Haught, you couldn't appreciate the finer fabrics of the emperor's wardrobe, and so weren't entitled to say he was naked. Personally, I've addressed instrumental convergence and power-seeking (in the context of evolutionary doom scenarios) at some length, but it's never enough. There's always another theoretical argument or doom scenario that raises the entry fee to the discussion. Essentially the theologians (or AI doomers) set up their own theoretical game with its own rules, and won't take you seriously unless you play by those rules. The dynamic is self-selecting: only people already convinced that the doom scenarios deserve attention will invest the effort to keep up with the sprawling literature, so skeptics can always be dismissed as "not well informed." I'm not claiming the two cases are on an epistemic par, since AI risk at least has some empirical basis that theism lacks entirely. But the defensive move looks very similar, and does not help to dispell the suspicion that AI doomerism has some cult-like features. Let's also tag @sapinker.

With all due respect, @gmiller, but this reminds me of the "Courtier's Reply" from the New Atheism era, when atheists were told that they had no standing to question theism until they'd mastered the ontological argument, the Kalam cosmological argument, skeptical theism, the Five Ways, and whatever the latest iteration of Sophisticated Theology happened to be. Back then, unless you had read your Richard Swinburne, your Alvin Plantinga and your John Haught, you couldn't appreciate the finer fabrics of the emperor's wardrobe, and so weren't entitled to say he was naked. Personally, I've addressed instrumental convergence and power-seeking (in the context of evolutionary doom scenarios) at some length, but it's never enough. There's always another theoretical argument or doom scenario that raises the entry fee to the discussion. Essentially the theologians (or AI doomers) set up their own theoretical game with its own rules, and won't take you seriously unless you play by those rules. The dynamic is self-selecting: only people already convinced that the doom scenarios deserve attention will invest the effort to keep up with the sprawling literature, so skeptics can always be dismissed as "not well informed." I'm not claiming the two cases are on an epistemic par, since AI risk at least has some empirical basis that theism lacks entirely. But the defensive move looks very similar, and does not help to dispell the suspicion that AI doomerism has some cult-like features. Let's also tag @sapinker.