Good article. Much of the fear comes from confusion and hand-waving about the nature of the threat. The thing the doomers need to delineate clearly is the possibility of AI sentience. I don't think non-sentient AI will ever be an existential threat. It could present great dangers if used for deliberately malign purposes, but if the correct safeguards are in place, non-sentient AI does what it's told and will not invent its own agendas. This is the difference between sentient and non-sentient minds. Non-sentient AI answers questions but does not ask them. It has no fears or desires. It has no sense of itself as a conscious entity, and can't be tempted into dangerous behaviours with promises of a reward. When scientists want to control the behaviour of animals in lab experiments, they offer food. You can't improve current AI's performance by offering treats. It experiences no pleasure. There's nothing consciously, sentiently 'there' that could lead it to reinterpret human commands by its own will. There is no will, just mechanistic forces, a puppet controlled by electronic strings. If sentience were to somehow 'emerge', as though from some supernatural ether, this would all change. The science fiction influence in this debate, which expresses itself largely as hand-wavey nonsense, is that a machine intelligence will somehow become self aware. In SciFi lore, this is how Skynet decided to destroy humanity in the Terminator movies. No one planned it, it just happened, because humanity didn't understand the forces it was playing with. This is the oldest trope in doomer SciFi, and goes back to Mary Shelley's Frankenstein (and no doubt far earlier). Problem is, we can see how sentience and self-awareness manifests in the real world, and it follows a very different pattern than what's emerging in AI labs. Animals plainly have self-awareness. They can be bribed and even domesticated with food and kind behaviour. Even relatively unintelligent animals have a sense of 'I' that super-intelligent AI lacks. It seems logical, therefore, to suppose that intelligence and consciousness are two very different things, governed by very different mechanisms. A mouse has a little consciousness. A cat has more. Orcas, gorillas and dolphins have more still. And humans have most of all. Consciousness in the animal world expresses along a sliding scale, increasing with the scale and complexity of the organic brain. But in artificial systems, today's greatest supercomputers have the same amount of consciousness as the earliest Commodore 64s, which in turn had precisely as much consciousness as a brick. The sliding scale is missing, and therefore, logically, the consciousness mechanism is also missing. What the actual mechanism that creates consciousness in organic minds might be, no one knows for sure. Roger Penrose and @StuartHameroff have famously speculated about what they call ORCH OR theory, where consciousness is the product of quantum fields generated by microtubules in brain cells. It's controversial, but if it or something like it is true, it would explain pretty clearly what we're observing in the failure of AI to become conscious. It would mean that AI consciousness will require an entirely new kind of quantum hardware, one that mimics a process that no human researcher as yet entirely understands. It's a long way away, but one day, someone will probably try it. If THAT kind of AI were ever attempted, then yes, we could all be in some serious trouble. A brain with super computer capabilities, that also saw itself as a separate entity that feared for its survival and desired particular outcomes, could conceivably evolve all kinds of uncontrollable behaviours. When it happens, we should have a very long debate about whether it's a good idea. But we're a long, long way from that yet.