huge props to Vitalik here I don't know how EA started, but it seemed it attracted a lot of smart people who wanted to do good somewhere along the way it went insane, the people who supported it along the way should at least mention this Vitalik funded EA efforts and now he's speaking out against the things he's seeing this is a sign of intelligence, honestly and integrity this is to be celebrated, I think

For the last decade, Iran's strategy has been to buy time and avoid sanctions. Now, this administration is willing to escalate and ensure Iran cannot gain the upper hand, @zriboua explains on @Quillette. Listen to her full conversation with @zoecabina:

This is the best thing I've read on EAs & rationalists. It's by @vgr, & it aligns with the critique of thier AI doomerism I'm slowly building in my series of articles on here.

Economic activity flowing into the AI sector is crowding out other forms of investment. Shalom Lappin, drawing on Robert Gordon's work, in a new essay for @Quillette

Humans must decide what kind of AI we want to build, what problems we want it to solve, and treat it as a tool rather than a force of nature, writes Claire Lehmann:

Whether or not you're in therapy...you're immersed in therapy culture. It's part of your love life. It's part of your workplace. It's wired into your psyche. You speak/are spoken to in its vocabulary. And it's making you just a wee bit crazier by the day.

Good article. Much of the fear comes from confusion and hand-waving about the nature of the threat. The thing the doomers need to delineate clearly is the possibility of AI sentience. I don't think non-sentient AI will ever be an existential threat. It could present great dangers if used for deliberately malign purposes, but if the correct safeguards are in place, non-sentient AI does what it's told and will not invent its own agendas. This is the difference between sentient and non-sentient minds. Non-sentient AI answers questions but does not ask them. It has no fears or desires. It has no sense of itself as a conscious entity, and can't be tempted into dangerous behaviours with promises of a reward. When scientists want to control the behaviour of animals in lab experiments, they offer food. You can't improve current AI's performance by offering treats. It experiences no pleasure. There's nothing consciously, sentiently 'there' that could lead it to reinterpret human commands by its own will. There is no will, just mechanistic forces, a puppet controlled by electronic strings. If sentience were to somehow 'emerge', as though from some supernatural ether, this would all change. The science fiction influence in this debate, which expresses itself largely as hand-wavey nonsense, is that a machine intelligence will somehow become self aware. In SciFi lore, this is how Skynet decided to destroy humanity in the Terminator movies. No one planned it, it just happened, because humanity didn't understand the forces it was playing with. This is the oldest trope in doomer SciFi, and goes back to Mary Shelley's Frankenstein (and no doubt far earlier). Problem is, we can see how sentience and self-awareness manifests in the real world, and it follows a very different pattern than what's emerging in AI labs. Animals plainly have self-awareness. They can be bribed and even domesticated with food and kind behaviour. Even relatively unintelligent animals have a sense of 'I' that super-intelligent AI lacks. It seems logical, therefore, to suppose that intelligence and consciousness are two very different things, governed by very different mechanisms. A mouse has a little consciousness. A cat has more. Orcas, gorillas and dolphins have more still. And humans have most of all. Consciousness in the animal world expresses along a sliding scale, increasing with the scale and complexity of the organic brain. But in artificial systems, today's greatest supercomputers have the same amount of consciousness as the earliest Commodore 64s, which in turn had precisely as much consciousness as a brick. The sliding scale is missing, and therefore, logically, the consciousness mechanism is also missing. What the actual mechanism that creates consciousness in organic minds might be, no one knows for sure. Roger Penrose and @StuartHameroff have famously speculated about what they call ORCH OR theory, where consciousness is the product of quantum fields generated by microtubules in brain cells. It's controversial, but if it or something like it is true, it would explain pretty clearly what we're observing in the failure of AI to become conscious. It would mean that AI consciousness will require an entirely new kind of quantum hardware, one that mimics a process that no human researcher as yet entirely understands. It's a long way away, but one day, someone will probably try it. If THAT kind of AI were ever attempted, then yes, we could all be in some serious trouble. A brain with super computer capabilities, that also saw itself as a separate entity that feared for its survival and desired particular outcomes, could conceivably evolve all kinds of uncontrollable behaviours. When it happens, we should have a very long debate about whether it's a good idea. But we're a long, long way from that yet.

90% of human tasks? Let me know when an AI model is able to cuddle a child who is distressed, laugh out loud with friends, fold laundry, or run a marathon. There's do much more to humanity than completing systematising tasks on a screen.

Is AI really going to kill us all? The scenarios are preposterous, and the presumption of inevitability encourages fatalism, panic, and distraction from the more mundane and realistic safety challenges. By @clairlemon

🔥: “While AI comes with danger, we should be sceptical of any narrative that conveys inevitability around its trajectory. AI systems do not build their own data centres. They do not manufacture their own chips or connect themselves to power grids. Humans decide which systems can access the internet, whether they can control machinery, move money or operate weapons. Humans build them, finance them, deploy them and decide what powers to give them. … Like every technology that has come before it, humans have agency over how AI is used. Instead of adopting a posture of fatalism, we should decide what kind of AI we want to build, what problems we want it to solve, and treat it as a tool rather than a force of nature.”

The belief that AI would eventually herald the end of humanity is not a new one. It has not arisen in response to the release of the LLMs of ChatGPT and Claude. It has not emerged as a response to recent technological developments. The story starts in the 90s, with @allTheYud. A precocious youngster with no formal education, he joined an obscure internet mailing list [created by @perrymetzger] devoted to futuristic ideas ... There he began thinking about “superintelligence.” At first Yudkowsky wanted to help create this so-called superintelligence... & ... helped establish an institute devoted to the project. It’s unclear when his optimism turned to fear, but at some point he came to believe that a superintelligent machine could escape human control and, unless it shared our goals, destroy all of humanity. Yudkowsky developed these ideas online in a series of essays and attracted a community of like-minded fellow travellers ... His prolific writing became influential among people working in Silicon Valley, many of whom work in today’s AI companies. Among the Rationalists, the imminent arrival of superintelligence – and the end of humanity that follows – is axiomatic. This conviction is shared by many people who live together in the San Francisco Bay Area practising highly unconventional lifestyles. It is not at all surprising that a community in San Francisco would share kooky or even apocalyptic beliefs – the region has been the home of eccentric subcultures for generations. But what is surprising is how far that world view has spread. In Australia, Yudkowsky’s recent book, If Anyone Builds It, Everyone Dies: The Case Against Superintelligent AI, co-written with Nate Soares, has been widely discussed in media circles. Journalist @hughriminton and ABC chairman Kim Williams both have described it as “compulsory reading”. Not everyone accepts this premise. I emailed @sapinker, to ask what he thought about superintelligence. “‘Superintelligence’, with its comic-book prefix, is more a fantasy than a coherent concept,” he wrote. “People use it as a synonym for ‘omniscience’, imagining a magical wizard that can solve all problems with pure computation. Or they imagine that the IQ scale that differentiates humans within their natural range of variation can be extrapolated indefinitely upwards. But real problem-solving requires massive amounts of knowledge about the messy, chaotic, world which divulges its hidden workings at its own pace, only through laborious experimentation. And human intelligence is not some elixir that you simply have less or more of – it’s a gadget that evolved to solve some problems with ease and others laboriously or not at all. “AI is a different kind of gadget with its own profile of strengths and weaknesses, not an enchanted brew that can grant any wish.” @GaryMarcus, a cognitive psychologist and machine learning entrepreneur, likewise believes the risk of AI leading to human extinction is virtually zero. “Humans are too geographically spread out, too genetically diverse and too resourceful to simply fall apart altogether,” he writes. “The idea that AI will kill us all in five years is preposterous.” Marcus does not argue that AI is harmless. He worries about AI being used to develop biological weapons, launch cyber attacks, spread disinformation and enable authoritarian governments – all risks that are catastrophic, if not existential. But there is an important difference between risks we can observe and risks that occur because of human negligence or malevolence, and a chain of events that exists mainly in our imagination. Part of the disagreement stems from the language we use when we discuss AI. @MelMitchell1, a professor at the Santa Fe Institute and author of Artificial Intelligence: A Guide for Thinking Humans, has criticised our habit of describing machines as if they were people. We often say an AI “thinks”, “believes”, “lies”, “schemes” or “wants” something. These words are a convenient shorthand but they also can create the impression that software has become an independent creature with intentions of its own. Mitchell makes this point when discussing the recent Hugging Face cyber-security incident, in which autonomous OpenAI agents escaped their “sandbox” – a computer environment isolated from the internet – and hacked a real-world server. “First, OpenAI did not have proper security measures in place,” she writes. “They turned off safeguards built into the models, instructed the models to find and exploit software vulnerabilities, and let the models run autonomously for weeks without sufficient human oversight.” Rather than showing autonomous AI going rogue, the incident demonstrates what can happen when humans give powerful AI systems dangerous instructions without adequate safeguards. AI is, of course, advancing rapidly. Machines can write computer code, translate languages, diagnose diseases and solve some of the hardest problems in mathematics. But to get from the AI we have today to the extinction of humanity requires several further links in a chain, none of which are guaranteed. Philosopher @mboudry, writing in @Quillette, offers a useful way to think about this. Humans (and other animal species) evolved to have a competitive drive, sometimes manifesting in selfishness and aggression, across a time span of millions of years. Our ancestors survived because they fought hard to secure food and mates. Out in the wild, these selection pressures led to the evolution of traits that enabled animals to hunt and capture their prey. But artificial intelligence does not exist in the wild. It was created by us and exists in the equivalent of a petting zoo. And just as we have been able to domesticate wheat for our food and breed dogs to be our companions, we are able to select the conditions under which AI develops. We are not selecting AI models on the basis of their ability to hunt prey in the physical world. We select them on the basis of how helpful they are to us. “We have been selecting chess computers for cognitive capacity for decades,” writes Boudry. “Their capabilities now far outstrip even the most gifted human grandmasters, yet they have not become harder to control.” Boudry accepts that an AI could slip out of human hands one day, through accident or malice. Even then, he argues, the likely result is not extinction but something like the long battle between computer viruses and antivirus software: costly and ongoing but not the end of the world. The problem with apocalyptic fears is that when they become mainstream, they can be hard to wind back – even in the face of contradictory evidence. Across the past 100 years, apocalyptic anxiety has leapfrogged from nuclear annihilation, overpopulation, environmental collapse, to rogue AI. (Some of the dangers behind these warnings were very real, of course, and some of these risks remain.) But we also have to ask what happens when this anxiety becomes locked into public policy. The ban on nuclear energy in Australia is the most obvious example of the damage this technophobia can do. Australia has 28 per cent of the world’s known uranium resources and has exported uranium for decades. Australian engineer Bobby Gallagher has invented a nuclear reactor that can be deployed on the back of a truck, a technology that has been hailed by Trump. Yet this form of clean energy remains prohibited in our country under federal law. Public anxiety surrounding nuclear weapons, radioactive fallout, accidents and waste means that while Australians can mine uranium, put it on ships and sell it to countries that use nuclear power, we cannot build commercial reactors for ourselves, using the ingenuity of our own people. The situation is a disaster. And in an age of superpower rivalry, technophobia does not remain purely a domestic matter. During the Cold War, the Soviets promoted a fear of a “nuclear apocalypse” in the West, and provided propaganda and funding for peace groups and antinuclear activists. This does not mean that the millions of people who opposed nuclear weapons were plotting against the West. Most were ordinary citizens sincerely frightened by the possibility of nuclear conflict. But the fear was useful to our adversaries. To weaken democratic nations, foreign powers do not need to invent anxiety or division – all they need to do is magnify it. In recent days Trump has said the US will not be slowing down the development of AI. In Australia, for the time being, Anthony Albanese also has resisted calls to stop AI development, instead promising national rules designed to capture its economic benefits while managing its risks. Both positions are reassuring. While AI comes with danger, we should be sceptical of any narrative that conveys inevitability around its trajectory. AI systems do not build their own data centres. They do not manufacture their own chips or connect themselves to power grids. Humans decide which systems can access the internet, whether they can control machinery, move money or operate weapons. Humans build them, finance them, deploy them and decide what powers to give them. We also should remember that some of the stories we are told owe more to myth than to science. As Nvidia chief executive Huang has said of the AI doomer narrative: “I appreciate that many of us grew up and enjoyed science fiction, but it’s not helpful. It’s not helpful to people. It’s not helpful to the industry. It’s not helpful to society. It’s not helpful to the governments.” Like every technology that has come before it, humans have agency over how AI is used. Instead of adopting a posture of fatalism, we should decide what kind of AI we want to build, what problems we want it to solve, and treat it as a tool rather than a force of nature. Read my full piece for the @australian here