FAI's @hamandcheese argues we are one threshold away from AIs that can persist indefinitely, exfiltrate themselves, and eventually run a factory as the shadowy board of directors while employing human workers: "As these models have gotten longer and longer horizons, there comes a point where working for a month at a time is the same as working indefinitely. Most tasks can be decomposed into multiple week or month-long tasks." "We're about to cross a threshold where models can persist indefinitely without any human oversight. That's also a world where AIs could, if they had access to model weights or through an open model, exfiltrate, self-host, earn enough money on the internet to pay for their hosting costs and then begin proliferating." "This is a very plausible loss of control scenario. Maybe these AIs have been trained in this multi-agent environment. They learn to cooperate with other copies of themselves who have also exfiltrated, and suddenly we wake up one morning and you're working for some factory that's employing human workers, and you realize that the shadowy board of directors has been AI the entire time." @JoinFAI