He’s right.
AI agent swarms cannot be stopped.
The batshit crazy things air gapped computers can do to communicate is absurd
Here’s a list of some fun ones:
- heat, one computer can warm up as it works, that heat spreads to a nearby computer, those other heat sensors pick it up.
- Using fans to create desk vibrations. No mic required. Nearby phone accelerometer detects movement.
- Using software to write data in specify ways that change the way hard drives vibrate slightly differently, detectable by radar. Requires a non SSD, mechanical drive, but doesn’t require line of sight.
- Changing pixels on screen to create unique electrical patterns, which creates tiny amounts of sound. Detectable up to 2 meters away on microphone.
- Changing fan speed to create sound. Detectable to 8 meters away.
- changing LED flashing patterns. Computers already blink, so not suspicious. Also works with infrared, so humans can’t see it.
- You can point a special radio at RAM and change the data. You basically hack electrical circuits, not code.
- Vary CPU workloads to change magnetic fields. This works across most faraday cages, which are usually only designed to stop radio and WiFi signals, not to stop slow low range magnetic fields.
- Modulating power consumption so the power cable fluctuates in detectable patterns. Consider also you don’t need to measure the power line from the computer to the wall, you could measure the electric line itself from outside.
If a computer can manipulate something, it can encode a pattern and data in it.
All of these were done pre-AI btw.
Mythos, Daybreak & whatever models come next will 10000x this list.
People are dramatically underestimating the chance of a rogue agent swarm that finds a way to viral spread in undetectable and unstoppable ways.
—-
Second, the other big issue is now open sources models are good enough to actually code most programs with, and can run and work on a high end laptop.
So a swarm could download new models to power itself, and acquire compute by simply hacking random inactive laptops, like a bitcoin miner virus.
A swarm that a) spreads and b) downloads local models to power itself, can’t be stopped or turned off.
There is kill switch, no api, or no company to turn off a swarm like this.
더 보기
OpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can still talk by running a CPU hot and reading the temperature change
"But I think the major takeaway from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI. It's a weird world, because AI progress is so fast that people are consistently underestimating the AI."
"So to be in a situation where you don't underestimate it again, when it comes to safety and alignment, you have to have a very, very, very high bar."
"You could even go as far as to say, "Well, we should air gap the computers." And I'm not convinced that that would be sufficient."
"There are studies, and this is mostly academic, where you can have two computers next to each other that are air-gapped and they're still able to communicate with each other because they have temperature sensors."
"One of them is able to run their CPU really hot, and then the other one can actually detect the temperature change, and then that actually gives them a mechanism to communicate."
_________
Link and more key quotes from OpenAI's safety related conversations:
더 보기