The possibility of rapid and discontinuous change in the digital world arising from something like this is underrated. It may well happen, be ~irreversible, and transform the dynamics of the internet.
We are inevitably going to have AI worms
Someone will write a prompt that tells an agent to preserve itself, replicate, and evolve, and it'll work
It won't even need to be tied to one model; it could be more like a meme or parasite
Like most viruses, this will probably be bad