I agree with @DavidRBellamy that people are totally miscalibrated on the risk of AI designing dangerous viruses. We should not be talking about AI-engineered bioweapons like it is the literal end of the world. The upside from medicine is going to be so much larger than the downside from engineered bioweapons. In addition David's points, there is actually an even more fundamental point here in our favor, which is evolution. As soon as you release an engineered virus into the wild, the virus is no longer under your control: it will evolve however it wants. And, what viruses want is to maximize their ability to replicate. The thing that maximizes their ability to replicate is to infect as many people as possible, which means being extremely contagious and not killing their hosts (dead hosts don't spread virus). The viruses that are most evolved for human biology are the common cold viruses: extremely contagious and not at all lethal. Viruses that kill humans do so by accident, usually because they are new to human biology (e.g. COVID, when it first jumped, or flu when it jumps from birds). If you stick a bunch of machinery into the virus to kill humans, I guarantee you that machinery will disappear from the virus very quickly. In response to this, I hear people say things like "the AI could engineer a kill switch so that the virus doesn't kill the humans initially but then once it has spread through the entire population the AI will hit the switch and kill all the humans." No. What would actually happen in practice is that the kill switch would accumulate deleterious mutations because there would be no evolutionary pressure to preserve it, and would quickly become non-functional. Evolution is fundamental. There is no way around it. For AI readers, trying to engineer a virus by setting its initial genetic code and then releasing it into the wild is like trying to train a model by setting the initial weights and then hill-climbing on a hidden training mixture you have no control over. And you're not even allowed to run any experiments in advance! You may be able to influence the behavior of the model in the first few iterations, but you will quickly lose control. There are big dangers. AI will be great at making one-off, non-replicating biological weapons, which are also scary (but much less scary than replicating bioweapons). AI will be great at helping people to weaponize existing pathogens, which is also a major danger. Also, a misanthropic model could get creative: it could release viruses repeatedly to counteract the effects of evolution, for example. Many people could die this way. Sensible surveillance is important, as is having proportionate controls on wet lab equipment. But we don't live in the dark ages anymore, we're not going to have a smallpox or black death-style pandemic where 50% of people die. COVID killed ~0.1% of humanity. A pandemic 100x worse than COVID would be horrific, but it would also not be civilization-ending. I get the sense that many AI researchers (with the notable exception of Dario personally) actually expect that AI in biology will do more harm than it will do good. This could not be further from the truth. As someone who likewise falls into the very small group of people who have actually physically made viruses with their own hands and has also worked on frontier AI, the upside here is huge, and the downside is not anything like what is being portrayed.