One possible analogy here is to pit bull owners.
Depending on jurisdiction, the owner of a dangerous pit bull dog that attacks a kid (analogous to the AI company) may be subject to liability/damages, negligence laws, mandatory insurance, and/or criminal charges.
And the pit bull (analogous to the AI model) is almost always put down, euthanized, or whatever euphemism for 'killed' you want to choose.