Why a Catastrophe Caused by Artificial Intelligence May Be Inevitable—and What COVID-19 Has to Do with It

The argument begins with gain-of-function research: potentially dangerous capabilities are studied partly because understanding offence may strengthen defence. Yet every additional laboratory, actor, interface and year of operation also creates another possible route to accident, theft or misuse.

COVID-19 provides an analogy—not established proof. The origin of SARS-CoV-2 remains unresolved. The WHO’s independent scientific assessment says the available evidence currently favours zoonotic spillover, while a laboratory incident cannot be excluded because important information remains unavailable.

My argument does not depend on resolving that dispute. Its central premise is simpler: high-consequence systems can reduce residual risk towards zero, but never guarantee absolute zero.

Frontier AI increasingly operates under the same attack–defence logic. Developers test biological, chemical, cyber, autonomous and other potentially dangerous capabilities because they need to understand what their models can do and how misuse might be prevented. Such evaluations are necessary—but growing capability, diffusion and scale also enlarge the landscape of possible accidents, security failures and deliberate abuse.

The boundary is already moving. A Science study published on August 6, 2026 reported that genome language models generated complete bacteriophage genomes. Sixteen designs produced viable viruses that infect bacteria, and an AI-designed phage cocktail overcame phage-resistant E. coli.

These were bacteriophages—not human pathogens—and the work could advance antimicrobial innovation. But it demonstrates that generative AI can now help design complete, functional viral genomes.

“Inevitable” should therefore be understood as a governance warning, not a calibrated prediction or mathematical theorem. Safeguards can improve, systems can change, and dangerous pathways can be closed. Nevertheless, if consequential failure remains persistently possible across an expanding technological ecosystem, prevention alone is insufficient.

We also need containment, monitoring, secure access, incident reporting, rapid response, redundancy and societal resilience.

Humanity is extraordinarily robust. An AI catastrophe is unlikely to resemble a Hollywood apocalypse, and civilisation will probably endure. But survival at the level of the species offers little consolation to a patient, a premature newborn or a family harmed by a preventable failure.

My optimistic–sober conclusion remains: Andrà tutto bene—probably for most, but not automatically for all.