Why AI-assisted bioweapons won't kill us all

I interviewed dozens of biosecurity experts. The fear that AI will enable lone wolves to build pandemics is a case study in how domain expertise gets sidelined.

Some of my friends think we are going to die because AI will make building a bioweapon easier.

The fear is often that AI will walk an ordinary person through unleashing a pandemic. Or that AI will facilitate the creation of an engineered pandemic that could kill millions or billions on Earth.

I spent ten years as a US diplomat. My job was often to check claims before Washington acted. I wanted to double check this too.

I interviewed dozens of biosecurity experts who had decades at the lab bench.

Yes, AI makes virology work a little easier. But it is still extremely difficult. Media headlines often say “AI can make this step easier” without the broader context that a bioweapon involves a lot of hard steps that are not easier.

The public doesn’t hear this context because of our information ecosystem.

Let me first explain why I’m no longer worried:

Why bioweapons are rare

Since the 1980s, there has been exactly one fatal bioterror attack.¹

They are rare because 1. they are difficult, and 2. they are bad weapons.

People who want to cause harm still do cost-benefit thinking. For almost any goal they have, a bomb, a gun, a chemical, or a cyberattack is cheaper, faster, and more controllable.

Practitioners flagged several more barriers:

1. You have to be comfortable killing your own people. Pathogens kill indiscriminately. Few groups are this malicious and extreme. Groups that are tend toward paranoia and rigid hierarchies that make them less effective.

2. You need significant resources. Bioweapons capable of harming many people need equipment, chemicals, and professionals trained in different fields.

3. You need an institution for access to specialized equipment and materials. Many items are too niche for black markets. Essential equipment like biosafety cabinets or fume hoods requires professional installation and ongoing manufacturer support. Some materials are restricted; for example, viruses need host cells, cell lines are sold only to institutions, and plasmids require institutional credentials. In the United States, only tens of thousands of people have sustained access to such facilities and supplies.

4. Your team must be both highly competent and completely secret. A contamination incident could destroy the pathogen or infect the team.

5. You need to convince and retain experts from different domains e.g. biology, chemistry, engineering, aerosolization, or fieldcraft to carry out attacks. Recruiting without detection is hard too.

For a detailed version of this list, see my full piece.

Even well-resourced groups with scientific talent still fail. Aum Shinrikyo, a Japanese cult with ~$1 billion, failed in all ten of their attempts to release biological agents.

Why AI models don’t change this

AI can supply biology and chemistry knowledge, but not hands-on skill.

Reading every surgery textbook doesn’t make you a good surgeon. The biology laboratory works in a similar apprenticeship model. You learn by failing repeatedly under a mentor who has also failed repeatedly.

To get a sense of the physical nature of biology work, I explored one aspect: moving liquids.

Biologists often use pipettes, very precise eye-dropper-like tools. Students will often try fifty times to get it right. Practitioners develop a feel for specific materials over the years.

Photo by CDC on Unsplash

This skill appears in almost every bioweapon protocol. It is hard because:

Materials are often fragile. Some DNA strands can shear from hand tremors. Some proteins are so temperature sensitive that the pressure of moving the liquid generates enough heat to destroy them.

Concentration has to be exact. If a sample is too dilute, DNA strands may never find each other. Instructions might say “check with a spectrophotometer,” but a practiced scientist recognizes when a bubble or residual chemical is corrupting the reading.

Contamination in biology is unforgiving. A one-in-a-million contaminant in chemistry is usually negligible. In biology, this can matter because organisms multiply.

Creating a bioweapon involves many more complex steps. Converting viral DNA into a live virus is hard even for researchers. Stabilizing a pathogen into an effective aerosol often requires specialists from different fields. AI can describe the process but doesn’t give you the skills to execute it. (I walk through the full chain of steps here.)

Even success probably wouldn’t be catastrophic

A motivated team of scientists with institutional access could keep trying and eventually succeed at making a pathogen.

But success would not necessarily produce a new pandemic. Any pathogen that a team could plausibly assemble is one that already exists in nature. A lab-assembled version of COVID-19 would have about the same effect as an infected person coughing in a crowded room.

The more serious worry is that someone could revive a pathogen no longer in circulation, like the 1918 influenza or engineer something entirely new.

The 1918 flu infected roughly a third of the world’s population and killed an estimated 50 million people. Releasing it today would be very bad. It would not be 1918-bad. Most people’s immune systems recognize flu descendants. We also now have antivirals and antibiotics to treat the secondary bacterial pneumonia that killed most victims. Today’s vaccines offer cross-protection.

Engineering a virus that is both extremely lethal and highly contagious, worse than anything nature has produced, would be bad. But many biologists doubt this is possible even with perfect technology. People have very complex immune systems that often react differently. Viruses mutate as they spread. But more importantly: biology has built-in tradeoffs: the more lethal a virus is, the less transmissible it is (because it kills its human host).

When it comes to designing such a virus, any edits to make a virus deadlier could make the virus unstable or incompatible with human life. In physics, you can tell when an experiment is getting closer to a goal. Biology doesn’t work that way, meaning you can spend a long time on an endpoint that is not biologically possible.

Nature makes new viruses constantly, but by brute force. It spins out countless mutations and lets billions of them die every day.

Why this isn’t conventional wisdom

When I published this research, leaders in biosecurity and national security organizations thanked me privately. “My organization found the same conclusion, hence why we don’t work on that.”

Null results aren’t often published. Looking at the media sphere specifically: “bioweapons are hard” is not news. It isn’t a headline; it’s always been true.

Advocacy organizations have similar constraints. If you are pushing for AI oversight, a finding that undercuts urgency on one specific threat does not help.

Biologists, the people best positioned to explain this, mostly don’t write for general audiences. Academic incentives don’t reward op-eds, especially about things that are obvious within the field.

Our information systems get less accurate as issues become politicized. Alarm, not context, ends up on the news partly because of who is incentivized to speak up.

I’ve worked through the evidence across three Golden Gate Institute for AI pieces, if you want to go deeper:

• To Forecast AI’s Impact on Biosecurity, We Asked: Why Are Attacks So Rare?

• Tacit Knowledge: The Missing Factor in AI Bio Risk Assessments

• AI Can Help Plan a Bioweapon. Building One Is Still Hard

Update: To be clear, biosecurity is still critical! My point above is narrow, aimed at a claim common in EA and x-risk circles. Even EA-affiliated organizations like CLTR and GovAI have published a similar point: an outsized focus on lone-wolf bioterrorism and engineered pandemics can crowd out other threats, like smaller bioweapons, non-transmissible pathogens, chemical weapons, and explosives. Broad biological resilience helps across many of these scenarios, especially naturally occurring pandemics, which carry a 2-3% chance each year.

Huge thanks to Golden Gate Institute for AI for supporting this research!

I am giving a version of this talk at the Mind and Machine Alignment Summit at Ohio State University tomorrow.

AI and cyber risk is my next AI deep dive. I’d love your favorite resources in the comments! Reach out directly if you’d like to talk.

1. The 2001 anthrax letters.