The pandemic threat of DEEP VZN—notes on a podcast with Kevin Esvelt

[Kei]

In 2021, USAID announced a program called DEEP VZN, with the goal of identifying unknown viruses with the potential to cause pandemics and publishing their genomes. A number of biologists, including Kevin Esvelt, who leads the ‘Sculpting Evolution’ lab at MIT, have raised the alarm about this program. Esvelt argues that this program would increase the risk of nasty future pandemics by providing publicly-available blueprints for creating pandemic-grade viruses. He estimates that there are tens of thousands of people who currently have this skill of building a virus from its genome. (This was mentioned in an EA forum post from last month.)

Esvelt recently went on a podcast with Rob Reid to discuss the DEEP VZN program, why it would increase our risk of pandemics, and what people should do to influence USAID to go down a different path. Below, I write an overview of their conversation. I start out with a summary of the podcast’s main case, followed by detailed notes discussing all of the main topics that were covered.

Summary

USAID recently launched DEEP VZN, a five-year, $125 million program designed to monitor and better understand future pandemics. The main component of DEEP VZN involves identifying roughly ten thousand novel viruses, determining which are capable of causing dangerous pandemics, releasing a list of these viruses rank-ordered by how dangerous they are, and publishing the genomes for these viruses online. These pandemic-grade viruses could be as dangerous or more dangerous than any other viruses we have discovered so far.

Kevin Esvelt, a biologist and assistant professor at MIT, argues that this program would pose a significant risk to society as a whole. Were DEEP VZN to identify pandemic-grade viruses and publish their genomes, they would implicitly provide blueprints that could be used to generate the viruses, and spawn novel pandemics, by any sufficiently skilled biologist. Currently it is estimated that tens of thousands of people are capable of this work—animating viruses from a genome—even if they don’t have access to the original viral sample. We expect this number to go up significantly as technology continues to advance.

Risks surrounding pandemics and biological weapons are not just a hypothetical. There are a number of people in recent decades who had the skill and/​or desire to release biological weapons. Most infamously, Seiichi Endo, a terrorist involved in a chemical weapons attack on a Tokyo subway, was a graduate-trained virologist. He once tried to obtain samples of Ebola for use against civilians, although thankfully he failed. Releasing the genomes for pandemic-grade viruses would make it easier for terrorists with sufficient training to launch one or even multiple pandemics at the same time and cause vast numbers of deaths.

Anyone worried about these risks should encourage the USAID to shift money away from identifying pandemic-grade viruses, and towards the component of DEEP VZN dedicated to disease monitoring. Disease monitoring programs will give us an early-warning system for emerging pandemics. Specific points of contact include:

• Messaging your elected representatives on this topic—they have staff to field in-bound messages

  • If you are from MD, HI, VA, CT, MA, TN, KY, TX, WI, FL, NJ, or IA, you have a senator on the US Senate Foreign Relations Committee subcommittee overseeing USAID (List of senators on the subcommittee: https://​​www.govtrack.us/​​congress/​​committees/​​SSFR/​​14)

• Contacting USAID directly via their online contact box at usaid.gov/​contact-us

• Tweeting at the USAID Twitter account: https://​​twitter.com/​​USAID

• Spreading the message via social media or blogging

• If you know heavy hitters within the USAID, reach out to them and share your feelings or link to the podcast

Detailed notes

Judging when technologies are dangerous

• Whenever a technology is accessible to many people, there are some people who think about misusing that technology

  • Technologies that are slow, obvious, and easily blocked are not at much risk of being misused as a weapon

  • An obvious example of a technology that can’t be easily countered is a nuke, which we don’t know how to stop with high probability even after decades of research. We deal with this by minimizing access to nukes, and deterring those who wield them

• Blueprints for dangerous technologies like nukes are an example of an information hazard. Information hazards exist when the release of certain information may cause harm or enable someone else to cause harm

  • One type of information hazard is called an attention hazard, where merely drawing attention to a certain fact increases risk

    • An example of an attention hazard: Leo Szilard, the physicist who first spurred the US to create an atomic bomb, argued once Nazi Germany was defeated that the US should not use nukes on Japan. He claimed that using a nuke would advertise to the world the power of such a weapon, which would make other countries race to acquire one themselves, which would imperil the US

  • The genetic sequences for potential pandemic-causing viruses, in particular ones that are not currently targeting humans, are another example of an information hazard

    • Due to recent advances in synthetic biology, tens of thousands of people are now capable of building and spreading viruses using their genetic sequences. We should expect that number to go up significantly in the coming years

How the Spanish flu was brought back over 80 years after it disappeared

• Influenza researchers at the CDC were worried something like the Spanish flu might come again, so they wanted to know what had caused it to be so bad for humans

• These researchers collected samples of the Spanish flu from a combination of old museum specimens, as well as from a person who had died of the flu and been buried in the Alaskan permafrost for 80 years.

• They were able to extract the virus’s RNA sequence from these samples, and proceeded to publish that sequence online for anyone to find

• Because of this, there now exists exact blueprints to build a disease that killed about 1 out of every 30 people on Earth

• While the Spanish flu is still very dangerous, thankfully we would expect it to kill less now due to improved antibiotics (~½ of deaths were from bacterial infections), and the fact that we have been exposed to less deadly descendents of the Spanish flu

• Kevin Esvelt’s very rough estimate is that it would kill between 200k and 10 million people today if it were released

• Why did the CDC do this?

  • The CDC has the mindset that if we understand viruses better, we can come up with better treatment and interventions that we can not create otherwise. Namely, they think it is worth the risk of giving hundreds to tens of thousands of people the ability to recreate pandemics on demand

  • The benefits we did see from resurrecting the Spanish flu came from a handful of specialized laboratories, and there was no need to publish the genome sequence

  • The CDC and the NIH do not seem to regret the decision to publicize the Spanish flu genome sequence, they have more recently even talked about doing gain of function research to take non-transmissible viruses and make them very transmissible in order to study them

Risks and timeline of virus synthesis

• The first virus ever created from synthetic DNA was the poliovirus in 2002

• At the time the CDC published the Spanish Flu sequence online in 2005, approximately 100 people in the world were capable of synthesizing the virus

• The cost of assembled DNA in the past 20 years has fallen by a factor of 1000, which has made synthesizing viruses significantly easier over time

• Now, between 20k-50k people in the world have the ability to build the Spanish flu, and this number will only grow over time

Is it too late to stop the risk associated with the dissemination of genetic sequences of pandemic-causing viruses?

• Some have argued that enough genetic sequences of dangerous pandemic-grade viruses are already public, and it is useless to try to stop the publication of additional sequences. Kevin Esvelt disagrees with this mindset

• While the sequences for some dangerous viruses are publicly available, and those viruses could cause significant damage if released, there are limitations to how much damage those viruses could do

• Some examples of viruses with publicly available sequences and why those viruses are limited in how much damage they could do if re-released (Esvelt estimates the death toll for these diseases would be less than COVID for example)

  • Variola virus (smallpox)

    • We have around 350 million doses of vaccine ready to go which could be deployed in an outbreak scenario

    • Smallpox is much harder to synthesize than most other viruses (only about 100 can make it now)

    • Smallpox does not have asymptomatic transmission like COVID, which gives us a better chance of getting rid of it

  • Ebola, SARS, MERS

    • Transmissibility is low enough that we should be able to bring a pandemic under control

  • Standard H5N1 flu has a 60% death rate, but is barely contagious in nature. A recent experiment made it transmissible in ferrets

    • Transmissible in ferrets does not mean it could cause a pandemic in ferrets, and moreover does not mean transmissible in humans

    • It seems to be that in order for this disease to become more transmissible in humans, it will naturally become much less deadly

USAID is building a new program called DEEP VZN

• DEEP VZN is a five-year, $125 million program designed to “strengthen global capacity to detect and understand the risks of viral spillover from wildlife to humans that could cause another pandemic.”

• The main component of DEEP VZN involves “virus hunting”, where scientists take samples from wet markets, bushmeat markets, wild animals, and the general environment, isolate the viruses in those samples, sequence them, and make those sequences public

  • DEEP VZN hopes to find 8k-12k viruses over the course of the program

• DEEP VZN will then run tests on all of these viruses to identify which of them are most likely to cause a dangerous pandemic, and will release a list of these viruses, rank-ordered by how dangerous they are

  • This is by far the most expensive and technically difficult part of the program, requiring the work of many virologists, and probably would not be done if not for the DEEP VZN program

  • This work of determining whether a virus could cause a dangerous pandemic in humans is called characterization work

• DEEP VZN also wants to train scientists from selected countries in Asia, Latin America, and Africa, such that they too can identify and publicize pandemic-grade viruses

• If DEEP VZN’s program is a success, it will provide blueprints to generating new pandemics to any sufficiently skilled virologist, and will train teams all around the world to build additional blueprints themselves

• DEEP VZN is the successor to a program called PREDICT, which sequenced roughly 1200 unknown viruses in 20 countries

  • PREDICT took twice as long (10 years) and $200 million to sequence ¹⁄₁₀ the viruses as DEEP VZN would, signifying how quickly this technology is improving. That said, the characterization work is now the main bottleneck, and the speed of that is not increasing nearly as quickly

  • PREDICT had controversy surrounding it, such as giving money (through Ecohealth alliance) to fund research on coronaviruses in Wuhan, in labs known to have shoddy safety processes by the US government

  • There is some chance (Esvelt says various people have estimated between single digit% and 90%) this resulted in a leak that led us to COVID-19

• Despite the above, DEEP VZN has some good components, like monitoring human populations regularly exposed to animals likely to transmit dangerous pathogens. This would give us the ability to identify pandemics early and increase our chance of containing them

How many potential pandemic-causing viruses is DEEP VZN likely to find?

• There are four key classes of experiments virologists perform on animal virus to determine if it is likely to cause pandemic in humans

  • How tightly does the virus bind to human target cells?

  • How readily does the virus infect those cells?

  • How readily does the virus replicate and churn out new copies in relevant tissue types?

  • How transmissible is it in animal models that are chosen for similarities to humans?

• Once those four experiments are run, you can identify which viruses have major pandemic potential, which you don’t have knowledge about without running those experiments

• Rob/​Kevin’s best guess is that DEEP VZN would find roughly mid to high single digits numbers of pandemic-grade viruses, possibly, but not likely, into the dozens, though this guess is very approximate. They specifically use the number 8 as a plausible number a few times later in the podcast. Some potentially relevant information for forming an estimate is:

  • There are estimated to be between 40k to 500k viruses in mammals, of which about 20% are capable of human-to-human transmission (~10k to 100k capable of human transmission)

    • Note that human-to-human transmission does not mean pandemic if it isn’t transmissible enough

  • PREDICT, which sequenced about 1200 viruses, did not find any pandemic-grade viruses, although we have gotten better since then at identifying when a virus is pandemic-grade (some of the viruses PREDICT marked as not pandemic-grade could be false negatives)

  • The runners of the DEEP VZN program think they are likely to find some pandemic-grade viruses

A case for why USAID needs to shut down the pandemic-grade virus identification component of DEEP VZN

• If USAID identifies and makes publicly available the genomes of pandemic-grade viruses, tens of thousands of people will be in the position to follow recipes and animate those viruses even if they don’t have the original sample. And we expect the number of people with this capability to be significantly more than tens of thousands in the future

• The current population of frightening publicly-available genomes is not that bad, but DEEP VZN could substantially change that

• COVID has killed many more people than any single nuclear detonation

  • There are nine nation states currently with nuclear weapons

  • As soon as viruses likely to cause pandemics are identified, the number of actors capable of causing million+ deaths in expectation goes up over 1000 fold

• Nature throws us about 4 pandemics per century, if we expect DEEP VZN to find, say, 8 pandemic-grade viruses, someone could make and release all 8 at once if they wanted to

  • Furthermore, standard pandemics start from one place and slowly emanate outwards from there, a malicious actor could make it more dangerous by simultaneously releasing the pandemic across multiple travel hubs at once. This would make the death toll significantly higher as the whole world would be caught unawares

    • Deliberate release, even for a single virus, is worse than a natural pandemic for this reason

  • Our best-case goal right now is to make a vaccine within 100 days of a disease becoming known. We know from Omicron that this is too slow even for a natural pandemic, as a disease can spread all around the world within those 100 days

• The best case scenario for the DEEP VZN program is that with knowledge we gain, we prevent on average 4 natural spillover pandemics per century. But this is in exchange for providing tens of thousands of people the ability to launch more pandemics than that, simultaneously across multiple travel hubs

• Even if we didn’t know for sure whether any suspected pandemic-grade disease identified by DEEP VZN would really cause a pandemic, we would still be in incredible danger. For example, even if we thought each suspected pandemic-grade disease had a 50% chance of causing a pandemic, if someone assembled and released 5 such diseases, they would have a 96% chance of at least one succeeding, and an 81% chance of at least 2 succeeding

What types of actors should we be worried about synthesizing and releasing pandemic-grade viruses?

• There is a lot of risk from non-state actors and even individuals. There are a number of stories of mass-murdering individuals with the skill or desire to use biological weapons

  • Seiichi Endo was a member of an apocalyptic terrorist cult which released chemical weapons in the early-mid 1990s. Prior to joining the cult, he was a graduate-trained virologist, and sought to obtain samples of Ebola for use against civilians. Thankfully, he failed

  • James Holmes, a convicted mass murderer—the Aurora shooter, quit a life science PhD before he shot up the theater

  • The Unabomber talked about the immense power of biotechnology in his manifesto, even though it was decades ago. He may have tried to cause a pandemic if the technology was available to him at the time

• There are questions around whether ISIS, Al Qaeda, or other terrorist groups would be willing to release pandemic-grade viruses. Al Qaeda did once ask for people with skills in microbiology to build weapons of mass destruction

• There are people who desire to see society taken down, or are willing to kill many, many people, like the Germanwings pilot who decided to commit suicide, and did so by flying a plane with 144 passengers into a mountain

  • These sorts of people could be motivated to synthesize and release a pandemic-grade virus

What are the arguments in favor of DEEP VZN? What are responses to those arguments?

• Shouldn’t we want to know where dangerous pandemics dwell, and know exactly what they look like before they strike?

  • This program does not just help us learn about dangerous viruses—it also creates the schematics detailing how one would build such a virus, and makes those schematics public

  • In the best case scenario, we stop all ~4 natural pandemics that occur every century, but we’ll simultaneously give tens of thousands of people the ability to release more pandemics than would happen in the entire century all at once

  • It is not always good to know more about a threat, since there are costs of knowing. This is the whole point of information hazards

    • As an extreme/​absurd example, suppose there were some way of creating a singularity on Earth that would destroy the planet. It is clear in this example it is better if no one ever knows how to do this

  • Without this program, there won’t be credible pandemic-capable blueprints for quite some time. The longer we can push it off the more time we have to build actually effective defenses

    • Examples of effective defenses include:

      • Placing sequencing monitors everywhere, which enables us to pick up on any virus extremely quickly

      • We can build comfortable equipment at scale that keeps people from getting viruses and can distribute that equipment to everyone required to keep society going

    • We can build effective defenses in the future, but we can’t build them today

  • People assume that all technologies favor defense, and that if we know enough about a technology, we will be able to come up with some effective defense

    • But this is not always true. We know that nuclear weapons do not favor defense, and we know this from COVID—even long after we knew everything about it, we were not able to reliably defend against COVID

• If we identify a dangerous virus before it spillovers into the general population, wouldn’t we get a huge head start to counteract the virus?

  • In order to get a head start in vaccine production, we need to know whether the vaccine works. The only way to establish efficacy for these novel diseases would be to deliberately infect many people with a virus of unknown lethality which might never otherwise see a human, and which we think could cause a pandemic, and see if a vaccine protects them. This is what is called a challenge trial. Given that we weren’t even willing to do a challenge trial for COVID, it seems unlikely we would run a challenge trial for such a novel virus, and even less likely we would run a challenge trial on the potentially 8 or more pandemic-grade viruses we would find through DEEP VZN

  • Due to recent and near-future technological developments, we should be able to begin simultaneous phase 1 and phase 2 trials of an mRNA vaccine for a novel virus almost immediately after we get the sequence. Knowing the sequence beforehand will not give us much of a head start

    • Something many people don’t know is that we are now capable of designing an mRNA vaccine in a day—Moderna’s vaccine famously took less than 48 hours to design, and a similar vaccine would likely take a day to design by now

      • Most of the ~340 day gap between the publication of the COVID genome and the approval of the vaccine was due to testing, safety, and regulation

    • NIH is working on developing one vaccine for every viral family—this will help us learn what the right, safe dose for an mRNA vaccine against each family, and will allow us to run phase 1 and phase 2 trials simultaneously

    • In addition, we now have significant production capability to build mRNA vaccines quickly and at scale, in a way where a vaccine for a new virus can easily be swapped into the existing assembly line. This would allow us to produce the vaccines needed for a phase 1/​phase 2 trial almost immediately

  • Because of this, investigating the virus before-hand should only give us a day head start—the day involved in designing the vaccine

• Should we go through with the DEEP VZN program, and then build and stockpile vaccines for all pandemic-grade viruses so we can stop the new viruses immediately?

  • The way to stop new viruses before they become a pandemic is ring vaccination, where you vaccinate everyone who could plausibly have been exposed or could plausibly be exposed to the people who have the virus

  • The number of doses needed to perform ring vaccination is not very high, and with the mRNA vaccine production we expect to have in the future, it should be possible to produce those doses extremely quickly on command

  • Producing all the vaccines in advance would be overly expensive, and a better use of that money would be to spend it on increasing mRNA vaccine production capacity to enable us to ramp up quickly in a disaster scenario

• Would knowing which diseases are dangerous allow us to monitor the human-animal interfaces of those diseases better, and give us the ability to stop those diseases from transferring to humans?

  • It’s not clear how much better monitoring works when you know the specifics of a disease. You can monitor the human-animal interfaces well without that knowledge

• While the US can control whether or not DEEP VZN does this work, the US cannot stop the rest of the world from doing it. As an example, wouldn’t it be bad if China performed similar work under the cloak of darkness, and then there would exist an information asymmetry?

  • It’s unclear what US would have to lose and what China has to gain in this scenario. Pandemics are not strategically useful to great powers in the way nukes are because they can’t be precisely targeted without a risk to your own citizens

  • China would not be able to vaccinate a billion people without intelligence agencies noticing and learning about it. Even if they somehow could, it would be very suspicious when the virus ravages every other country but has no impact on their own citizens. Standard deterrence should be sufficient here

  • Of all countries out there, the US seems to be the hardest to convince not to do this type of work. If we can convince the USAID and NIH, then we may be able to convince the world and set up an international treaty to ban the tiny subset of virology at play here that is the equivalent of nuclear weapons testing

• There is talk about building a pan-coronavirus vaccine, which would provide at least some protection across all coronaviruses. Isn’t it true that building a pan-coronavirus vaccine relies on collecting a large sample of different coronaviruses, which we could get from the type of virus hunting that DEEP VZN does?

  • This is true—you need a sizable sample of coronaviruses to build a pan-coronavirus vaccine. But you do not need to know which coronaviruses could cause pandemics in humans

    • While it might be worth it to do virus hunting to collect a large sample of viruses, it is certainly not necessary, and is actively dangerous, to identify which of the viruses are pandemic-grade and rank-order them by how dangerous they are

Why is USAID going forward with DEEP VZN, despite all these dangers?

• USAID made the important point that pandemics are one of the greatest threats to the poor and vulnerable—and made the further reasonable sounding inference that they can best deter against these pandemics if they know exactly what viruses are the most dangerous

• It turned out that no one ever mentioned that PREDICT, the precursor program to DEEP VZN, could pose a security risk until much later, around when DEEP VZN was announced. These risks were not self-evident even to security professionals, so it is unsurprising that virologists without security training would overlook the risks

• Current USAID leadership inherited this program from previous leadership, and may be amenable to a change in DEEP VZN

• There is some uncertainty around where the DEEP VZN program stands right now, though it likely has not launched yet. At the very least, characterization (identifying which viruses are pandemic-grade) has not yet begun. This raises the chance that funding for the worst parts of DEEP VZN could be shifted to other use cases

• USAID has already done the right thing by stopping virus enhancement research—hopefully they will also recognize that pandemic-grade virus identification is a problem

What can people do to influence USAID?

• Message your elected representatives on this topic—they have staff to field in-bound messages

  • If you are from MD, HI, VA, CT, MA, TN, KY, TX, WI, FL, NJ, or IA, you have a senator on the US Senate Foreign Relations Committee subcommittee overseeing USAID (List of senators on the subcommittee: https://​​www.govtrack.us/​​congress/​​committees/​​SSFR/​​14)

• Contact USAID directly via their online contact box at usaid.gov/​contact-us . Tell them something like:

  • While DEEP VZN could do a lot of good, there is significant security risk around pandemic virus identification. Because of this, USAID should shift money over to disease monitoring.

• You can tweet at the USAID Twitter account: https://​​twitter.com/​​USAID

• If you know heavy hitters within the USAID, reach out to them and share your feelings or link to the podcast

• Spread the message via social media or blogging

[finm]

Just commenting to say this was a really useful resource summarising an important topic — thanks for the time you put into it!