Tara O'Toole, Director of the Johns Hopkins Center for Civilian Biodefense Strategies and co-author of the article, was also the principal author of "Operation Dark Winter" (in 2001-2002) and "Atlantic Storm" (2005), the disaster response exercises that simulated covert outbreaks of smallpox in the United States. She went on to become Under Secretary of the Science and Technology Directorate of the Department of Homeland Security and, on the 10-year anniversary of the "anthrax letters," reiterated her Johns Hopkins group's earlier concerns with synthetic biology in testimony to the Senate Committee on Homeland Security and Governmental Affairs:

More than a decade ago, the Defense Science Board affirmed that, "there are no technical barriers to a large-scale bioattack." We are living in the midst of a biotechnology revolution where the knowledge and tools needed to acquire and disseminate a biological weapon are increasingly accessible. It is possible today to manipulate pathogens' characteristics (e.g., virulence, antibiotic resistance), and even to synthesize viruses from scratch. These procedures will inexorably become simpler and more available across the globe as technology continues to mature (39).

Concerns about terrorist use of DNA synthesis to create biological weapons spread internationally, and synthetic biology has become a regular feature of the science and technology reviews of the international treaty banning biological weapons: the BWC. In one of these reviews for BWC members, the Chinese delegation noted that:

With the spread of synthetic biology, some small scale research groups and even some individuals are now able to make the deadly Ebola and smallpox viruses and even some viruses against which all drugs are ineffective, thus making it much harder to counter bioterrorism. Furthermore, it has become much easier to obtain sensitive information. Using publicly available DNA sequences, terrorists can quickly synthesize pathogenic microbes that had previously been eradicated. [(40), p. 4].

During a 2012 Meeting of Experts of the BWC, the US delegation noted that:

These technologies [enabling technologies, including highthroughput systems for sequencing, synthesizing and analyzing DNA; bioinformatics and computational tools; and systems biology] could potentially be used for purposes contrary to the Convention, including making pathogens or toxins easier and less expensive to manufacture *de novo*, and further into the future, enabling development of biological weapons agents designed to evade countermeasures or target certain human populations [(41), p. 1-2].

Similar concerns have also been highlighted by individual bioweapons experts. Recent examples include Laurie Garrett's9 article in the November/December 2013 issue of Foreign Affairs (42), which was widely disseminated and became the subject of a "Foreign Affairs Focus" video interview with the author published online on 15th January 201410. In this article Garrett asserts that:

All the key barriers to the artificial synthesis of viruses and bacteria have been overcome, at least on a proof-of-principle basis (42).

Another example is the article written by Adam Bernier and Patrick Rose for the CBRNePortal, which states:

Non-state actors who wish to employ biological agents for ill intent are sure to be aware of how tangible bio-weapons are becoming as applications of synthetic biology become more affordable and the probability of success increases with each scientific breakthrough (28).

Synthetic biologists have not sought to deny these risks, and have led several initiatives to consider how these potential biosecurity risks could best be addressed. These initiatives re-enforced the association between synthetic biology, DNA synthesis, and biosecurity threats. For example in his "Synthetic Biohazard Non-proliferation Proposal," George Church stated:

While the likelihood of misuse of oligos to gain access to nearly extinct human viruses (e.g. polio) or novel pathogens (like IL-4-poxvirus) is small, the consequences loom larger than chemical and nuclear weapons, since biohazards are inexpensive, can spread rapidly world-wide and evolve on their own (14).

Similarly, the JCVI report mentioned above concluded that:

today, any synthesis of viruses, even very small or relatively simple viruses, remains relatively difficult. In the near future, however, the risk of nefarious use will rise because of the increasing speed and capability of the technology and its widening accessibility. [. . .] Ten years from now,it may be easier to synthesize almost any pathogenic virus than to obtain it through other means [(15), p. 12–13].

And a group of synthetic biologists (including Drew Endy and George Church) published, together with leading DNA synthesis companies and four FBI staff, a commentary in *Nature Biotechnology* on "DNA synthesis and biological security," which stated that:

9Laurie Garrett is a science writer with a special interest in emerging infectious diseases, global health and biosecurity. She works at the Council on Foreign Relations Council, a think-tank that publishes the journal *Foreign Affairs*.

10http://www.foreignaffairs.com/discussions/audio-video/foreign-affairs-focuslaurie-garrett-on-synthetic-biology

Like any powerful technology, DNA synthesis has the potential to be purposefully misapplied. Misuse of DNA-synthesis technology could give rise to both known and unforeseeable threats to our biological safety and security [(43), p. 627].

#### **Challenges to Myth 3**

When speaking about DNA synthesis, it is useful to distinguish between (a) the synthesis of oligonucleotides, commonly referred to as "oligos," which are typically less than 100 nucleotides in length; (b) "gene synthesis," a term used to refer to the *de novo* synthesis of "gene-length" DNA sequences, typically 200–3,000 base pairs (bp); and (c) the assembly of *de novo* synthesized gene-length fragments into genetic circuits and whole genomes.

There are a number of ways in which DNA synthesis could be used to create a synthetic viral genome [(44), p. 134]. An entire viral genome could be ordered online from a commercial gene synthesis company. Short, single stranded oligonucleotides could also be ordered from different gene synthesis companies and "stitched" together to create a complete viral genome. Alternatively, oligonucleotides could be synthesized using a purchased or custom-built DNA synthesizer, and these fragments could then be assembled into a complete viral genome. Several challenges should be taken into account when assessing the potential for this technology to be misused.

Ordering short oligos and then assembling them into a genome was the method used in the polio and Spanish flu experiments, but this required specialist expertise, experience, and equipment, which were all available in the academic laboratories involved but would not be easily accessible to an amateur working from home. Obtaining the oligos (as was done by *The Guardian* journalist for the smallpox virus) is only the first step in a complicated process. This is the first challenge to Myth 3.

The second challenge to Myth 3 is that,contrary to what is stated in Wikipedia, and what is often implied in the policy discourse described above, even specialized DNA synthesis companies cannot easily synthesize *de novo* any desired DNA sequence. Several commercial companies provide routine gene synthesis services for sequences under 3,000 bp, but length is a crucial factor, the process is error prone, and some sequences are recalcitrant to chemical synthesis (those that are"complex,"have high GC content, or result in the expression of particular proteins when cloned). Thus, in a recent review of large-scale *de novo* DNA synthesis, Kosuri and Church conclude that:

Today, reconstructions of complete viral and bacterial genomes are testaments of how far our synthetic capabilities have come. Despite the improvements, our ability to read DNA is better than our ability to write it [(45), p. 499].

The polio and phi174 viruses both have relatively small genomes, but these are still 7,400 and 5,400 bp, respectively. Thus, several *de novo* synthesized DNA fragments would have to be assembled in order to produce a full genome and (even if this was not already regulated by voluntary guidelines adopted by DNA synthesis companies) it would not be possible to simply order the full-length genome sequence of a small virus online.

The third challenge is that for sequences longer than 5–10 kb, *assembly* of DNA fragments becomes the crucial step, not *de novo*

DNA synthesis. This was the major technological feat in the work conducted at the JCVI that produced the "synthetic" bacterial genome, and the "Gibson Assembly method" developed for that project is now widely used. The description of that work, however, demonstrates how the assembly of smaller fragments into larger ones and eventually into a functioning genome required substantial levels of expertise and resources, including those needed to conduct trouble-shooting experiments to identify and correct errors when assembled DNA constructs did not perform as expected (46).

The fourth challenge to Myth 3 relates to cost. The price of gene synthesis has declined greatly over the last 20 years, and the policy discourse that underlies biosecurity fears often implies that it will naturally become even cheaper over time, and thus widely affordable. The decline in price has, however, more or less stagnated around 0.3 US\$ per base pair since 2008; and Carlson (47), Kosuri and Church (45), and Shetty (48) each discuss reasons why investment in this area may not be sufficient or well directed enough to generate further significant advances.

The fifth and fundamental challenge to Myth 3 is that constructing a genome size DNA fragment is not the same as creating a functional genome. In particular, ensuring the desired expression of viral proteins is a complex challenge, which has been well documented in Vogel's (5) account of the 2002 poliovirus synthesis experiment. Drawing on interviews with the researchers involved in the experiment,Vogel found that making HeLa cell-free extracts was a crucial step in translating the synthetic genome into infectious virus particles; and it was also one of the most difficult parts of the experiment. Successful preparation of the HeLa cell-free extracts depended on craft-like techniques that require specialized and localized know-how. Yet, as Vogel notes, despite the difficulties encountered in this step of the process, published protocols of the experiment give no indication of this contingency:

As this case study illustrates, successful replication of the published 2002 poliovirus experiment hinges not only on the availability of the genetic sequence of the virus, commercial pieces of DNA, or the posting of the publication on the internet but also on the ability to master the mundane yet idiosyncratic biological techniques and adhere to specific laboratory disciplines [(5), p. 86].

Published accounts of science imply that experiments are readily replicable and transferrable from one lab to the next, but Vogel's analysis demonstrates the significance of tacit knowledge in scientific practice and how this would limit the "proliferation" of skills anticipated in the dominant narrative on synthetic biology. Recognizing the importance of such tacit knowledge would enable more refined analyses of the potential biosecurity threat posed by advances in DNA synthesis technologies.

Additional challenges to Myth 3 include the fact that while DNA or RNA sequence data are available for many pathogenic viruses, genomes published in publicly available databases can contain errors or may be derived from attenuated laboratory strains (49). Producing viral particles in a laboratory is, moreover, not the same as creating and deploying an effective biological weapon. Challenges to the processes of scaling up, storage, and developing a suitable dissemination method are discussed under Myth 5.

#### **MYTH 4**

## **Synthetic biology could be used to design radically new pathogens**

In addition to recreating dangerous viruses, concerns have also been expressed that synthetic biology could be used to enhance the virulence or increase the transmissibility of known pathogens in order to create novel threat agents.

The 2001 mousepox experiment is the most widely cited examples of the dual-use potential of life science research and has come to epitomize the potential to create more virulent viruses. In this experiment, researchers inserted the gene for interleukin-4 into the mousepox virus (50). They aimed to produce an altered virus that would induce infertility in mice and serve as an infectious contraceptive for pest control. However, the altered virus was found to be lethal to mice. Moreover, and most surprisingly, it was lethal to mice that were naturally resistant to mousepox as well as to mice that had been recently immunized against ordinary mousepox. The publication of these findings led to concerns that they could provide instructions to terrorists to produce novel biological weapons.

An early, formative report that shaped concerns about radically new pathogens was *Biotechnology Research in an Age of Terrorism* from the US National Research Council. It noted:

The effects of naturally occurring pathogens are limited by the evolutionary advantage gained by not eliminating their hosts. Among the many implications of the anticipated progress in biotechnology is the presumption that it may be feasible to create novel biological agents that are far more predictable and dangerous than any of the naturally occurring pathogens that have been developed as biological weapons in the past. It may be difficult to engineer a more successful pathogen than those already present in nature that have been perfected by evolution for their niche in life. However, application of the new genetic technologies makes the creation of "designer diseases" and pathogens with increased military utility more likely [(51), p. 25].

These concerns have been echoed in a number of other high profile reports. For example, the very first European Commission report dedicated to synthetic biology, published in 2007, stated that:

The possibility of designing a new virus or bacterium "*à la carte*" could be used by bioterrorists to create new resistant pathogenic strains or organisms, perhaps even engineered to attack genetically specific sub-populations [(52), p. 18].

A 2012 report from United Nations Interregional Crime and Justice Research Institute (UNICRI) raised similar concerns:

Experts felt that as an enabling tool, synthetic biology [. . .] would in the long term likely facilitate the work of those attempting to acquire and use biological weapons. More dangerous and controllable pathogens could be engineered that lead to novel possibilities in designing bioweapons. Advances in modeling could enable improvements in weapons design. Metabolic engineering might confer new qualities and attributes upon agents and offer options for new types of weapons. [. . .] This could have the negative effect of making bioweapons cheaper and easier to acquire, making their use eventually more likely; more reliable and controllable, making them more desirable; and more effective, increasing their potential impact [(53), p. 34].

These concerns are also evident in the statements made by the Chinese and US delegations in the BWC reports identified under Myth 3.

Influential experts have also highlighted concerns about"superpathogens," for instance Marc Collett, a virologist who was commissioned by the JCVI to provide advice for their work on the risks and benefits of synthetic genomics, concluded that:

While nature has provided would-be terrorists an ample supply and selection of quite virulent viruses, there is concern that genetic technologies will be used to modify these already pathogenic agents and create "super-pathogens," viruses that are more lethal and disruptive than naturally occurring pathogens, and that are designed to evade vaccines or to be resistant to drugs [(54), p. 95].

Maurer and Zoloth, in the article mentioned above, similarly stated that:

Synthetic biology's efforts to reprogram life have raised concerns in some quarters that the technology could one day be used to make radically new weapons, such as pathogens that could be narrowly targeted towards populations with known genetic susceptibilities [(37), p. 16].

Laurie Garrett, in her 2013 article for *Foreign Affairs*, raised her concerns as follows:

a simple, ubiquitous microbe such as *E. coli*, a bacterium that resides in the guts of every human being, can now be transformed into a killer germ capable of wreaking far more havoc than anything on [the US National Select Agent] registry (42).

The 2011–2012 controversy over publication of H5N1 "bird-flu" research also centered on concerns that the published research would provide "blueprints" to terrorists to create highly virulent viruses with increased transmissibility. H5N1 does not spread easily from human to human, but it kills between 30 and 80% of people infected (55). In this experiment, researchers in the Netherlands and the US independently developed a novel strain of the H5N1 avian influenza virus that could spread more easily to humans and other mammals. They passed H5N1 among ferrets and found that a mutated H5N1 virus that was air transmissible could emerge, and that this variant was still highly virulent. When two papers relating similar experimental results were submitted for publication to *Science* and *Nature*, concerns were raised about the dual-use risk and the NSABB recommended against full publication of the study. After additional consultations at the World Health Organization, the NSABB reversed its position and recommended publication of revised versions of the papers (56).

#### **Challenges to Myth 4**

The mousepox and H5N1 experiments are frequently cited to demonstrate how dangerous new pathogens could be designed. However, assessments of this threat tend to overlook the fact that, in both these experiments, the researchers did not actually *design* the pathogens. With respect to H5N1, researchers had indeed been trying to design an air-transmissible virus variant for some time, without success. The ferret experiment was set up as an alternative approach, to see whether "natural" mutations could generate an air-transmissible variant. The researchers had no influence on the specific mutations induced. In the IL4 mousepox experiment, the results were unanticipated by the researchers. In other words, they were not planned for.

Moreover, some of the key lessons that came out of the extensive Soviet program to weaponize biological agents were about the trade-offs between improving characteristics that are "desired," in the context of a bioweapons program, such as virulence, and diminishing other equally "desired" characteristics, like transmissibility or stability. One project, for example, aimed to develop strains of *F. tularensis* (which causes tularemia) that were resistant to current vaccines and to multiple antibiotics. Genes coding for antibiotic resistance were successfully transferred into *F. tularensis*, but the new strain lost its virulence. Domaradsky, who led the research, wrote:

Everyone who has ever dealt with the genetics of bacteria knows how complicated it is to produce a new strain, indeed, to create a new species! [quoted in (57), p. 186)].

The Soviets did, however, eventually succeed in developing a strain of *F. tularensis* that was resistant to multiple antibiotics and retained its pathogenic characteristics. They also worked on four additional bacterial strains – *B. anthracis* (which causes anthrax), *B. mallei* (glanders)*, B. pseudomallei* (melioidosis), and *Y. pestis* (plague) – with the goal of making each of them resistant to 10 antibiotics, but this proved too technically difficult. As Leitenberg and Zilinskas note in their account of the process:

The most difficult problems had to do with pleiotropic effects and a lack of stability in engineered strains. Antibioticresistant cells had a distressing habit of losing virulence or exhibiting lesser yields (or both) when propagated in culture. As for stability [. . .] when the construct for resistance to one antibiotic was introduced into the host cell, an earlier emplaced construct was often lost. This sort of problem required additional rounds of research,which were both labor intensive and time consuming [(57), p. 188].

Pleiotropic effects (where a single gene affects more than one characteristic) and genetic instability are common in microorganisms, and while it is too simple to say that increased transmissibility will always be associated with reduced virulence, this is often the case for strains produced in laboratories. In the case of viruses, this is in part because the production of virus molecules necessitates passage through a series of host organisms, and that during this scaling-up process the virus is not subject to any evolutionary pressure to maintain virulence, and thus – although this cannot be taken as a definitive rule – the virus tends to accumulate mutations that generate an attenuated strain. Similarly, bacteria cultured in laboratories will tend to lose virulence.

#### **MYTH 5**

#### **Terrorists want to pursue biological weapons for high consequence, mass casualty attacks**

Underlying the first four myths are certain assumptions about who the terrorists might actually be, what their intentions are, what capabilities they might pursue, and the level of skills and resources available to them. Despite a lack of analysis of the potential adversaries involved in the misuse of life science research, the bioterrorism threat has generally been portrayed in policy circles as an imminent concern, and emphasis is placed on high consequence, mass casualty attacks, performed with "weapons of mass destruction" (WMD).

For example, in one of the President George W. Bush's earliest statements following 9/11 and the "anthrax letter" attacks that drew the American people's attention to the biological weapons threat, he said:

Since September 11, America and others have been confronted by the evils these [biological] weapons can inflict. This threat is real and extremely dangerous. Rogue states and terrorists possess these weapons and are willing to use them (58).

Later, he set up a WMD Commission and tasked it with examining the threat posed by the nexus of international terrorism and the proliferation of weapons of mass destruction. In its report, this Commission asserted:

Unless the world community acts decisively and with great urgency, it is more likely than not that a weapon of mass destruction will be used in a terrorist attack somewhere in the world by the end of 2013. The Commission further believes that terrorists are more likely to be able to obtain and use a biological weapon than a nuclear weapon. The Commission believes that the U.S. government needs to move more aggressively to limit the proliferation of biological weapons and reduce the prospect of a bioterror attack [(59), p. xv].

Bioterrorism became one of the Bush Administration's key security concerns over its two terms in office. One estimate of civilian biodefense expenditure across the federal government since 2001 is that more than \$70 billion have been spent (60). Despite this, on the 10-year anniversary of 9/11 and the "anthrax letter" attacks, the former US senators who chaired the WMD Commission, Bob Graham and Jim Talent, released a "report card" on America's bioresponse capabilities that concluded the US was still unprepared to respond to large-scale biological attacks. It also warned:

Naturally occurring disease remains a serious biological threat; however, a thinking enemy armed with these same pathogens — or with multi-drug-resistant or synthetically engineered pathogens — could produce catastrophic consequences. A small team of individuals with graduate training in several key disciplines, using equipment readily available for purchase on the Internet could produce the type of bioweapons created by nation-states in the 1960s. Even more troubling, the rapid advances in biotechnology, such as synthetic biology, will allow non-state actors to produce increasingly powerful bioweapons in the future [(61), p. 11].

We see here how the myths we previously discussed, about deskilling and increased access, and about the ease of designing new dangerous pathogens, underlie concerns about terrorists' potential ability to launch a mass attack, and how these are connected, by actors, with the advent of synthetic biology.

The senators were not alone in their assessments. For instance, the US Senate Majority Leader Bill Frist made a similar warning in an earlier speech outlining the global threat of infectious disease and bioterrorism, and the need to better prepare the US and the world to respond to epidemics and outbreaks:

No intelligence agency, no matter how astute, and no military, no matter how powerful and dedicated, can assure that a few technicians of middling skill using a few thousand dollars worth of readily available equipment in a small and apparently innocuous setting cannot mount a first-order biological attack . . . Never have we had to fight such a battle, to protect so many people against so many threats that are so silent and so lethal (62).

Similar messages were reinforced at the highest level. Addressing BWC members at their five-yearly meeting in 2011, Secretary of State Hillary Clinton said:

The advances in science and technology make it [. . .] easier for states and non-state actors to develop biological weapons. A crude, but effective, terrorist weapon can be made by using a small sample of any number of widely available pathogens, inexpensive equipment, and college-level chemistry and biology (63).

She also acknowledged, however, that not everyone in the international community shared the US assessment:

I know there are some in the international community who have their doubts about the odds of a mass biological attack or major outbreak. They point out that we have not seen either so far, and conclude the risk must be low. But that is not the conclusion of the United States, because there are warning signs, and they are too serious to ignore (63).

The belief that the focus should be on mass attacks was bluntly stated by an FBI agent at a symposium on synthetic biology this year (1st May), when she warned: "These technologies do not just pose a risk to individual buildings or cities, but if cleverly deployed, can reduce our population by significant percentages" (64).

## **Challenges to Myth 5**

There are two dimensions to Myth 5. The first is about the intention of would-be terrorists, and the assumption is that terrorists would seek to produce mass casualty weapons and pursue capabilities on the scale of twentieth century state-level bioweapons programs. While most leading biological disarmament and nonproliferation experts believe that the risk of a small-scale bioterrorism attack is very real and very present, they consider the risk of sophisticated large-scale bioterrorism attacks to be very small (65). This is backed up by historical evidence. The three confirmed attempts to use biological agents against humans in terrorist attacks in the past were small-scale, low casualty events aimed at causing panic, and disruption rather than excessive death tolls: (i) the Rajneesh cult's use of *Salmonella* on salad bars in local restaurants to sicken potential voters and make them stay away from the polls during Oregon elections in 1984; (ii) the 1990–95 attempted use of botulinum toxin and anthrax by the Japanese Aum Shinrikyo cult; (iii) and the "anthrax letters" sent to media outlets and members of US Congress in 2001 resulting in at least 22 cases of anthrax, five of which were fatal (66, 67).

The second dimension to Myth 5 is the implicit assumption that producing a pathogenic organism equates producing a weapon of mass destruction. It does not. Considerable knowledge and resources are necessary for the processes of scaling up, storage, and developing a suitable dissemination method. These processes present significant technical and logistical barriers. Drawing from her in-depth study of the Iraqi, Soviet, and US bioweapons programs (3, 4), Ben Ouagrham-Gormley explains:

Scaling up fragile microorganisms that are sensitive to environmental conditions and susceptible to change — and viruses are more sensitive than bacteria — has been one of the stiffest challenges for past bioweapons programs to overcome, even with appropriate expertise at hand. Scaling-up requires a gradual approach, moving from laboratory sample, to a larger laboratory quantity, to pilot-scale production, and then to even larger-scale production. During each stage, the production parameters need to be tested and often modified to maintain the lethal qualities of the agent; the entire scaling-up process can take several years (68).

The dissemination of biological agents also poses difficult technical challenges. Whereas persistent chemical agents such as sulfur mustard and VX nerve gas are readily absorbed through the intact skin, no bacteria and viruses can enter the body via that route unless the skin has already been broken. Biological agents must either be ingested or inhaled to cause infection. To expose large numbers of people through the gastrointestinal tract, possible means of delivery are contamination of food and drinking water, yet neither of these scenarios would be easy to accomplish. Large urban reservoirs are usually unguarded, but unless terrorists added massive quantities of biological agent, the dilution effect would be so great that no healthy person drinking the water would receive an infectious dose (66). Moreover, modern sanitary techniques such as chlorination and filtration are designed to kill pathogens from natural sources and would probably be equally effective against a deliberately released agent. Bacterial contamination of the food supply is also unlikely to inflict mass casualties. Cooking, boiling, pasteurization, and other routine safety precautions are generally sufficient to kill pathogenic bacteria.

The most likely way to inflict mass casualties with a biological agent is by disseminating it as a respirable aerosol: an invisible cloud of infectious droplets or particles so tiny that they remain suspended in the air for long periods and can be inhaled by large numbers of people. A high-concentration aerosol of *B. anthracis* or some other pathogen, released into the air in a densely populated urban area, could potentially infect thousands of victims simultaneously. After an incubation period of a few days, depending on the type of agent and the inhaled dose, the exposed population would experience an outbreak of an incapacitating or fatal illness. Although aerosol delivery is potentially the most lethal way of delivering a biological attack, it involves major technical hurdles that most terrorists would be unlikely to overcome. To infect through the lungs, infectious particles must be microscopic in size – between 1 and 5µm in diameter. Terrorists would therefore have to develop or acquire a sophisticated delivery system capable of generating an aerosol cloud with the necessary particle size range and a high enough agent concentration to cover a broad area. Overall, an important trade-off exists between ease of production and effectiveness of dissemination. The easiest way to produce microbial agents is in a liquid form, yet when such a "slurry" is sprayed into the air, it forms heavy droplets that fall to the ground so that only a small percentage of the agent is aerosolized. In contrast, if the bacteria are first dried to a solid cake and then milled into a fine powder, they become far easier to aerosolize, yet the drying and milling process is technically difficult.

The Aum Shinrikyo cult struggled with dissemination (67, 69, 70). In one of its anthrax dissemination attempts, it sprayed unknown, but probably very large, quantities of a liquid aerosol (most likely crude culture, unprocessed in any way) of *B. anthracis* from the roof of the Aum's headquarters building in Tokyo. For the dissemination, the Aum set up two sprayers on the roof of the eight-story building,each within a large round cooling tower. Pipes were extended from the cooling towers to tanks below, which were filled with a liquid suspension of *B. anthracis*. The device worked poorly, producing large droplets rather than the very fine aerosol needed for effective transmission of anthrax. It also appears the spore concentration was very low (at least five orders of magnitude below that necessary for a highly infectious wet aerosol).

In another dissemination attempt, targeting the area around the Kanagawa prefectural office and the Imperial Palace, the Aum equipped vehicles with spraying devices, but according to prosecutors' statements, the nozzle of the sprayer clogged and the operation failed. Despite its 200 m2 laboratory containing, amongst other equipment, a glove box, incubator, centrifuge, drier, DNA/RNA synthesizer, electron microscope, two fermenters each having about a 2,000 litre capacity, and an extensive scientific library, and despite its repeated attempts at dissemination, the Aum was unsuccessful in causing any disease, and in retrospect it is clear that the cult did not even make the first substantive step toward an effective bioweapon.

If, despite the odds, aerosolization was achieved, the effective delivery of biological agents in the open air is highly dependent on atmospheric and wind conditions, creating additional uncertainties. Only under highly stable atmospheric conditions would the aerosol cloud remain close to the ground where it can be inhaled, rather than being rapidly dispersed. Moreover, most microorganisms are sensitive to ultraviolet radiation and cannot survive more than 30 min in bright sunlight, limiting their use to night-time attacks. One major exception is anthrax, which can be induced to form spores with tough outer coats that enable them to survive for several hours in sunlight. Terrorists could, of course, stage a biological attack inside an enclosed space such as a building, a subway station, a shopping mall, or a sports arena. Such an attack, if it involved a respiratory aerosol, might infect thousands of people, but even here the technical hurdles would by no means be trivial.

Finally, even if a biological weapon had been disseminated successfully, the outcome of an attack would be affected by factors like the health of the people who are exposed to the agent, and the speed and manner with which public health authorities and medical professionals detected and were able to respond to the resulting outbreak. A prompt response with effective medical

countermeasures, such as antibodies and vaccination, can significantly blunt the impact of an attack. Simple, proven ways to curtail epidemics, such as wearing face masks, hand washing, and avoiding hospitals where transmission rates might soar, can also prove effective in stemming the spread of a disease. Indeed, this aspect of a bioterrorism attack is often underplayed in scenarios like Tara O'Toole's "Dark Winter" and "Atlantic Storm," where the rates of contagion used are often significantly higher than those in historical cases of natural outbreaks (71).

## **DISCUSSION**

We have identified a number of assumptions that underlie policy discourse on the biosecurity threat posed by synthetic biology. We characterize these assumptions as "myths" that pervade discussion on this issue and have identified important challenges to those myths. In particular, we argue that the myths overlook significant difficulties faced when seeking to design and/or produce a pathogen because they focus mostly on material features, thus missing important socio-technical factors, such as tacit knowledge. We have also shown that this dominant narrative underestimates a crucial step needed to mount a terrorist attack, especially a mass attack: the need to produce weapons, not just pathogens. Thus, we conclude that the five myths that recur in the dominant narrative embody misleading assumptions about both synthetic biology and bioterrorism.

The purpose of identifying and challenging these "myths" is not to dismiss the threat of a bioweapons attack. Of course, it is prudent to take measures to prepare against the possibility of a biological weapons attack and concerted action across a policy continuum that extends from prevention through preparedness to consequence management is necessary. However, as we have demonstrated, any bioterrorism attack will most likely be one using a pathogen strain with less than optimal characteristics disseminated through crude delivery methods under imperfect conditions, and the potential casualties of such an attack are likely to be much lower than the mass casualty scenarios frequently portrayed. This is not to say that speculative thinking should be discounted as it can, in some policy contexts, be helpful to represent possible, though not necessarily probable, future scientific developments, in order to encourage thinking on long-term security challenges. However, problems arise when these speculative scenarios for the future are distorted and portrayed as scientific reality in the present, which, as this paper demonstrates, has occurred in policy narratives related to synthetic biology and biosecurity.

We have shown that much of the debate in policy forums about the biosecurity threat of synthetic biology is based on naïve and simplistic interpretations of synthetic biology's ability to "make biology easier to engineer," and in particular on the misleading assumption that the skills and knowledge necessary to perform synthetic biology will necessarily become accessible to people with no specialist expertise working outside professional scientific institutions, including hostile actors who would seek to misuse the technology to develop biological weapons.

In order to understand why such myths develop and persist, it is important to consider the role that they play in the social dynamics of synthetic biology. Drawing on the literature in the sociology of expectations (72), we suggest that particular portrayals of synthetic

biology are mobilized by various actors – deliberately or not – to strengthen their own perspectives and interests, and to help bring into being their own "hoped-for" future. The myths act as "prospecting retrospects": prospects that are deployed in the realtime now, in order to construct particular futures (72). Discourses about the future are *performative*, meaning that they "perform" functions (they "do work") and are also *relational*, meaning that they bind together and enroll actors and other resources into networks (73). Thus, discourse is "wishful enactment" not just "wishful thinking" (74).

With respect to synthetic biology, different communities of actors stress particular issues in particular contexts. This frames the debate in particular ways and plays an important role in constructing and maintaining resources and support for each of these communities. For example, scientists such as Rob Carlson, George Church, or Drew Endy, who are heavily engaged in the promotion of synthetic biology, need to portray an optimistic vision of the potential of the engineering approach to biology as part of their endeavors to develop support for a new field of research which they believe has great significance and potential. Actors in the security field (including some policy makers, social scientists, and natural scientists) play a different role and often exaggerate the "dual-use threat" in order to attract attention and resources to their own work. Researchers from our own field of science and technology studies (STS) are not immune from such processes: we will generally seek to emphasize the complexity of real world situations and the importance of social dimensions of science, in order to justify the need for our expertise. However, at least until now, STS framings have had less influence on the dominant narrative than the discourse mobilized by actors from the fields of synthetic biology and biosecurity. Thus, the myths we have discussed in this paper have played an important role in defining synthetic biology as a "promissory" field of research and as an "emerging science and technology" in need of scenario forecasting, regulation and governance. Our aim is not to denigrate the behavior of those who deploy these narratives. Rather, we suggest that when discourse is understood as something that seeks to change the social world, we can move beyond the battle that we have regularly encountered in discussions about synthetic biology, that focuses on whose prognosis is most accurate and whether or not "it is just hype" (19, 20).

We believe that a better understanding and acknowledgment of the social dynamics at play would help to develop more productive discussions in which the different communities involved could move beyond simply promoting their own interests and perspectives. This is important because in some cases the discourse deployed can have unintended consequences that are detrimental to the interests of the actors themselves, and to the nature of public debate. Thus, overstating the"promise"of synthetic biology applications manifestly leads to parallel overstatements about the "perils" of the field: the promissory discourse of synthetic biology is bolstered by the "promised peril" of misuse by malevolent actors. The fact that these myths (or at least the first 4) serve to bolster the positive promises of synthetic biology helps to explain why these myths continue to persist, despite the fact that they do not accurately reflect current or foreseeable realities for the practice of synthetic biology. This is somewhat incongruous since the hoped-for futures of the actors who promote the benevolent development of synthetic biology do not, of course, include large-scale fatal bioterrorist attacks.

If we are to disentangle synthetic biology and biosecurity concerns, and to have a more refined assessment of both the biosecurity threat and the anticipated benefits, we believe that it is necessary to have more nuanced discussions about the extent to which synthetic biology is, or ever will be, an engineering discipline, and whether, in practice, this would reduce the importance of tacit knowledge, specialist expertise of different kinds, collective work, large infrastructures, and organizational factors. Such discussions would need to identify those aspects of the work that would become easier – in the sense that they can, for example, be automated and reliably performed by a robot – and those which are likely to remain difficult, in the sense that they still require craft skills to be successfully achieved. This would need to take into account not only the material and informational aspects but also other important socio-technical dimensions that will shape the development of the field.

## **AUTHOR CONTRIBUTIONS**

The three authors are listed in alphabetical order and have all contributed equally to this paper, including the conception, analysis and data collection of the research, drafting the text, and revising it critically for intellectual content.

#### **ACKNOWLEDGMENTS**

This research was supported by two awards from the UK Engineering and Physical Sciences Research Council (EP/G036004/1 and EP/J02175X/1). Filippa Lentzos was supported by a UK Economic and Social Research Council award (RES-070-27-0003). The authors would like to thank all the participants at the workshop held at King's College in February 2014, which has informed this research. We would also like to thank Jane Calvert, Piers Millett, Kathleen Vogel, and Luis Campos for providing useful comments on earlier drafts of this paper.

#### **REFERENCES**

**Conflict of Interest Statement:** This research was conducted at the Centre for Synthetic Biology and Innovation (CSynBI), which is a partnership between synthetic biology researchers at Imperial College London and social scientists at the Department of Social Science, Health and Medicine at King's College London. CSynBI seeks to influence the trajectory of synthetic biology and contribute to the development of an appropriate governance regime.

*Received: 11 May 2014; accepted: 23 July 2014; published online: 21 August 2014. Citation: Jefferson C, Lentzos F and Marris C (2014) Synthetic biology and biosecurity: challenging the "myths". Front. Public Health 2:115. doi: 10.3389/fpubh.2014.00115 This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Jefferson, Lentzos and Marris. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# PUBLIC HEALTH

## Dual-use research as a wicked problem

## **Gregory D. Koblentz\***

School of Policy, Government, and International Affairs, George Mason University, Fairfax, VA, USA \*Correspondence<EMAIL_ADDRESS>
**Edited by:** Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Kenneth A. Oye, Massachusetts Institute of Technology, USA

**Keywords: dual-use research of concern, biosecurity, wicked problems, H5N1, bioterrorism**

The challenge of dual-use research in the life sciences emerged vividly in 2011 as scientists and policy-makers debated what to do about article manuscripts that described how to modify the H5N1 avian influenza virus so that it could spread between mammals (1, 2). Since H5N1 emerged in Southeast Asia in 2003, it has sickened 667 people and caused 393 human deaths, as well as the deaths of millions of domestic and wild birds (3). The virus has not, however, demonstrated the ability to engage in sustained humanto-human transmission. If a new strain of H5N1 emerged with that capability, and it retained a high level of virulence, it could cause a global pandemic. The experiments by Yoshihiro Kawaoka from the University of Wisconsin-Madison and Ron Fouchier from Erasmus Medical Center in the Netherlands not only demonstrated that mammalian transmission of the virus was possible but also provided information on how to construct such a virus.

These H5N1 experiments are only the latest demonstration of the dual-use dilemma at the heart of the biotechnology revolution: research conducted for peaceful purposes has the potential to be misused for malicious purposes. The H5N1 controversy highlighted the widely divergent views on the benefits and risks of dualuse research held by different stakeholders, including scientists, publishers, biosecurity experts, the national security community, and public health officials. On the one side, proponents of the research focused on the public health benefits of knowing that H5N1 can be transmitted between mammals and which specific mutations can confer this ability on the virus. Opponents of the research highlighted the risks of a laboratory accident and the potential for a

nefarious actor such as a terrorist group or rogue scientist to replicate the research and deliberately release the virus.

The concept of wicked problems provides a new lens for understanding the public policy challenges posed by dualuse research. This concept was first introduced in the 1970s to describe the challenges posed by poverty, urban development, and other social issues (4). Wicked in this context does not mean evil or cool, but instead refers to the intrinsic properties of an issue that make it resistant to long-lasting solutions.

Wicked problems are characterized by multiple, overlapping subsets of problems and high levels of social complexity driven by the number and diversity of players involved in problem-solving. The parties who have a vested interest in how (or whether) the problem is solved are likely to come from different organizations and disciplines with different values and objectives so they will define the problem and acceptable solutions differently. The complex interactions between interconnected issues and the diversity of stakeholder preferences impede the wide acceptance of a definitive statement of the problem. As a result, wicked problems tend to defy traditional linear methods of problemsolving, which rely on a clear specification of the problem to drive the data collection and analysis process. Furthermore, the environment in which stakeholders are trying to solve a wicked problem is dynamic. The constraints on the solution, such as availability of resources and political ramifications, change over time, and stakeholders enter and exit the problemsolving process, change their preferences, or otherwise change the rules by which they address the problem. Since there is

no definitive statement of the problem, there can be no definitive solution. As a result, the problem-solving process ends only when stakeholders run out of time, money, or energy, not when the perfect solution emerges. In addition, solutions to wicked problems are at high risk of having unanticipated effects. Wicked problems are never permanently solved since solutions have implications for other policy domains, which can generate feedback loops or have unintended consequences. The potential for this type of ripple effect increases the scope of stakeholders affected directly or indirectly by policy-making and creates the need for a wider array of information from a broader range of sources to identify the universe of potential solutions and their costs and benefits. In sum, wicked problems are "ambiguous, fluid, complex, political, and frustrating as hell." [(5): p. 2].

Based on these characteristics, dualuse research has all of the signs of being a wicked problem. As two congressional researchers (6) wrote about the H5N1 controversy:

The current issues under debate cut across traditional policy areas, involving simultaneous consideration of security, scientific, health, export, and international policy. Because of the complexity of these issues, analysis according to one set of policy priorities may adversely affect other policy priorities (p. 24).

While wicked problems defy easy and long-lasting solutions, there are several strategies that can be used to manage them. The choice of strategy is dictated by two factors: how concentrated or dispersed power is among stakeholders and how strongly stakeholders struggle for power

amongst themselves (5). Based on these criteria, four coping strategies for wicked problems can be identified: authoritative, hegemonic, competitive, and collaborative (see **Table 1**).

Stakeholders following an authoritative strategy cede the authority to define and solve the problem to a small group of experts. Reducing the number of stakeholders involved in decision-making simplifies and speeds up the process. The use of experts also increases the perceived objectivity, and therefore, legitimacy of the outcome. A drawback to this strategy is that even experts can be wrong or have too narrow of a view (5).

The George W. Bush Administration initially employed an authoritative strategy to address dual-use research. In 2002, the National Academy of Sciences was commissioned to provide recommendationsfor how to balance the costs and benefits posed by dual-use research. As a result of this study, the Bush administration created the National Science Advisory Board for Biosecurity (NSABB) to advise the government on dual-use research oversight. Between 2005 and 2012, NSABB was at the forefront of dual-use research oversight, education, and outreach activities.

When one party is so powerful that it is able to impose its preferred problem definition and solution on other stakeholders, it can employ a hegemonic strategy. While other stakeholders may disagree with the way a problem is defined or solved, the hegemonic strategy simply excludes them from the decision-making process. The main advantage of this strategy is its speed and simplicity: problem-solving by decree. The major disadvantage of this approach is that it is more likely to try to "tame" a wicked problem than actually solve it.

In late 2004, spurred by fears that recent breakthroughs in gene synthesis technology could be used to create dangerous pathogens from scratch, the U.S. Congress made it illegal to synthesize the variola virus,which was defined as"a virus that can cause human smallpox or any derivative of the variola major virus that contains more than 85% of the gene sequence" of variola (7). Scientists objected that the seemingly precise language of the new law could potentially cripple research on smallpox vaccine and other orthopoxviruses since all of these viruses are closely related. This hegemonic strategy was a heavy-handed attempt to"tame" the problem posed by the growing sophistication of synthetic biology by simply outlawing a specific use of the technology.

When power is dispersed and contested, stakeholders view problem-solving as a zero-sum game. Stakeholders pursue a competitive strategy to consolidate their own power in order to define the problem in their preferred way and impose their preferred solution. This strategy can result in more innovative policies due to the struggle by stakeholders to persuade others of their preferred definition and solution. Another advantage of this strategy is that it impedes the centralization of power and creates opportunities for reform when the balance of power among stakeholders shifts. A disadvantage of this strategy is that it is likely to end in stalemate as different stakeholders maneuver to implement their preferred approach and block others from doing likewise (5). The competitive strategy is the default setting for resolving wicked problems in the American political system.

The 2011 controversy over the H5N1 experiments marked a shift from an authoritative to competitive strategy for dealing with dual-use research. The debate over these experiments quickly moved beyond the NSABB and the small community of biosecurity specialists to include the World Health Organization, politicians, scientific publishers, and the scientific community, especially influenza researchers. In an explicit acceptance that the scientific authority of the influenza community was no longer sufficient to shield it from oversight, Dr. Anthony Fauci, director of the National Institutes for Allergies and Infectious Disease (NIAID), told international influenza experts, "The flu

scientific community can no longer be the only players in the discussion of whether the experiments should be done." (8). At the same time, the NSABB's charter was revised to remove its authority to review dual-use experiments and it has not met since late 2012 (9).

Collaborative strategies are best suited for situations where power is dispersed but not contested. Under these conditions, stakeholders can move beyond the zerosum mentality and work together for"win– win" outcomes. This strategy seeks to alter the structure of payoffs to encourage cooperation through repeated iterations to build up trust or create linkages between unrelated issues to expand the potential gains achievable through cooperation. Collaboration can enable stakeholders to achieve results they would not have been capable of reaching on their own and to do so more efficiently. Increasing the number of stakeholders and seeking solutions that are acceptable to as many parties as possible increases transaction costs and delays decision-making.An additional hurdle to collaboration is that each stakeholder brings practice-based "local knowledge" to the table, which is hard to share and difficult for other stakeholders with different identities to internalize (10). Despite these disadvantages, a collaborative strategy has the potential to yield longer lasting policies that are more widely accepted by the relevant stakeholders (5).

Unfortunately, people often have to fail into collaboration. According to Roberts (5),

People have to learn what does not work before they are willing to absorb what they perceive to be the extra 'costs' associated with collaboration. This learning is especially important for people who come from cultures that place a high premium on taking charge, making decisions, being competitive, and using authorities and experts to settle whatever disputes arise (p. 12).

Although the authoritative strategy for addressing the wicked problem posed by dual-use research has now run its course, it is unclear what will replace it. The scientific community views the competitive and hegemonic strategies with a mixture of fear and contempt: contempt for the

push-and-pull of politics that privileges sound bites over the complexities of science and fear of draconian solutions imposed by scientifically ignorant politicians and bureaucrats.

Successful collaboration on dual-use research is more likely to emerge if stakeholders engage in intensive dialog as a means of building a shared understanding about the problem and a shared commitment to solving it. Dialog is not an instrument for decision-making or a negotiating tactic to lead to agreement, but an integral part of the process of creating a shared vision among a diverse group of stakeholders. Getting the right answer is not as important as having stakeholders accept whatever solution emerges (11).

Collaboration can also be facilitated by the emergence of a "collaborative capacity builder" whose role is to ensure the integration of knowledge among stakeholders as part of a long-term strategy to foster a collaborative environment for continuously addressing the dilemmas posed by dual-use research. An individual or organization is empowered to play the role of collaborative capacity builder due to its legal authority, expertise valued by other stakeholders, reputation as an honest broker, or some combination of these values (10).

Recognizing that a problem is wicked is the first step to coping with the problem. Viewing dual-use research as a wicked problem highlights the need for stakeholders to engage in dialog with one another and to adopt collaborative strategies for managing risks in this area. Admitting that the experts do not have all of the answers and giving up the zero-sum view that dominates policy-making in a pluralistic society will be difficult, but the potential benefits of seeking collaborative solutions is well worth the discomfort caused by this mode of problem-solving.

#### **ACKNOWLEDGMENTS**

The author would like to thank the participants in the "Dual Use Dilemma in the Life Sciences" round table at the 2010 International Studies Association Annual Convention. The author would also like to thank Marie Chevrier for her valuable comments.

#### **REFERENCES**

Washington, DC: Congressional Research Service (2013).

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 09 May 2014; accepted: 21 July 2014; published online: 04 August 2014.*

*Citation: Koblentz GD (2014) Dual-use research as a wicked problem. Front. Public Health 2:113. doi: 10.3389/fpubh.2014.00113*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Koblentz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

#### **Keelie Lyn Elektra Murdock 1\* † and David Koepsell 2**

1 Rathenau Institute, The Hague, Netherlands

2 Department of Philosophy, Delft University of Technology, Delft, Netherlands

\*Correspondence<EMAIL_ADDRESS>
†Present address: FueledMinds, St. Catharines, ON, Canada

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Gregory Koblentz, George Mason University, USA David Malet, The University of Melbourne, Australia

**Keywords: principal–agent problems, H5N1, dual-use, science-policy**

### **INTRODUCTION**

When the news broke that Ron Fouchier and his research team at Erasmus Medical Center (MC) in the Netherlands had genetically modified the highly pathogenic avian influenza (HPAI) H5N1 virus and that it had acquired the ability to transmit between mammals, it was a story of scientific discovery and progress and an exciting new development in the international effort to prevent the next pandemic. However, public anxieties and national security concerns would soon become a point of contention between virologists and biosecurity experts in the media and in highly politicized discussions about science-policy. In considering the controversy and the conflicts between scientists and policy-makers, we propose that regarding the situation as a principal– agent problem can yield useful analytical results.

Principal–agent problems occur when two parties that are driven by competing self-interest negotiate the terms of a relationship or contract and act together toward a mutually defined end but an informational asymmetry provides one party (typically the agent) with certain advantages, thus creating tensions. Principal– agent theory provides a template of relational action and the conditional effects of actions in contract situations defined by a functional differentiation, such as between scientists and the government (on behalf of its citizens). Exploring the sciencepolicy nexus from this perspective may further efforts to develop effective policies that address dual-use concerns in the life sciences by offering insights into methods of dispute resolution and the effective design of institutional mechanisms that balance the interests of the parties involved thereby level the playing field.

### **A PRINCIPAL–AGENT PROBLEM?**

Drawing from traditional tools of economic analysis and the theory of rational actors, principal–agent models provide a useful heuristic to explore the behavior of actors and intuitions engaged in contractual relations when there is informational asymmetry and incompletely overlapping or opposing goals. Based on the assumption that the principal requires an agent with a specific skill-set to perform specific actions or functions and both actors enter an agreement to further their interests, the relationship is based on a division of labor. The problem is one of delegation. With imperfect or incomplete information about the interests or the abilities of the applicant, it is possible that the principal will select the wrong agent to pursue its goals and endure the opportunity cost (i.e., adverse selection). If the principal and the agent enter into an agreement, the agent is offered an opportunity to gain from specialization and the informational advantage that it provides, including the conditional authority to act on behalf of the principal, the concomitant financial and professional rewards and some degree of autonomy. The principal must in turn relinquish valued resources and ensure that they receive an adequate return on the investment, in terms of productive labor and output realization.

The central difficulty for principals during the post-contract stage of the relationship, as articulated by Arrow (1), is that "by definition the agent has been selected for his specialized knowledge and the principal can never hope to completely check the agent's performance" (2). The principal must thus bear the risk that the agent will act definitely or in ways that will have consequences for which it will be liable (i.e., moral hazard). The principal also has an incentive to minimize the risk of the transaction by reducing uncertainty and negative externalities (i.e., agency costs). This can be achieved via the strategic introduction of information revelation and generation mechanisms or by offering the agent incentives based on self-interest (such as pay by performance or profit sharing schemes) to create better alignment and ensure better communication and cooperation from the start (3, 4).

#### **RULES OF THE GAME**

As the principal in the relationship between the scientists and the US government, the National Institutes of Health (NIH) retained the Erasmus MC Department of Virology to conduct research in support of the US Department of Health and Human Services (HHS) Pandemic Influenza Plan (5). The terms of the contract between the NIH and the Centers of Excellence for Influenza Research and Surveillance (CEIRS) are defined in the solicitation document and include the provisions for all grant recipients, including foreign institutions, to comply with NIH policies and relevant US regulations (6).

The Erasmus MC Department of Virology was selected to perform research within the CEIRS network because the research proposal defined a problem of mutual interest, in scientific terms as well as in terms that were consistent with the objectives of pandemic preparedness1 . In addition, Ron Fouchier is specialized in the pathogenicity of respiratory viruses and has an established publication record in this particular domain of research expertise, signaling to the scientific community and funding agencies alike his competency in virology. However, the decision to conduct research on the transmissibility of avian influenza was not a decision that followed directly from NIH funding but had been under consideration at Erasmus MC since the initial detection of the virus in 1997 (7). It was thus against a backdrop of scientific uncertainty and questions about an emerging virus that that Fouchier defined his research questions and the methods of experimentation (8). Given some industry affiliations (including patents), an interest in future commercial applications can also be presumed2 .

#### **CONTROVERSY**

At an influenza conference, Dr. Fouchier announced that a "stupid" experiment succeeded in creating an airborne strain of the virus and the result was "very bad news" (9). The media had a field day with the story about what was in his terms, "probably one of the most dangerous viruses you can make" (10). It was soon revealed that Fouchier had submitted a paper to *Science* and intended to openly publish the intricate details. The headlines that ensued expressed strong reactions and objections to the research and the publication, including references from reputable biosecurity experts (11).

Upon review by the US National Scientific Advisory Board for Biosecurity (NSABB), the HHS was advised to request redacted versions of two manuscripts, including the paper co-authored by Ron Fouchier and a similar paper submitted to *Nature* by Yoshihiro Kawaoka from the University of Wisconsin-Madison. According to an NIH Press Statement, the board's recommendations called for key details to be removed to prevent the replication of the experiments "by those who would seek to do harm" (12).

The Chair of the NSABB further clarified that the recommendation was based on the perception that the potential negative consequences of publishing the manuscripts outweighed the benefits. The intention, however, was neither to restrict the dissemination of information to persons with a legitimate need to know nor for the US to dominate what was essentially a global issue. Rather, the US government was also considering a mechanism to enable secured access and would pursue "broad" discussions with "global leaders" on matters of policy, science, and public health (13). This point gave credence to concerns that the limitations would interfere with scientific progress and public health preparedness, particularly the recently established and hard wrought Pandemic Influenza Preparedness (PIP) Framework of the World Health Organization (WHO).

The scientists connected to the contentious studies and the Editor-in Chief of both journals conceded to the request for a redaction but on the condition that further progress would be made on matters of policy. The former imitated a "pause" on H5N1 gain-of-function experiments to buy time for scientists to communicate with the public and policy-makers, for governments to consider policy solutions and for the scientific community to assemble and discuss the issues in an international forum (14). The latter indicated that it is next steps would rest on the US government capacity to share the omitted details (15).

At a "technical consultation" hosted by the WHO in Geneva, it was decided by

consensus that the research was essential, that the papers should be published without restrictions, and that limiting access to the research results was missing a practical vision (16) 3 . The NSABB was thereafter requested on behalf of the US government to review two new manuscripts. Clarifications provided by the authors and "non-public data" discussed at the WHO meeting were named as key factors influencing the NSABB decision to revisit the matter (17).

The NSABB concluded by a majority rule that the publications should be "communicated in full" (18). The consideration of new epidemiological data and classified security information relevant to the riskbenefit calculation and the release of a new United States Government Policy for Oversight of Life Sciences Dual-Use Research of Concern (DURC) informed the discussion and influenced the decision. The moratorium, however, was upheld until the following year as NIH funded scientist awaited pending changes in the funding policy for transmissibility studies (19).

#### **DISCUSSION**

Within constraints of science-policy, the roles of the government and of science are institutionally mediated (20). Governments are appointed to serve the interests of the public and retain scientific information to these ends. Scientists are delegated with the authority to conduct specialized research in pursuit of particular goals and missions. The government is empowered to dictate how the agent should act and define the limits of autonomy. Scientists, however, have an informational advantage because they are on the front lines of knowledge development and have other motivations influencing their decisions. The relationship manifests as a dynamic series of moves taken by the principal and the agent to protect their respective interests, beginning with the negotiation of the contract4 .

1The identification of mutations influencing influenza transmission can facilitate the development of medical countermeasures such diagnostics, vaccines, and drug therapies.

2At the time the paper was submitted, Fouchier was a part-time employee, the Chief Scientific Officer (CSO) and a shareholder in ViroClinics Biosciences, a lab that conducts virology research in support of clinical trials, diagnostics, and medical treatments. Conflicts of interest in this case, however, were avoided by letting the shares to the Stichting Administratiekantoor Erasmus Personeelsparticipaties (a financial institution established to hold shares for Erasmus University staff).

3According to the report, "the group recognized the difficulty of rapidly creating and regulating such a mechanism in light of the complexity of international and national legislation," and thus decided that it was not viable.

4 In scholarly discussions about the dynamics of science-policy, this process is referred to as "boundary-work" and serves to stabilize the boundary between science and politics.

The contract between the NIH and Ron Fouchier aligned their interests on pandemic preparedness. However, different preferences about how the research results should be communicated combined with insufficient incentives from the government can be perceived as creating a moral hazard. Fouchier's colorful description of his research and outspoken views on openness in science landed him in the political hot seat but the risk was not borne by him. Rather, it is in the interest of scientists to increase their visibility within the research community and to publish in prestigious journals because science communication facilitates research progress and because it can improve their career prospects.

Had the NIH thoroughly considered the objectives of the research methods and the implications in relation to existing dual-use concerns or had Fouchier been compelled to be more explicit about the potential outcomes, the controversy and at least some of the agency costs could have been mitigated.

The agency costs include the research delayed by the extension of the moratorium, the drafting of comprehensive changes in government policies and the implementation and performance thereof. The consequences of the monitoring and new bureaucratic rules, however, will also be felt by the scientists engaged in this type of research.

While not a typical adverse selection problem, the international dimension to this problem raises questions about the relationship between principals and foreign agents and whether discrepancies between national dual-use policies and legal requirements will impact future funding decisions.

The Netherlands for instance requested an export license for Fouchier's manuscript, which delayed the publication and complicated the redaction option (21). In addition, the new US government policies introduced selection mechanisms that exclude certain research projects, including those that cannot be openly communicated or conducted in civilian (non-classified) research laboratories. The review and oversight procedures may also provide a disincentive for scientists to pursue particular research projects or seek certain funding opportunities. This may

impact the international marketplace for research grants. Ron Fouchier for instance has claimed that he "would not be silenced by the Americans anymore," and has continued his studies using funding from the European Union (22).

We propose that exploring the complex entanglement of decisions taken by the many principals and agents in this case, including those on the international level such as the WHO, can provide further insight into the tensions between scientists and policy-makers and indicate what can be done to help all parties involved to achieve their goals. The analytical framework can also help alleviate conflicts and prevent similar problems. For instance, if principals and agents have nonaligned interests, and a stand-off may prevail instead of progress. In addition, when incentives are strong, there is essentially no need for monitoring, which can be an ineffective mechanism if it excessive constraint are placed on the agent (23). These issues have worked to the detriment of the relationship between the principal and the agent in the H5N1 case and of the general public, which may have lost confidence in the institutions involved. This may well have been avoided if a more meaningful negotiation process was pursued or boundary spanning mechanisms were developed to bridge the asymmetry of information, which can improve trust and transparency and increase the possibility of interest mediation (20).

## **ACKNOWLEDGMENTS**

The authors would like to thank Kathleen Vogel for notifying us of this Research Topic and facilitating our participation and also the reviewers for their comments.

## **REFERENCES**

by the federal trade commission.*J Polit Econ* (1983) **91**(5):765–800. doi:10.1086/261181

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 May 2014; accepted: 20 July 2014; published online: 05 August 2014.*

*Citation: Murdock KLE and Koepsell D (2014) Principals, agents, and the intersection between scientists and policy-makers: reflections on the* *H5N1 controversy. Front. Public Health 2:109. doi: 10.3389/fpubh.2014.00109*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Murdock and Koepsell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## **Brett Edwards\***

Department of Politics, Languages and International Studies, University of Bath, Bath, UK \*Correspondence<EMAIL_ADDRESS>
#### **Edited by:**

Jonathan E. Suk, European Centre for Disease Prevention and Control, Sweden

**Reviewed by:**

Samuel A. Weiss Evans, University of California Berkeley, USA

**Keywords: synthetic biology, biosafety, biosecurity, ethics, dual-use, public health, policy**

#### **INTRODUCTION**

In early May, the German Ethics Council produced an in-depth report on the oversight of dual-use research of concern (DURC) (1). The report follows in the wake of recent international emergency reviews of avian influenza research and builds on discussions, which have been taking place internationally for over a decade (2).

In addition to calling for greater awareness raising and education in the scientific community, the report also calls for the establishment of a new legal framework to address DURC within Germany. This framework would provide a legal definition of DURC and would require researchers to report to a newly established central DURC committee before embarking on certain lines of research. Such a legal framework would also generate new responsibilities for those outside the research team who impact upon the research process; from funding right through to publication. For example, this would include new legal responsibilities for Laboratory Biosecurity Officers.

Such an approach would be in stark contrast to the patch-work of largely voluntary measures, which are in place in the rest of the world. The German Ethics Council has also taken the view that Germany should encourage the adoption of similar review models at EU level and internationally.

#### **DUAL-USE AND PRECAUTION**

The summary report of the full 300 page document produced by the German Ethics Council, which is yet to be published in English, notes that:

ethical analysis leads to the conclusion that scientific responsibility in the area of DURC is mainly to be governed by the *precautionary principle* (emphasis added) [Ref. (3), p. 3].

In essence, the precautionary principle places the burden of proof upon the scientific community to demonstrate that research of DURC should be carried out, and is being carried out in a responsible way. The precautionary principle tends to be brought into play in the context of complex risks, which are political challenges characterized by complexity, uncertainly, and ambiguity [Ref. (4), p. 235]. Dual-use issues are complex as they do not involve simple causal chains of events with easily quantifiable consequences, but rather a large set of intervening variables with unknown or even unknowable consequences. This is true not only in relation to thinking about the harms of research, but also its potential benefits. Dual-use issues also involve uncertainty, as there is insufficient data or information to convincingly produce risk verses benefit assessments of single experiments or lines of research. Dual-use issues are also ambiguous, as they typically involve conflicts over ethical and professional values.

Developing a legal and ethical framework to address these issues requires intricate webs of collaboration between institutions, in the context of policy strategy and design, as well as in the context implementation. This creates a challenging environment in which to develop and sustain policy initiatives, directed at problems, which receive only periodic interest from publics and governments.

The reassertion of the role of the precautionary principle in the context of dualuse research provides suitable moment to reflect on broader security concerns related to emerging techno-sciences such as synthetic biology, which extend beyond the single experiments commonly described as constituting DURC. These concerns relate to trends, which could undermine existing models of oversight (such as material and technology containment strategies). This includes concerns about the proliferation of foundation technologies,which could be utilized to modify or synthesize pathogens. These concerns also relate to broader trends in the underlying structures and funding of innovation (5). This includes concerns about de-skilling and proliferation dynamics in life-science research, which could potentially undermine existent and advocated approaches,which place emphasis on local level ethics review, as well as laboratory safety and security (6).

Many of these broader concerns are best thought of as *anxieties* rather than risks, in that discussions about them are largely speculative. However, nonproliferation experts have been keen to reassert, not only that new security challenges are inevitable, but also that existing national and international systems of oversight are poorly prepared (7). It is in this context that the field of synthetic biology has become somewhat of a test-bed for novel security initiatives. In the following section, there is an introduction to how dual-use concerns have emerged in relation to the broader field of synthetic biology, as well as some of the political realities facing those developing policy in this area.

### **EMERGENCE OF DUAL-USE CONCERNS ABOUT SYNTHETIC BIOLOGY IN A US AND EUROPEAN CONTEXT**

There have been discussions of security concerns related to the practices and technologies of synthetic biology, as far back as the community and institutions of the field can be identified. This is perhaps unsurprising, considering that the field emerged in the post 9/11 political environment. However, what is surprising is the high levels of attention this field has received as compared to other contemporary fields of innovation (such as nano-biotechnology).

A key reason for this is that engagement with misuse concerns has been a stipulation of research funding in both the US and the UK. The requirement to address dual-use concerns was incorporated into the National Science Foundation funding criteria for synthetic biology, when the first major publicly funded research center was established [Ref. (8), p. 15]. This led to the establishment of the first major ethical, legal, and social issue (ELSI) thrust with an explicit mandate to consider bioweapon issues. As a result, such concerns also took hold in a European context, as the field was being institutionalized by the research funding bodies. During these early stages a broad range of misuse concerns were under discussion,including those related to the threat of bioterrorism and biowarfare (9–11).

There are two key factors, which are important to thinking about dual-use as an ELSI issue. The first is that dual-use concerns have been a novel addition to more traditional ELSI concerns associated with new and emerging science and technology (such as safety). This means that the issue often competes with more established issues on the ELSI agenda. Security concerns have been more dominant in a US context, but less pronounced in a European ELSI context (12). Added to this, misuse concerns have emerged at a time in which the very concept and practice of ELSI governance is being made subject to transformation.

Both funders and society are increasingly demanding "up-stream" engagement by ELSI thrusts with the innovation process [Ref. (8), p. 15; Ref. (13)]. Up-stream engagement with the innovation process involves engineering safety and security into technologies and research practices, rather than just responding to the challenges raised by the products of innovation. Up-stream engagement is also typically understood to involve pro-active

engagement with key regulators and stakeholders to pre-emptively address potential ethical and legal concerns. Increasingly, such engagement is understood as a part of national government policy to address forward looking concerns about new and emerging science and technology (14, 15).

However, there are several characteristics of dual-use politics, which de-limits the scope and feasibility of such endeavors, which have been reflected in the recent history of dual-use synthetic biology governance in a UK and US context.

### **FROM BROAD ANXIETIES TO NARROW ACTIONS**

The first issue is that despite some of the regulatory back-lash myths, which linger in the US and Europe, governments have not tended to exhibit appetites to legislate *specifically* in relation to dual-use concerns related to synthetic biology. Particularly, with respect to those concerns, which could not be addressed through incremental amendments to law covering laboratory security and safety. Such a situation is symptomatic of a more general trend in dual-use governance in national contexts, in which there is an absence of clear institutional responsibility to develop such policy programs. It is worth noting, however, that even in the absence of "top-down" approaches, regulatory bodies can still play a fundamental role in the fate of so-called "bottom-up" initiatives; by providing financial, political as well as technical support. Such collaboration is also essential if up-stream engagement with the field is actually to result in the development, adoption, and sharing of best-practices nationally.

The second issue is that despite the emphasis on up-stream engagement and best practice sharing at institutions such as SynBERC there has not been substantial investment into systematic and nationwide examinations of the way in which dualuse issues are currently dealt with in different institutional contexts. This is even the case in relation to the field of synthetic biology. Such engagement is necessary if policy discussions about cuttingedge fields are going to be tied to concrete risk identification and management activities in the institutions in which research is taking place. It would seem that without such data gathering, much discussion, particularly in ELSI forum will be condemned to remain an exercise in "speculative ethics" (16).

A final issue is that while considering how to improve security practices at local level is important, there is still a requirement for institutional capacities to identify and respond to much more fundamental trends in S&T, which go beyond the scope of the local level review. In relation to synthetic biology in the US for example, the emphasis on the centrality of local level review, has led to an artificial narrowing of dual-use discussions. For example, the so-called "Sloan Report" (17) still represents one of the most substantial and influential technical reviews of security concerns related to synthetic biology. Yet this report largely externalized those concerns, which could not be identified and managed at local level. Other major reports produced in the US on synthetic biology have also tended to adopt this framing (14, 18).

In particular, the prospect of state level misuse of advances in the life-sciences in the development of weapons has been largely absent from US discussions of synthetic biology as a security concern. In a European context, the issue has only received substantial attention in more recent years. A report on an expert meeting hosted by the United Nations Interregional Crime and Justice Research Institute (supported by the European Commission), describes the various ways in which developments in synthetic biology could re-ignite military interest in biological weapons and potentially undermine existing oversight regimes at national and international level (19).

Such concerns are particularly pressing when one considers the existing challenges, which face the international regime tasked with preventing the development and use of biological weapons. This includes the absence of a system to verify state compliance, which is unlikely to change. While there is slightly more hope for improving the science and technology review system within the regime, improvements continue to be frustrated by a range of bureaucratic and diplomatic issues (20, 21).

#### **CONCLUSION**

To sum up, the point of this article was not to argue that synthetic biology poses an imminent security threat, but instead to argue that while our capacity to imagine misuse scenarios is boundless, our institutional capacities to engage with the less whimsical of these concerns remains quite limited, and developments in policy in this area have been hard won. For some this will not be a cause for alarm, but for others, particularly those with less faith in the resilience of the current norm against biological weapons, this issue continues to be a source of unease.

#### **ACKNOWLEDGMENTS**

I am grateful to Prof. Kathryn Nixdorff for drawing my attention to the work of the German Ethics Committee at a recent meeting on Regional Innovation Biosecurity hosted by the Biochemical Security 2030 Project, at the University of Bath. I am also grateful to the reviewers of this paper and the editors of this Research Topic.

#### **REFERENCES**

*Biotechnol* (2007) **3**(4):333–65. doi:10.1089/ind. 2007.3.333

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11May 2014; accepted: 26 June 2014; published online: 09 July 2014.*

*Citation: Edwards B (2014) Taking stock of security concerns related to synthetic biology in an age of responsible innovation. Front. Public Health 2:79. doi: 10.3389/fpubh.2014.00079*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Edwards. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

#### **Jonathan E. Suk, Cornelius Bartels, Eeva Broberg, Marc J. Struelens and Amanda J. Ozin\***

European Centre for Disease Prevention and Control, Stockholm, Sweden \*Correspondence<EMAIL_ADDRESS>
#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Todd Kuiken, Woodrow Wilson Center, USA Kathleen Vogel, Cornell University, USA

**Keywords: laboratory biosafety, Public Health, infectious diseases, dual-use research of concern, biosecurity, laboratory biorisk management, risk assessments**

#### **INTRODUCTION**

The rapid pace of discovery in the lifesciences can have profound implications for public health, and the focus of much deliberation in recent years has been on how best to ensure that they are positive and not negative. A key focus of debate has been on dual-use research of concern (DURC), which has been defined as life-science research that "could be directly misapplied to pose a significant threat with broad potential consequences to public health and safety, agricultural crops, and other plants, animals, the environment, materiel, or national security (1)." Debates such as the one that surrounded gain-of-function (GOF) research on avian influenza have led to many existential questions about contemporary life-science research, including whether or not such research should even be conducted in the first place, what viable alternative experimental approaches exist, if or how the findings should be made public, and how – or whether – such research can be governed (2–6).

Responding to these questions at a policy level necessarily involve a broader sphere of actors than life scientists alone as they have potential ramifications in different sectors and for society-at-large. Often, it is the security and research communities that have been at the frontline of such debates and driving policy. The public health community tends to enter the fray at later stages, such as after the completion of "concerning" research, at which

point it is asked to either facilitate discussion or comment on the potential public health risks and benefits of research (7, 8). By that stage, public health organizations risk being viewed of as a partisan supporter of dual-use research (9).

In this paper, we demonstrate how the public health sector could more substantially contribute to the debate, guide policy decisions, and promote actions along all phases of the research life-cycle. Before doing so, we articulate the key aspects of the dual-use debate as they are relevant to public health.

#### **"DO NO HARM": PUBLIC HEALTH AND DUAL-USE RESEARCH**

Medical research is intended to promote the health of humans at an individual and a population level along a set of ethical principles laid down in the Declaration of Helsinki1 and overarching professional ethos in health care to "above all, do no harm" – *primum non-nocere.* What complicates matters with regards to the dualuse research in the area of health is that harm could potentially be done by both promoting and preventing research. On the one hand, research fuels innovation in new medicines, vaccines, and diagnostics, which are fundamental components of public health intervention strategies. The fine balance required for tightening up the regulation of life-science research may, in some instances, lead to some undesirable consequences, such as greater barriers and costs for doing research on listed agents2 (10). On the other hand, promoting DURC increases the possibility of malevolent use, through providing the information or material that would help state or non-state actors to develop agents for a biological attack. After pathogens are modified or created, responsibilities are created for their physical containment; the issue is thus one of both laboratory biosafety and biosecurity. Even more problematically, once DURC experiments are published, it is nearly impossible to control access to information. In this sense discussions about the risks and benefits of DURC nearly always occur too late – ideally, they should occur before and not after the research has been conducted.

The wide spectrum of potentially problematic issues related to dual-use research necessitates multi-stakeholder engagement including life-science researchers, research funders, regulators, the scientific media, ethicists,social scientists,and security communities. In addition, given the potential societal implications of the topic, greater transparency, and public engagement surrounding dual-use debates is required (11).

There are numerous highly debated issues that have surrounded dual-use research. At the broadest level has been the question of what sorts of life-science research should be conducted, and what constitutes DURC (12, 13). Regulatory measures, such as security clearances for researchers and export controls (10, 14) as well as other risk mitigation measures (1) have been contemplated alongside

1http://www.wma.net/en/30publications/10policies/b3/, accessed June 7, 2014.

2http://www.australiagroup.net/en/human\_animal\_pathogens.html, accessed May 15, 2014

self-governance measures such as raised awareness and codes of conduct and continued professional development for life scientists (15–18). Furthermore, much attention has focused on whether or how results should be published (6, 19) and how practical measures, such as enhanced laboratory biorisk management processes along (global) standards, can be achieved (7, 20).

From the perspective of public health, these are all important discussions, each aimed at identifying and mitigating potential harm. However, in order to ensure better integration of public health aspects into future policies and actions, a few other key issues must be addressed. One relates to the way in which risk assessments of specific dual-use issues could be strengthened. Another builds upon previous arguments for managing dual-use research by taking into account risk-based approaches and biorisk (biosafety and biosecurity) management in all areas of implementation of the research cycle (7, 21). Addressing all of these public health relevant aspects could offer a next step in structuring the dualuse debate into options to mitigate harm through joint action between the research, security, and policy making communities.

#### **REDUCING INFORMATION ASYMMETRIES IN RISK ASSESSMENTS**

A striking feature of the dual-use debate is that the benefits to public health are often invoked, regardless of the extent to which actors in the public health sector have been consulted. This is particularly ironic given that one core area of public health expertise is the undertaking of risk assessments. For example, one of the rationales for moving beyond the moratorium on publishing A(H5N1) transmission studies was they were"essential for pandemic preparedness" (4). This statement was made after an expert consultation organized by the World Health Organisation (WHO) in February 2012 (7). However, it is important to note that the WHO consultation occurred after the research had been completed and this consultation has been criticized for the absence of a broad range of expertise3 . It should furthermore be noted that although

the obtained study results are of scientific value, some have pointed out that incorporating detailed sequencing studies on the viral samples from infected patients is, for many countries, likely not the most feasible or effective way of strengthening global pandemic preparedness in the short term (19, 22). Previous research has noted that there are, globally, key gaps in viral sample collection and analysis (23). In addition, although the longer-term benefits may well be substantial,translating the findings from advanced life-science research into tangible and usable products and knowledge can take years, if not decades.

It is essential to keep in mind that advanced research is not always predictable. Discoveries can be serendipitous and researchers themselves may not always be able to predict what sort of outcomes certain experiments might lead to. The point here is not to argue against advanced life-science research but to caution against the creation of overly ambitious expectations, particularly, if these will be the basis for risk-benefit discussions about dual-use research. Scholars of science and innovation, for example, have long observed that the creation of sometimes overly optimistic expectations about the future benefits of research helps to secure funding and lower regulatory hurdles (24, 25). Similarly, it has been pointed out that large funding bodies bring particular interests and institutional cultures to the dual-use debate, one which tends to emphasize the positive benefits of such research (9). In the specific example of GOF research on influenza viruses, it is notable that at least a few prominent scientists believed that the benefits had been overstated (22, 26, 27). One way of mitigating the development of over-expectations for research is to ensure balanced debate and doing so could be achieved by comprehensively including a broader range of perspectives, including public health, in both pre- and post-experimental discussions (11, 26).

There is another important information asymmetry on the other side of the spectrum in debates about dual-use research. This relates to the intents and capabilities of would-be bioterrorists and how this affects risk assessments (28). Presumably, the principle reason for worrying about publishing experimental protocols or genomic information relates to the concern that rogue scientists could replicate the results with malevolent aims. Given the rapid advances in life-science research,the advent of synthetic genomics and the numerous social issues that it raises (29, 30), and the declining costs of doing research, it would seem quite reasonable indeed that a wider range of actors might be able to replicate advanced research. Yet this assumption may appear to overstate the ease with which advanced research can be undertaken, which relies not only on information and materials but on experimental know-how and tacit knowledge (31). The recent history of actual bioterrorist events as well as the findings from a risk analysis appears to support this claim (32). This is a key point. Overstating – or understating – the capabilities and intents of bioterrorists can affect the perception of the "riskiness" of research. Here, the public health sector needs to raise and reiterate the importance of this question. The aim should be to ensure risk assessments integrate the best available information from a variety of sectors, meaning that life scientists, regulators, ethicists, public health actors, *and* the security and intelligence communities will need to become more adept at and comfortable with exchanging information and ideas. This has not always been the case.

## **ADDRESSING ALL PHASES OF THE RESEARCH CYCLE**

The WHO advocated managing dual-use risks by taking account of all stages of the research cycle, which is an approach that we would like to briefly elaborate upon here (7). Public health activities already encompass many of these phases, and thus existing expertise could be harnessed to ensure a broad and comprehensive public health engagement with dual-use research (**Table 1**). Following the discussion above, in the "pre-research" phase, the public health sector could contribute to discussions about the possible risks and benefits of research through consultation with research funding bodies, scientists, and institutional review boards. In addition, good laboratory practice means following

3http://www.psandman.com/col/WHO-H5N1.htm, accessed May 14, 2014

ethical and legal guidelines for conducting research on infectious diseases, and for ensuring that robust biorisk management systems are in place. Some European public health institutes have developed tools to facilitate this, such as the Dutch Biosecurity Self-Scan Toolkit4 or the German development of codes of conduct for potential dual-use research5 .

During research, biorisk and biosecurity managers should communicate with researchers about emerging findings, and should regularly monitor the adherence to laboratory biosafety and biosecurity standards for research deemed potentially "risky." It is essential to remember that all laboratory research carries a risk. The public health consequences of laboratory accidents are quite serious, particularly if the possibility exists that laboratoryinfected workers expose the general public, as nearly happened with SARS (33). Early communication with public health agencies about findings likely to generate particular attention could also occur during this phase.

Post-research, a clear discussion on the potential public health benefits should be incorporated into risk analyses concerning the implications and publication of findings. Existing protocols for the physical containment of experimental materials should be reviewed and, if necessary, strengthened should DURC be approved.

The recent misplacement of samples of SARS, anthrax and smallpox demonstrates the continued importance of maintaining high biosafety and biosecurity standards for storing physical specimens6,7 .

During all phases of the research cycle, the public health sector works to strengthen its core functions such as surveillance, preparedness, prevention, response, risk communication, and training. In regions where vast amounts of funding dedicated to potentially "risky" life-science research, the public health community should argue for a greater role in participating in research funding prioritization, and risk-benefit assessments. It could also make the case for strengthened and sustainable investments

4http://www.biosecuritytoolkit.com/mainMenu.html;jsessionid=AAE30461C0ABD5A9A0D7BC33E91DBBD5, accessed May 20, 2014.

5http://www.rki.de/EN/Content/Institute/Dual\_Use/code\_of\_conduct.html;jsessionid=A2FBBD63B5FE63F62EC4B84C0603C295.2\_cid298?nn=4005636, accessed May 20, 2014.

6http://www.pasteur.fr/fr/institut-pasteur/presse/documents-presse/communique-presse-l-institut-pasteur, accessed May 20, 2014.

7http://www.cdc.gov/media/releases/2014/p0711-lab-safety.html, accessed August 4, 2014

in biosafety, biosecurity, and core public health services: should a dangerous pathogen be released, whether intentionally or unintentionally, then strengthened general defenses against infectious diseases will be essential. Finally, but essentially, should the public health sector seek to develop the role of "honest broker" in dualuse discussions, then it will need to work seriously at fostering engagement with key stakeholders, such as security communities and ethicists in addition to life scientists so as to ensure a comprehensive "web of protection" (4). Hosting international meetings focused on bringing together scientists, the security community, public health workers, regulators and possibly even the public could be an initial way of engaging important sectors of society in the debate.

#### **CONCLUSION**

Advances in life-science research are staggering with the advent of synthetic biology, the latest in a long line of technological breakthroughs. There is little to suggest that this pace of change will slow in the future and the increasingly global nature of science means that all countries have a stake in ensuring that research is conducted and disseminated responsibly. Thus far, debates about dual-use research have tended to invoke public health rationales but there is much room for improvement for ensuring that public health perspectives are fully integrated into discussions. This will be challenging, and mechanisms and fora for doing so will need to be created. Yet there is much to be gained from doing so, for the ethics behind the debate closely matches the public health ethos: *primum non-nocere*.

#### **ACKNOWLEDGMENTS**

The author team acknowledges the inputs from the Chief Scientist Johan Giesecke and other colleagues from the ECDC Disease Programmes, Core Sections, and Directors office for their collaboration and support for this work.

#### **REFERENCES**

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 June 2014; accepted: 23 July 2014; published online: 12 September 2014.*

*Citation: Suk JE, Bartels C, Broberg E, Struelens MJ and Ozin AJ (2014) Dual-use research debates and public health: better integration would do no harm. Front. Public Health 2:114. doi: 10.3389/fpubh.2014.00114 This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Suk, Bartels, Broberg , Struelens and Ozin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## A new role for public health in bioterrorism deterrence

## **Margaret E. Kosal \***

Georgia Institute of Technology, Sam Nunn School of International Affairs, Atlanta, GA, USA \*Correspondence<EMAIL_ADDRESS>
## **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Jonathan E. Suk, European Centre for Disease Prevention and Control, Sweden Amanda J. Ozin, European Centre for Disease Prevention and Control, Sweden

**Keywords: bioterrorism, biosecurity, bioweapons, national security, public health, deterrence, polio, Ebola**

When this commentary was submitted in April 2014, only a handful of scholars and policy-makers in the defense and security communities were following the Ebola outbreak in West Africa, which was over 4 months old at that time. Now that thousands of people have died, cases have spread to the US and Europe, and thousands of US uniformed military are being deployed on humanitarian assistance/disaster relief missions, attention and interest are significantly heightened. The events of the last few months demonstrate the criticality for interdisciplinary thinking, which is more challenging due to different historical contexts, knowledge bases, interests, lexicon, and perspectives.

This commentary will explore the creation of new relationships between deterrence, infectious disease, and public health to reduce the threat of biological terrorism and increase international security. Examining the global spread of re-emerging infectious disease, such as the re-emergence of polio from northern Nigeria, offers a novel case study for thinking about how to deter potential bioterrorists who seek to use infectious disease. Polio outbreaks have more directly affected the developing world compared to the US or other nations with robust public health sectors. This example suggests that a bioterrorist attack would also be more devastating for developing countries in low-resource settings compared to the western world. Credibly, communicating this may offer a new approach to deterring bioterrorism by foreign actors. Although a robust public health sector has long been noted to reduce the vulnerability to a bioterrorism attack, actively promoting the strength of US public health can also serve as a powerful deterrent in its own right. The issue of terrorist groups utilizing biological weapons against other states is a mounting concern, yet little deterrence research in the field of political science addresses methods of dealing with the threat of bioterrorism. Thus, creating new conversations among the life sciences, public health, and political science can lead to new perspectives on deterring bioterrorism.

The issue of bioterrorism deterrence, if addressed, has been often added or subsumed under the auspices of deterrence strategies associated with nuclear weapons. In the second half of the twentieth century, nuclear deterrence dominated geopolitics and national security strategies. At its height, the threat of mutually assured destruction (MAD) existed in which both superpowers possessed arsenals with second-strike capabilities, i.e., the ability to respond to a first nuclear strike on land via use of nearly undetectable submarine-launched ballistic missiles with nuclear warheads.

These historical approaches, however, undermine and oversimplify the distinct challenges of deterring bioterrorism. One such method attempted is focusing on pathogen security, or securing and denying access to the materials necessary to develop biological weapons (i.e., deterrence by denial). Based on the nuclear non-proliferation model, pathogen security strives to control the materials, equipment, and personnel involved with production and use of biological agents. With nuclear weapons, controlling fissile materials proved successful because of key characteristics of the critical materials: fissile material is man-made and can be tracked. Those same characteristics that

make nuclear weapons easier to track are those that make biological weapons material difficult to monitor. These characteristics include the presence of biological agents in nature, lower production costs, increased diversity of materials that could be used in bioweapons attacks, and multiple legitimate uses for biological materials. These differing features have not always been fully considered by policy-makers (1). Rather than focusing solely on securing biological materials and laboratories from misuse, other recommendations and strategies that the US has pursued include prevention measures such as biosurveillance, global laboratory and research cooperation, research and development of diagnostics and countermeasures,international stockpiles of effective medical countermeasures, and increased response and mitigation capabilities (2–6). These approaches aim to reduce consequences of an attack, afford earlier detection, and reduce vulnerability; they do not address the challenge of deterring use and reducing motivation directly, however.

To date, discussions about public health and deterrence have focused on measures such as regular vaccinations; access to timely medical care to treat infected, isolate suspected infected, and mitigate the spread of disease; confidence in the professional nature of health providers, etc. These are largely passive, defensive deterrence measures, in that they demonstrate credible capacity by a state to respond and mitigate the consequences of an attack (post-exposure) or reduce vulnerability to an attack by making it ineffective (preexposure) (7–9). Both approaches mentioned thus far, pathogen security and a defensive approach to terrorism, which ultimately aim to decrease vulnerability by fortifying civilian populations, are examples of deterrence by denial adapted from the realm of nuclear deterrence.

In contrast to these passive approaches, active deterrence strategies have not been explored. Active deterrence is actions and policies preventing a specific opponent from doing something they may wish to do. Traditionally, robust active deterrence has involved the application of expressive force to change the policy or character of the target government or group (10). Forces and policies are used to send a political message. In contrast to passive strategies, active deterrence is more dynamic and may incorporate escalating threats in response to an adversary. What this would look like at the nexus of international security and public health is largely an unexplored area of study or policy. Therefore, there are limited models for thinking about deterrence that have been developed exclusively for bioterrorism. As a consequence, the role of a robust public health system for twentyfirst century active deterrence remains to be explored. There has not been a substantive consideration of robust public health system as a strategic asset in a more active deterrence role.

The threat of inflicting punishing retaliation against some aggressor, not the ability to prevent some hostile act from occurring, is the core of traditional deterrence theory. Within new deterrence approaches in political science, however, there are several types of definable strategies that may be applied to bioterrorism by foreign actors (11). *Indirect deterrence* focuses on third party players and their roles in terrorist attacks. Third parties are most typically state sponsors or supporting financiers. This concept is based on the recognition that while a terrorist may be willing to die for his cause, it is less likely that explicit and tacit supporters are willing to pay a similar retribution. Appealing to or directing bioterrorism deterrence efforts toward tacit supporters is an untapped area. *Collective actor deterrence* utilizes the power and influence of institutions like the United Nations, NATO, or other broad coalitions to deter terrorist actions, highlighting the legitimacy of the organization and the international community rather than the interests of a single state. For bioterrorism, the WHO and African Union's disease

eradication efforts are examples. *Internalized deterrence* plays off the psyche of a terrorist, combining abstract concepts of criminology and social constructivism to subconsciously deter a terrorist through social taboos and norms (12, 13). This might involve leveraging fear of disease spreading to oneself or one's own community. *Tailored deterrence* attempts to individualize each situation to reach the best possible solution,leveraging cultural, political, social, and other specific knowledge. These newer deterrence strategies offer opportunities for dealing with bioterrorism threats by foreign actors, which could be combined with public health information and resources.

In thinking about public health infrastructure as an active or passive part of new deterrence strategies, it is useful to think about the role of missile defense. As the presence of a ballistic missile defense system is supposed to be an existential deterrent itself, so could be a strong public health system. Missile defense is both a passive deterrent and, if used, an active deterrent, as it stops somethingfrom occurring. A strong public health infrastructure is likely to be the key in reducing the vulnerability to bioterrorism attack, as well as having a potential role in deterring a foreign terrorist group from even considering such an attack. If a biological weapon launched by a terrorist group will have little or no effect on the target country because of a known robust public health sector, then a foreign terrorist may be discouraged from launching a biological weapons attack in the first place. If foreign terrorists are also aware of the weak public health infrastructure with their own borders, and the increased risks to them and their publics in the event of an accident in developing biological weapons and/or spread of an infectious disease that they might launch, this may also deter them from pursuing this work. In addition, even the accidental release of a dangerous pathogen or the spread of an infectious disease via attack will most likely cause disproportional negative effects to nations with limited public health infrastructures and affect tacit and explicit supporters in those states.

The role of a robust public healthcare system for its deterrence capacity can be explored through empirically driven case study methods against predominant theories of deterrence in political science (14, 15) and in comparison to other works considering the possibility of deterring bioterrorism (16–20). For example, the reemergence of polio offers a potentially useful example to think about the effects of a potential bioterrorist attack on the developed and the developing world. Polio is both a contagious infectious disease and transmissible from human-to-human (like smallpox and plague). The poliovirus is highly transmissible with a basic reproductive rate or secondary transmission rate (R0) exceeding most suspected biological agents, e.g., standard estimates of R0 for polio range from 5 to 7 (21, 22), whereas R0 for suspected bioterrorist agents like smallpox (1.8–3.2) (23–25); pneumonic plague (0.8–3.0) (26, 27); and even Ebola (1.34– 2.0) (28, 29) are lower. It is not a likely biological terrorism agent, however, due to the low-mortality associated with infection. It is, however, a useful model for thinking about the spread of infectious disease and the importance of a robust public health infrastructure as a deterrence strategy.

At the beginning of 2003, the complete eradication of polio appeared to be within the grasp of the World Health Association and its many partners. In 1998, the World Health Organization estimated there were over 365,000 new cases of polio; by early 2003, the rate of infection had declined to <1,000 new cases worldwide due to a vigilant vaccination effort (30). That trend was interrupted, however, when Nigerian citizens refused to be vaccinated after hearing unfounded allegations of contaminated vaccines that would lead to sterility or cause HIV/AIDs. Before 2003, polio had largely been confined to only a handful of countries; Nigeria, India, Pakistan, and Afghanistan accounted for 93% of the world's cases (31). What started with the refusal of local clerics to allow vaccination led to the reestablishment or importation of the poliovirus to 14 countries that were previously disease-free.

Transport of the contagious virus was not limited to neighboring African states. The poliovirus moved through Sudan to Ethiopia crossing the Red Sea to Lebanon andYemen. The latter was been particularly severely affected, witnessing more than 500 new cases in the first half of 2005. The poliovirus spread as far as Indonesia, where it afflicted more than 150 people in a single year in 2 provinces, predominantly children (32). Prior to this outbreak, Indonesia had been polio free for nine years. Genetic fingerprinting confirmed that the strain imported to Indonesia came from northern Nigeria through Sudan, most closely resembling an isolate recovered in Saudi Arabia in December 2004. A pilgrim returning from Mecca or a returning foreign worker is suspected to have brought the virus to the island of Java, across an ocean and thousands of miles from its source. The polio virus continues to persist in a limited number of states in the developing world, specifically in Nigeria, Afghanistan, and Pakistan, where a ban on vaccination by Islamist leaders in Waziristan remains in place. Since 2013, polio (linked genetically to the strain in Pakistan) has spread from Syria to Iraq (33).

Countries that have witnessed the reemergence of poliovirus outbreaks have some crucial links: social and political challenges that have impeded the development and implementation of appropriate public health infrastructures and measures. Not unexpectedly, there is an inverse relationship between government health expenditure in health and number of polio cases.

Looking at the spread of polio can provide us with a lens to think about the impacts of bioterrorism in states with developed public health infrastructures and those who do not. A bioterrorist attack, especially one with a contagious agent like smallpox or pneumonic plague, will likely impact the developing parts of the world substantially more than the US. One only has to look as far as polio's re-emergence (or more recently the outbreak of Ebola virus disease in West Africa) to see the very real repercussions of a contagious virus and how the most dire causes and effects of infection and spread stem from poor public health infrastructures (34).

Creating a new deterrence strategy for bioterrorism is needed. Credibly, communicating the differential capacities to respond and the comparative likely outcomes will require diplomacy, coordination with civil affairs, specialized knowledge of individual states, and regions of the developing world. These are fundamentally interdisciplinary efforts that should leverage small teams from

diplomatic, development, public health, and defense communities. One single parochial voice will be inadequate. Further improving the US domestic public health infrastructure would be beneficial and cost effective regardless of whether an outbreak is intentional or natural. The devastating Ebola outbreaks serve as a call for urgent investment in public health infrastructures worldwide, to provide both responsive and proactive actions to deter bioterrorism and to deal with natural disease outbreaks. Public health remains a powerful and often underutilized asset for bioweapons defense through vulnerability reduction; leveraging public health may also enable new approaches to deterring bioterrorism threats. International security scholars would benefit from better understanding of and leveraging the knowledge of the public health community.

#### **ACKNOWLEDGMENTS**

Thanks to Katherine Lange and Ana Terron for research assistance and to the reviewers for helpful suggestions.

#### **REFERENCES**

31. Minor PD. Polio eradication, cessation of vaccination and re-emergence of disease. *Nat Rev Microbiol* (2004) **2**:473–82. doi:10.1038/nrmicro906

32. Pallansch MA, Sandhu HS. The eradication of polio – progress and challenges. *N Engl J Med* (2006) **355**:2508–11. doi:10.1056/NEJMp068200

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 April 2014; accepted: 27 November 2014; published online: 10 December 2014.*

*Citation: Kosal ME (2014) A new role for public health in bioterrorism deterrence. Front. Public Health 2:278. doi: 10.3389/fpubh.2014.00278*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Kosal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## The irrationality of GOF avian influenza virus research

## **SimonWain-Hobson1,2\***

1 Molecular Retrovirology Unit, Department of Virology, Institut Pasteur, Paris, France 2 Foundation for Vaccine Research, Washington, DC, USA

## **Edited by:**

Amanda Jane Ozin, European Centre for Disease Control and Prevention, Sweden

#### **Reviewed by:**

Li Xu, Cornell University, USA Gigi Gronvall, UPMC Center for Health Security, USA Hervé Raoul, INSERM, France

#### **\*Correspondence:**

Simon Wain-Hobson, Molecular Retrovirology Unit, Department of Virology, Institut Pasteur, 28 rue du Dr. Roux, Paris 75015, France e-mail: simon.wain-hobson@ pasteur.fr