BOARD ON LIFE SCIENCES, DIVISION ON EARTH AND LIFE STUDIES BOARD ON HEALTH SCIENCES POLICY, INSTITUTE OF MEDICINE
MAY 2015
BioWatch PCR Assays:
Building Conidence, Ensuring Reliability
BIOWATCH IS AN AIR MONITORING SYSTEM deployed in jurisdictions around the country with the goal of detecting the presence of certain high risk pathogenic microorganisms. It relies on a network of federal and nonfederal collabora-tive relationships to be successful, and is one part of a larger array of disease surveillance, intelligence- gathering, and biomonitoring activities in support of public safety and health.
The assays used in the BioWatch system to detect the presence of pathogens in collected samples rely on the technique of polymerase chain reaction (PCR) to sensitively and speciically amplify target nucleic acid sequences. The program and its users need an understanding of each assay’s performance character-istics in order to have conidence in the results and have the ability to appropriately interpret them. This conidence is partly provided through a performance standard, which gives information on the minimum requirements that must be met for the assay to be considered acceptable for its intended purpose and describes how testing to validate this performance is to be carried out.
The National Research Council convened a committee of experts to examine standards for PCR assays appropriate for the BioWatch program. The committee’s report discusses principles of perfor-mance standards, reviews information from several existing guidance documents and standards that might be applicable to BioWatch, and discusses assay testing efforts that have occurred or are ongoing. It provides recommendations on general principles and approaches for a performance standard and valida-tion framework to meet BioWatch’s mission. It also provides a starting point for an approach to in silico and laboratory assay characterization for consider-ation by BioWatch and its stakeholders, with the aim of providing a reasonable amount of information on performance parameters such as an assay’s limit of
detection, sensitivity, and speciicity. As tasked, the report focuses on the PCR reaction portion of a PCR assay; although it discusses key processes before and after the PCR, these are not addressed in detail.
One fundamental aspect of assay valida-tion for a system such as BioWatch should be recognized and clearly
communicated to program stakeholders—BioWatch is designed to detect rare but high- consequence events and all assays will have some associated false positive and false negative rates. It is not practical in time, effort, or money to conduct laboratory-based assay performance testing using large enough numbers of replicates under enough different conditions to validate the assay’s long-term false positive or false negative rate at the high levels of statistical conidence program users are likely to desire for operational decision making (e.g., to determine with statistical conidence that a false result will not occur more than once per x thousands or hundreds of thousands of samples).
Committee on PCR Standards for the BioWatch Program: Georges C. Benjamin (Co-Chair), Executive Director, American Public Health Association, Washington, DC; Kenneth I. Berns (Co-Chair), Distinguished Professor of Molecular Genetics and Microbiology Emeritus, University of Florida College of Medicine, Gainesville; Bruce Budowle, Executive Director, Institute of Applied Genetics, and Professor, Department of Molecular and Medical Genetics, University of North Texas Health Science Center, Fort Worth; Charles Chiu, Associate Professor, Laboratory Medicine and Medicine, Infectious Diseases; Director, UCSF-Abbott Viral Diagnostics and Discovery Center at China Basin; and Associate Director, UCSF Clinical Microbiology Laboratory, University of California, San Francisco; John M. Hardham, Associate Research Fellow and Chair, Emerging Infectious Disease Program, Zoetis, Inc., Kalamazoo, MI; Grace Kubin, Director, Laboratory Services Section, Texas Department of State Health Services, Austin; M. Allen Northrup, Principal, Northrup Consulting Group, San Francisco, CA;
Tom Slezak, Program Leader, Global Security Program, E.O. Lawrence Livermore National Laboratory, Livermore, CA; Peter M. Vallone, Leader, Applied Genetics Group, Biomolecular Measurement Division, National Institute of Standards and Technology, Gaithersburg, MD; Katherine Bowman (Study Director, Board on Life Sciences), Autumn Downey (Program Oicer, Board on Health Sciences Policy), Kathryn Hughes (Senior Program Oicer, Board on Chemical Sciences and Technology), Jon Q. Sanders
(Program Coordinator, Board on the Health of Select Populations–until November 2014), Nia D. Johnson (Senior Research Associate, Board on Army Science and Technology–from November 2014), Bethelhem Mekasha (Financial Associate, Board on Life Sciences), Andrew Pope (Director, Board on Health Sciences Policy), Frances Sharples (Director, Board on Life Sciences), National Research Council
he National Academies appointed the above committee of experts to address the speciic task requested by the Department of Homeland Security. he members volunteered their time for this activity; their report is peer-reviewed and the inal product signed of by both the committee members and the National Academies. his report brief was prepared by the National Research Council based on the committee’s report.
For more information, contact the Board on Life Sciences at (202) 334-2187 or visit http://dels.nas.edu/bls. Copies of BioWatch PCR Assays: Building Conidence, Ensuring Reliability are available from the National Academies Press, 500 Fith Street, NW, Washington, D.C. 20001; (800) 624-6242; www.nap.edu.
Permission granted to reproduce this document in its entirety with no additions or alterations. Permission for images/igures must be obtained from their original source.
© 2015 he National Academy of Sciences
Locate additional information, including related reports, at http://dels.nas.edu/bls Read, purchase, or download a free PDF of this report at http://www.nap.edu responses to positive BioWatch detections. The
report emphasizes the importance of communication and, particularly, of discussing the details of perfor-mance data with jurisdictional laboratory experts and oficials. The report also emphasizes the importance of obtaining a better understanding of the micro-organism background that is present in BioWatch jurisdictions—information that remains very limited.
Finally, the report considers how developments in technology, particularly in multiplex PCR and next-generation sequencing (NGS), can contribute to the ability of the BioWatch program to meet current and future challenges. Sequencing currently can be used to follow up on unexpected assay results from the jurisdictions. In the nearer term, targeted approaches
