Correspondence

Good Laboratory Practices and Safety Assessments

Environ Health Perspect. doi:10.1289/ehp.0900884 available via http://dx.doi.org [Online 26 October 2009]

Referencing: Why Public Health Agencies Cannot Depend on Good Laboratory Practices as a Criterion for Selecting Data: The Case of Bisphenol A

Having confidence in scientific procedures and data is the sine qua non for determining the safety of chemicals and chemical products. For decisions of safety, there must be rigorous and thorough application of fundamental scientific practices, irrespective of the purpose of the study and where it is conducted—academic, industry, or a contract laboratory.

Investigations must be designed and conducted by experts; whenever possible, standardized and validated test methods and test systems should be used, test devices and instruments must be appropriately calibrated and their accuracy assured, and, most important, all of the data, including raw laboratory records, should be available for independent review. Good Laboratory Practice (GLP) requirements, based on these fundamental scientific principles and practices, are indispensable for providing scientific confidence in studies conducted for chemical safety determinations. These reasons explain why government agencies worldwide require GLP compliance, and why it is entirely appropriate for greater weight to be given to GLP studies than non-GLP studies that are only available as articles in scientific journals. In their commentary Myers et al. (2009) argued that noncompliance with GLP should not be used as the sole criterion for excluding studies from consideration in regulatory decision-making. We agree that GLP should not be the sole criterion, but we strenuously disagree with the authors’ mischaracterization of the purpose and function of GLP and with their conclusion that GLP has no utility for weighting the reliability of studies.

Evaluating the safety of any substance should include review of all relevant studies utilizing a systematic weight-of-evidence framework. Although not all studies that are useful for hazard characterization and risk assessment may be amenable to GLP (e.g., epidemiology and mechanistic studies, studies conducted before the acceptance of current GLP), this does not obviate their consideration. Each study, GLP and non-GLP, should be evaluated and weighed in accordance with fundamental scientific principles. Factors to be evaluated include a) verification of measurement methods and data; b) control of experimental variables that could affect measurements; c) corroboration among studies; d) power (both statistical and biological); e) universality of the effects in validated test systems using relevant animal strains and appropriate routes of exposure; f ) biological plausibility of results; and g) uniformity among substances with similar attributes and effects. Regulatory agencies [Food and Drug Administration (FDA) and U. S. Environmental Protection Agency (EPA)] and the National Toxicology Program (NTP) require studies to be conducted in accordance with GLP (FDA 2005; NTP 2006; U.S. EPA 2007a, 2007b), and the Organisation for Economic Co-operation and Development (OECD) GLP principles (OECD 1998) apply to all OECD member countries.

Academic basic research is very different from regulatory research and testing. Academic research focuses on developing and evaluating new hypotheses, on creating novel methods, and on discovering new findings. Academic research is open to wide interpretation and may require significant additional studies to clarify and determine whether and how broadly the results apply. Although novel techniques and discoveries of academic investigations stimulate further research, they must also stand up to the scientific method: hypothesis formulation, hypothesis testing, and validation by independent replication. Independent replication provides critical information on the strength of the hypothesis and reliability of test methods. Inconsistent results can arise from use of novel techniques, different test systems, uncertainty and differences in test chemical composition and purity, and a myriad of other factors. These facts, in conjunction with the more limited availability of actual data in most journal publications, means regulatory agencies can face significant challenges in confirming the quality, performance, or data integrity of results obtained solely from information available from a typical article in peer-reviewed journals. Whereas all study records and data from GLP investigations are available to agencies, rarely, if ever, are such details made available as part of the peer-review process for publishing a manuscript in a scientific journal. This can limit the ability of an agency to independently evaluate conclusions or to conduct alternative analyses of the data. The challenges faced by the peer-review procedures of journals have been recently highlighted (Nature 2006), and it has been pointed out that “…scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth” (Jennings 2006). Journal peer review relies on summarization of experimental procedures and results, and does not include examination of laboratory study records or raw data. The purpose for journal peer review is to judge whether the study has been conducted and reported according to internationally recognized, general scientific standards and whether the study meets the interest level for dissemination to scientific community. It is not designed to provide assurance of accuracy or to recalculate raw data, and it does not provide an opportunity for independent audit of the study. Myers et al. (2009) failed to clearly make these distinctions.

Relevant internationally agreed test methods are used by industry to generate toxicity data for safety determinations by regulatory agencies. Incorporation of GLP in these laboratory tests assures that written protocols and standard operating procedures for each study component are developed and carefully and completely followed. GLP also requires meticulous adherence to dosing techniques; the use of adequate group sizes to allow meaningful statistical analysis; characterization (identity, purity, concentration) of test and control substances, including dosing solutions; detailed recording of study measurements and data; and collection of all raw laboratory data in a manner that can be retained and made available for regulatory agencies to audit and reach independent conclusions. Quality control procedures, quality assurance reviews, and facility inspections are also used to monitor and enforce GLP compliance. The relevance, reliability, sensitivity, and specificity of most test methods required of industry by regulatory agencies are well understood because they have been subjected to extensive, round-robin validation programs conducted in numerous laboratories throughout the world. This high level of scientific rigor, in conjunction with the detailed processes of GLP, provides regulatory agencies increased confidence in both the relevance and quality of GLP scientific studies for safety decisions, and it is the reason it is wholly appropriate in regulatory decision making for greater weight and confidence to be afforded to studies conducted in accordance with GLP.

This letter has been reviewed in accordance with the peer- and administrative-review policies of the authors’ organizations. The views expressed here are those of the authors and do not necessarily reflect the opinions and/or policies of their employers.

The authors are employed by trade associations whose members manufacture and use chemicals.

Richard A. Becker
American Chemistry Council
Arlington, Virginia
E-mail: rick_becker@americanchemistry.com

Erik R. Janus
Crop Life America
Washington, DC

Russell D. White
American Petroleum Institute
Washington, DC

Francis H. Kruszewski
Soap and Detergent Association
Washington, DC

Robert E. Brackett
Grocery Manufacturers Association
Washington, DC

References

FDA. 2005. Good Laboratory Practices for Conducting Nonclinical Laboratory Studies. 21CFR58. Available: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=58&showFR=1 [accessed 6 October 2009].

Jennings CG. 2006. Quality and value: the true purpose of peer review. Nature doi:10.1038/nature05032. Available: http://www.nature.com/nature/peerreview/debate/nature05032.html [accessed 30 March 2009].

Myers JP, vom Saal FS, Akingbemi BT, Arizono K, Belcher S, Colborn T, et al. 2009. Why public health agencies cannot depend on Good Laboratory Practices as a criterion for selecting data: the case of bisphenol A. Environ Health Perspect 117:309–315.

Nature. 2006. Nature’s Peer Review Debate. Available: http://www.nature.com/nature/peerreview/debate/index.html [accessed 30 March 2009].

NTP (National Toxicology Program). 2006. Specifications for the Conduct of Studies to Evaluate the Toxic and Carcinogenic Potential of Chemical, Biological and Physical Agents in Laboratory Animals for the National Toxicology Program. Available: http://ntp.niehs.nih.gov/files/Specifications_2006Oct1.pdf [accessed 2 April 2009].

OECD (Organisation for Economic Co-operation and Development). 1998. OECD Principles of Good Laboratory Practice (as revised in 1997). Available: http://www.olis.oecd.org/olis/1998doc.nsf/LinkTo/NT00000C5A/$FILE/01E88455.PDF [accessed 6 October 2009].

U.S. EPA (U.S. Environmental Protection Agency). 2007a. Good Laboratory Practice Standards. 40CFR160. Available: http://www.access.gpo.gov/nara/cfr/waisidx_07/40cfr160_07.html [accessed 2 April 2009].

U.S. EPA. 2007b. Good Laboratory Practice Standards. 40CFR792. Available: http://www.access.gpo.gov/nara/cfr/waisidx_07/40cfr792_07.html [accessed 2 April 2009].

Good Laboratory Practices: Myers et al. Respond

Environ Health Perspect. doi:10.1289/ehp.0900884R available via http://dx.doi.org [Online 26 October 2009]

We are in complete agreement with the statement by Becker et al. that “having confidence in scientific procedures and data is the sine qua non for determining the safety of chemicals and chemical products.” Our aim in writing the commentary (Myers et al. 2009) was not to challenge the original intent of Good Laboratory Practices (GLP) requirements, which was to establish standards of record keeping in contract laboratory research so as to reduce the likelihood of fraud. Our goal instead was to show—through an analysis of the application of GLP data on bisphenol A (BPA) in regulatory proceedings—that GLP by itself is insufficient to guarantee valid and reliable science. Becker et al. appear to have missed the point of our commentary entirely.

In the case of BPA, three GLP studies have been offered by industry-sponsored laboratories as proof of the chemical’s safety (Cagen et al. 1999; Tyl et al. 2002, 2008). Each has errors in study design and/or data interpretation that are sufficiently serious as to invalidate the conclusions of these studies (Myers et al. 2009). Nevertheless, because the studies were conducted using GLP guidelines, they were judged by regulators as being more reliable than the many National Institutes of Health (NIH)-funded and peer-reviewed studies that have reported adverse effects (Richter et al. 2007; vom Saal et al. 2007).

As our commentary (Myers et al. 2009)clearly establishes, GLP did not guarantee the scientific validity of these three studies. Because previous analyses had identified serious flaws in the first two of those GLP studies, we focused critical attention on the most recent (Tyl et al. 2008), which both the European Food Safety Authority (EFSA 2006) and the U.S. Food and Drug Administration (FDA) had identified as key in their BPA risk assessments (FDA 2008). We found three main flaws: a) the animals were inexplicably insensitive to estrogen; b) the assays were outdated and insensitive compared with methods used in NIH-funded research showing adverse effects; and c) validity of the findings was challenged. For example, the prostate weights of control animals reported by Tyl et al. (2008) were > 70% larger (mean, > 72 mg) than those reported by numerous laboratories, including a previously published study using CD-1 mice [conducted at RTI, where the study by Tyl et al. (2008) was conducted] that reported mean prostate weights of 46 mg in CD‑1 males that were examined at a similar age (Heindel et al. 1995).

Since we published our commentary (Myers et al. 2009), a possible contributor to both the estrogen insensitivity and the enlarged control prostates has been suggested: Approximately 3 years before the experiments that formed the basis of the study by Tyl et al. (2008), there was a polycarbonate fire that released BPA into the RTI laboratory where the research was conducted (Kissinger and Rust 2009). An investigation revealed that animals in the laboratory were exposed to low doses of BPA that government-funded science (Richter et al. 2007) indicates could affect research animals.

Additional uncertainties about Tyl et al.’s study (Tyl et al. 2008) have now been identified by the lead author. Whereas the published paper reports that the animals were examined at approximately 14 weeks of age, Tyl testified at an FDA hearing in September 2008 that they were 6 months of age, and then at a German Environmental Protection Agency hearing in March 2009 that they were 5 months of age (Kissinger and Rust 2009). There she confirmed that the information in the original article was inaccurate. Because an animal’s physiology changes as it ages, these contradictory statements are problematic for all reported outcomes; even at 5–6 months of age, normal, healthy CD-1 male mice would not have the grossly enlarged prostates reported by Tyl et al. (2008).

The use of flawed science, however, is not the only concern. The type of multigeneration testing approach used in these studies is, quite simply, insufficient for the testing of endocrine-disrupting chemicals. This is not a new concept. The need for more specific tests for endocrine-active compounds led in 1998 to the establishment at the U.S. Environmental Protection Agency (U.S. EPA) of the Endocrine Disruptor Screening Program, mandated by Congress (U.S. EPA 1998). After virtually no progress for over a decade, in 2009 the U.S. EPA finally announced a set of testing procedures that will be examined. The proposed “new” methodology, heavily dependent upon traditional toxicologic methods used in multigenerational GLP studies, is still woefully inadequate (Colborn 2009).

The letter by Becker et al. provides a striking example of the reluctance of industry lobbyists to hear this message. In the eyes of the 36 scientific colleagues who coauthored our commentary (Myers et al. 2009), the BPA studies that Becker et al. attempt to defend are so seriously flawed as to be indefensible. Rather than continue to defend a dead issue, we encourage industry representatives to come into the 21st century and help us devise new paradigms for testing endocrine-disrupting chemicals that will safeguard human health.

The authors’ freedom to design, conduct, interpret, and publish this letter was not nor is compromised by any controlling sponsor as a condition of review and publication.

J.P. Myers is CEO/chief scientist for Environmental Health Sciences (EHS), a not-for-profit organization that receives support from several private foundations (listed at http://www.environmentalhealthnews.org/about.html) to support EHS’s mission to advance public understanding of environmental health sciences; no grants to EHS were received to support the writing of this letter. T. Colborn is the president of TEDX (The Endocrine Disruption Exchange), a not-for-profit organization receiving funding from several private foundations (listed at http://www.endocrinedisruption.com/support.php) in support of their mission to educate people about endocrine disruption and other toxic chemicals, and to assist them in their efforts to reduce the production, use, and exposure to chemicals that can interfere with development and function; TEDX also receives individual contributions solicited via the website. S. Jobling is the principal of Beyond the Basics Limited, a consultancy company that manages scientific projects and advises on scientific research. F. vom Saal is CEO of Xenoanalytical LLC, a small private laboratory that performs assays of xenobiotic compounds.

John Peterson Myers
Environmental Health Sciences
Charlottesville, Virginia
E-mail: jpmyers@ehsic.org

Frederick S. vom Saal
Julia A. Taylor
Division of Biological Science
University of Missouri
Columbia, Missouri

Benson T. Akingbemi
Department of Anatomy, Physiology & Pharmacology
College of Veterinary Medicine
Auburn University
Auburn, Alabama

Koji Arizono
Faculty of Environmental and Symbiotic Science
Prefectural University of Kumamoto
Tsukide, Kumamoto, Japan

Scott Belcher
Department of Pharmacology & Cell Biophysics
Center for Environmental Genetics
University of Cincinnati
Cincinnati, Ohio

Theo Colborn
The Endocrine Disruption Exchange
Paonia, Colorado

Ibrahim Chahoud
Institut für Klinische Pharmakologie und Toxikologie Charité–Universitätsmedizin Berlin
Campus Benjamin Franklin
Berlin, Germany

All 37 authors of the original commentary signed this letter but only the first 7 are listed here.

References

Cagen SZ, Waechter JM Jr, Dimond SS, Breslin WJ, Butala JH, Jekat FW, et al. 1999. Normal reproductive organ development in CF-1 mice following prenatal exposure to bisphenol A. Toxicol Sci 50:36–44.

Colborn T. 2009. EPA’s New Pesticide Testing Is Outdated, Crude. Available: http://www.environmentalhealthnews.org/ehs/editorial/epa2019s-new-pesticide-testing-is-outdated-crude [accessed 27 April 2009].

EFSA (European Food Safety Authority). 2006. Opinion of the Scientific Panel on Food Additives, Flavourings, Processing Aids and Materials in Contact with Food on a Request from the Commission Related to 2,2-Bis(4-hydroxyphenyl)propane (Bisphenol A). EFSA J 428:1–76.

FDA. 2008. Draft Assessment of Bisphenol A for Use in Food Contact Applications. Available: http://www.fda.gov/ohrms/dockets/ac/08/briefing/2008-0038b1_01_02_FDA%20BPA%20Draft%20Assessment.pdf [accessed 14 August 2008].

Heindel JJ, Chapin RE, George J, Gulati DK, Fail PA, Barnes LH, et al. 1995. Assessment of the reproductive toxicity of a complex mixture of 25 groundwater contaminants in mice and rats. Fundam Appl Toxicol 25:9–19.

Kissinger M, Rust S. 2009. Consortium rejects FDA claim of BPA’s safety. Milwaukee Journal Sentinel, 11 April. Available: http://www.jsonline.com/watchdog/watchdogreports/42858807.html [accessed 11 April 2009].

Richter CA, Birnbaum LS, Farabollini F, Newbold RR, Rubin BS, Talsness CE, et al. 2007. In vivo effects of bisphenol A in laboratory rodent studies. Reprod Toxicol 24(2):199–224.

Tyl RW, Myers CB, Marr MC, Sloan CS, Castillo N, Veselica MM, et al. 2008. Two-generation reproductive toxicity study of dietary bisphenol A in CD-1 (Swiss) mice. Toxicol Sci 104:362–384.

Tyl RW, Myers CB, Marr MC, Thomas BF, Keimowitz AR, Brine DR, et al. 2002. Three-generation reproductive toxicity study of dietary bisphenol A in CD Sprague-Dawley rats. Toxicol Sci 68(1):121–146.

U.S. EPA (U.S. Environmental Protection Agency). 1998. Endocrine Disruptor Screening Program (EDSP). Available: http://epa.gov/endo/index.htm [accessed 3 May 2009].

vom Saal FS, Akingbemi BT, Belcher SM, Birnbaum LS, Crain DA, Eriksen M, et al. 2007. Chapel Hill bisphenol A expert panel consensus statement: integration of mechanisms, effects in animals and potential to impact human health at current levels of exposure. Reprod Toxicol 24(2):131–138.