 harm. As anticipated, the vast majority of "good science" complaints filed to date under the Data Quality Act against the EPA have been submitted by regulated parties.54,55
Competence of Lay or Political Decisionmakers
Compounding worries about incomplete disclosures and skewed information provided by interested parties in their review of scientific quality are concerns about the decision-makers scientific competence to assess scientific reliability, especially when their sources of information on these questions are so badly biased and incomplete. Judges and regulatory officials, to a lesser extent, are not typically scientists, and we know from a large body of critical literature and a growing number of published opinions that they sometimes make decisions about science that are wrong.5658 The literature on judicial review of agency technical decisions is perhaps the most negative about the capacity of judges to review the science used by agencies in rulemakings.57,58 But scientific errors committed by lay persons who preside over challenges to the quality of scientific studies arise throughout the science-policy literature, with documented problems arising in scientific misconduct hearings, discovery disputes, and evidence determinations.59
To the extent that errors emerge from these frailties in the system, the most serious error is the possibility that good studies will be excluded from the policymaking process. This is especially troubling because scientific research is already in short supply in many areas of environmental and health policy. The probability that good science will be erroneously excluded seems most likely when an affected party has a great deal to lose from a research study and has the resources to invest in discrediting it.60 Indeed, one would expect Daubert and the "good science" challenges to be used only when a party is adversely affected by scientific knowledge and has the resources to mount an expensive technical attack against it. Research on Dauberts effects in erroneously excluding good research is ongoing, although preliminary evidence suggests that this is a problem in some cases.61 Experience thus far with the Data Quality Act indicates that the agencies are resisting petitions for correction or exclusion of scientific studies, but future responses will depend in part on the political inclinations of the agency.
The "Good Science" Reforms: Potentially a Greater Danger
In assessing the wisdom of the increased reliance on interest groups to review scientific quality, it is important to consider differences, as well as similarities, between courts and agencies. Because of several major institutional differences, use of the interest group model to review scientific quality is likely to present more serious dangers in the regulatory arena than in the courts. First, rather than a supposedly "neutral" jurist who will preside over challenges to the veracity of a scientific study, the "good science" laws task a politically appointed official to conduct this review. This presents significant added dangers that political considerations will bias the review of scientific quality. Even in the courts, covert values have arguably afflicted some judges Daubert rulings, particularly rulings on the sufficiency of a partys evidence under Daubert-Joiner.62 Because agencies are fundamentally political entities, one would expect their decisions to be subject to even greater blurring of science and policy.63 Moreover, because challenges can be brought when an agency first considers a study, that studys relationship to the larger scientific literature and to the agencys mandate for regulating in a preventive way is at risk of being ignored in the agencys determination of whether an individual study qualifies as reliable and objective. Even more perversely, the "good science" laws may enable the agency to advance its political agenda covertly, because the technical appearance of the complaints reduces the ability of attentive public, busy nonprofits, and even elected officials to understand the underlying policy choices, much less their implications.
Second, "good science" challenges in the administrative arena are likely to have wider ranging implications than their judicial analogs. Daubert rulings (including erroneous ones) are generally limited to a single federal district, which is often a section of a state. By contrast, the final decision on an interest group challenge to scientific quality under the "good science" laws is made by a federal agency and applied nationwide. Indeed, for this very reason a law firm defending the asbestos companies in the asbestos litigation has filed a Data Quality Act petition seeking the withdrawal of evidence damaging to their clients case.64 If successful, their Data Quality Act challenge will expunge the bothersome information, and they will no longer be forced to seek its exclusion, sometimes unsuccessfully, in individual court cases.
Third and finally, because the "good science" reforms create added processes for interest groups to unilaterally challenge the quality of scientific research, these adversarial processes are capable of becoming substantially more imbalanced than their civil litigation counterparts. The greater stakes at issuenational rules that apply across multiple industriesmean that the resources dedicated to challenges by regulated parties could be well in excess of those dedicated to single Daubert challenges. The opportunity for multiple interest groups to file cumulative, complementary challenges in this national setting could additionally increase the strain put on the validity of a single study. Perhaps the most important contrast to Daubert hearings, however, is the fact that there may be no interest groups willing or able to invest resources in defending a challenged study, leaving the uncompensated researcher and possibly the agency to rehabilitate deconstructed research without assistance.
Counteracting the Opportunities for Abuse and Imbalance in Review
Several modest adjustments to Daubert and the "good science" laws will help correct the built-in incentives for abuse, as well as the imbalances that result from the resource-intensive nature of these adversarial processes. First, in keeping with the courts sanctions for frivolous claims, agencies could promulgate rules that make it clear that abuse of the "good science" laws will be penalized, thus moderating incentives for parties to use the challenges in a harassing way.65 Thus, if there is no good-faith basis for a "good science" challenge, penalties would be levied against the challenger. Such sanctions would penalize only the very worst abuses, but the mere threat of sanctions could help to deter marginal abuses.
Second, indigent parties to litigation, or at least indigent criminal defendants, could apply for Daubert funds to support Daubert challenges or to defend against them. These funds could be provided by a small additional tax on court filing fees, through penalties collected from nonmeritorious challenges, or through general revenues.
Third, given the resource drag that might afflict the agencies, the costs agencies incur in responding to "good science" complaints and data requests could be paid for by those bringing a challenge if the challenge is ultimately denied. In the process of developing implementing regulations for the Data Access Act, the scientific community convinced OMB to include a provision requiring the requestor to reimburse nongovernmental scientists and other researchers for out-of-pocket costs incurred in responding to data requests.66 This internalization of processing costs could be extended to the Data Quality Act. There will be greater protection against abuse of process if complainants are required to pay for the agency costs associated with responding to challenges ultimately judged to be without merit. Such a mechanism will also ensure that the agencys scarce resources are not squandered and diverted, particularly if the resolution of a data quality correction request forces the agency into an expensive consultation with the National Academy of Sciences or a science advisory panel.
Discouraging the Blurring of Science and Science Policy
Decisionmakers should be vigilant in ensuring that policy disputes are not mixed into the assessment of scientific quality. Accordingly, in adjudicating Daubert challenges and "good science" complaints, the very first step a lay decisionmaker should take is to ensure that the challenge is one against science rather than policy or science policy. This formal decisionmaking step will introduce some accountability for affected parties who are otherwise free to mischaracterize value disputes as if they were instead technical disagreements.
Protection against Unproductive Deconstruction
One of the greatest dangers of the "good science" reforms is that they will be used to deconstruct valid studies and disparage credible researchers. Sanctions should be awarded when a challenge is blatantly frivolous as discussed earlier. In addition, particularly when researchers themselves are implicated in the deconstruction or disparagement of research, scientific organizations, like the American Association for the Advancement of Science or the National Academy of Science, could provide a valuable public service by establishing a balanced committee of scientists to investigate the complaints and defend innocent researchers, much as the American Association of University Professors investigates complaints of infringements on academic freedom.67 These panels could conduct full examinations of the legitimacy of these "good science" challenges and issue a report that would be made public and provided to the agency (or court) involved in the matter. In cases where individual researchers are unfairly disparaged, moreover, the group could come to the researchers assistance and even recommend professional or legal sanctions against those doing the disparaging. This type of respected panel could provide a valuable, neutral assessment of the challenged science. They could also raise the costs to interest groups, essentially through reputational damage, for exaggerating flaws in good research, conducting biased reanalyses, or otherwise obfuscating the value of well-done research under the pretext of "good science" challenges.
Accepted for publication September 1, 2004.
2. S. Shackley, "Science and policymaking," Environment 1997; 39: 3.
3. U.S. EPA, About the EPA Science Advisory Board, available at (last accessed April 17, 2005).
4. Jasanoff, 3435, 93, 244.
5. 5 United States Code App I,
5(b)(2), 5(c).
6. D. Dockery, C.A. Pope III, X. Xu, et al. "An Association Between Air Pollution and Mortality in Six U.S. Cities." New England J of Med, 329 (1993): 17531759.
7. National Research Council, Access to Research Data in the 21st Century: An Ongoing Dialogue Among Interested Parties: Report