Louis Anthony Cox, Jr.
Download this Public Interest Comment (PDF)
Introduction
EPA’s proposed determination that existing ozone NAAQS are not requisite to protect public health with an adequate margin of safety is not justified by the evidence it presents. Nor do EPA’s predictions that further reductions in ozone standards will cause future public health benefits follow from sound and reliable scientific methods of causal analysis and prediction. The U.S. EPA document entitled Health Risk and Exposure Assessment for Ozone, Final (July, 2014): Executive Summary states in its first paragraph that “The health effects evaluated in this HREA are based on the findings of the O3 ISA (U.S. EPA, 2013) that short term O3 exposures are causally related to respiratory effects, and likely causally related to cardiovascular effects, and that long term O3 exposures are likely causally related to respiratory effects.” The introduction also states that “The results of the HREA are developed to inform the O3 Policy Assessment (PA) in considering the adequacy of the existing O3 standards, and potential risk reductions associated with potential alternative levels of the standard.”
Thus, EPA positions causality and its implications for potential risk reductions as leading considerations in its proposed decision to reduce existing standards for ozone. The specific meaning of causality to which EPA refers is that it would allow readers of the policy assessment to consider “potential risk reductions associated with potential alternative levels of the standard.” We assume that EPA means “caused by” rather than “associated with” here, and that they mean that their assessment and causality determination will allow interested policy makers to understand how different reductions in the ozone standard would (probably) affect future risks to human health and other endpoints of interest. This crucial claim appears to be mistaken and misleading. EPA has presented no validated causal modeling that would enable correct prediction (and uncertainty characterization) of the effects (if any) that future changes in O3 levels might cause in future changes in health effects. Rather, EPA uses measures and models of historical associations between O3 levels and adverse health effects as if they were known to be entirely causal. This does not provide a sound or reliable basis for risk assessment (Appendix A; Dominici et al., 2014).
EPA follows a well-developed but unreliable quantitative risk assessment (QRA) process, critically discussed in the next section and in Appendix A, that interprets statistical associations as being causal based on weight-of-evidence (WoE) considerations. As discussed in detail later, WoE considerations are not logically or practically adequate for this purpose (e.g., Goodman et al., 2013; Morabia, 2013; Rhomberg et al., 2013; Appendix A). Table 1 of EPA’s ozone ISA presents these considerations. They incorporate several formal logical fallacies, such the ex post ergo propter hoc fallacy that “Evidence of a temporal sequence between the introduction of an agent, and appearance of the effect, constitutes another argument in favor of causality.” There is no theoretical or practical reason to suppose that EPA’s causal judgments based on these WoE considerations are factually correct. Experts, like other people, typically have high confidence in their own judgments, even when these lack objective validity (Kahneman, 2011). But subjective confidence in subjective judgments should not be used in place of sound, objective scientific methods. To do so, as in EPA’s risk assessment for ozone, replaces sound science with potentially arbitrary, biased, and mistaken judgments.
To obtain quantitative risk and benefit estimates, EPA applies a new quantitative model (The “MSS model”) of the relation between ozone levels and decrements in lung function. This model makes predictions that depend sensitively on assumptions that are known to be incorrect (e.g., that individual variability is normally distributed) but that are made anyway for the sake of convenience. There are several key uncertainties about the model assumptions and conclusions, none of which has been quantified.
EPA also develops an approach to quantitative risk assessment based on epidemiological data in section 7.1.2 that makes the crucial error of treating the slope of a curve (i.e., the change in y divided by change in x) as if it showed the future changes in the variable on the y axis (health effects) that would be caused by changes in the variable on the x axis (exposure). This conceptual error invalidates EPA’s quantitative risk and benefit predictions; it is a completely invalid interpretation of the slope of the model curve fit to past data (see Appendix A and Rothman et al., 2012).
A simple analogy may help to make this technical point clear. Dividing car accidents per year in a population by pounds of potatoes consumed per year in that population would produce a positive “slope factor” (i.e., ratio) linking these two quantities. One could meaningfully discuss the change in car accidents per unit change in potato consumption when discussing the slope of a regression curve fit to their historical values. But it would be utterly mistaken to interpret this as implying that reducing future potato consumption would cause a reduction in future car accidents. Figure 2 of Appendix A shows a more realistic example. EPA’s risk assessment makes essentially this conceptual error, confusing the changes used to calculate slopes of exposure-response curves with the changes in health effects that causal mechanisms might cause if exposure were changed.
EPA’s use of associational methods to drive key causal conclusions ignores a consensus in disciplines outside epidemiology that associational methods are unreliable, logically unsound, and inappropriate for drawing causal inferences (e.g., Dominici et al., 2014; Rothman et al., 2012; Cox and Popken, 2015). EPA’s judgment that model-based associations are causal also conflicts with quantitative causal analyses of historical data. Studies that have applied formal quantitative causal analysis methods have found no detectable human health benefits caused by past reductions in ambient ozone levels (e.g., Moore et al., 2012; Cox and Popken, 2015). These studies provide no objective reason to expect that further reductions in the ozone standard will cause future human health benefits.
The absence of any impact of relatively large reductions in ozone concentrations on public health outcomes in previous causal analyses suggests that there may be an exposure below which diseases do not occur, and that this threshold may be well above the current standard. If this is true, a NAAQS set higher than current levels would meet the statutory standard. This is consistent with proposed causal mechanisms involving inflammation of the lung, although such thresholds would not be apparent in population data unless errors in exposure estimates are properly modeled (Cox, 2011, 2012). EPA considers and rejects the hypothesis of an exposure threshold for adverse effects that is above current standards, but does not quantify exposure uncertainties and their effects on detection and estimation of thresholds. Such quantitative uncertainty analysis is essential for correct inferences about thresholds.
In summary, EPA’s quantitative risk estimate (QRA) provides no legitimate reason to believe that the proposed action is “requisite to protect public health” or that reducing the ozone standard further will cause any public health benefits. The QRA’s model-based projections to the contrary are known to rely on mistaken assumptions (for the MSS model) and mistaken interpretations of curve-fitting (for the epidemiological risk assessment in Section 7). Past data on human health before and after reductions in ozone do not reveal any such causal impacts. Given EPA’s information and the unquantified model uncertainty that remains, there is no sound technical basis for asserting with confidence, based on the models and analyses in EPA’s ozone risk assessment, that an ozone standard of 65 ppb would be any more protective than 70 ppb, or that 80 ppb is less protective than 60 ppb. To the contrary, available data suggest that further reductions in ozone levels will make no difference to public health, just as recent past reductions in ozone have had no detectable causal impact on improving public health.