In revisiting the National Ambient Air Quality Standards (NAAQS) for ozone, EPA recently concluded that current standards do not fully suffice to protect public health with an adequate margin of safety and that further reductions would probably further reduce mortalities and morbidities in the population. Central to this conclusion is EPA's determination that "O3 exposures are causally related to respiratory effects, and likely causally related to cardiovascular effects, and that long term O3 exposures are likely causally related to respiratory effects." Remarkably, this key conclusion is not supported by any reliable, objective statistical tests for potential causality. It rests solely on the subjective judgments of selected experts, applied to associational data that show that both ozone levels and adverse health effects are higher in some times and places than in others.
The track record of such expert judgments is poor, and such associational data do not in general provide reliable information about causation. Previous confident expert claims that reductions in air pollutants have caused improvements in public health have turned out to be unwarranted, mistaking the observation that both air pollution levels and rates of adverse health effects have decreased over time for evidence that decreases in air pollution caused reductions in adverse health outcomes. In reality, health outcomes improved at least as quickly where pollutant levels were not reduced as where they were, showing the original expert judgments of causation to be unfounded.
More objective, reliable statistical methods for testing causal hypotheses, such as that reducing ozone levels reduces adverse health effects, are now widely available. They have been extensively developed and applied outside the field of air pollution epidemiology in areas such as econometrics, social statistics, neuroscience, physics, artificial intelligence, and machine learning. Major companies such as Google, that make or lose money depending on how well they understand the causal relation between what they do and how customers respond, have contributed to a growing body of high-quality statistical algorithms and software for testing causal hypotheses and estimating causal impacts. Modern methods of causality testing are based on sound principles, such as that information flows from causes to their effects and that effects cannot be made statistically independent of their direct causes by conditioning on other variables, such as suspected confounders. These principles lead to independently reproducible and verifiable quantitative tests and conclusions about causality, rather than to subjective qualitative judgments that differ from expert to expert.
In contrast to the expert opinions relied on by EPA in proposing that a further reduction in ozone would be protective of public health, applying more objective and reliable statistical methods reveal no causal relation between past ozone reductions and past improvements in public health. The absence of any apparent causal impact of past ozone reductions on public health is nether mentioned nor explained in EPA’s health effects risk assessment (HERA) documents, which focuses instead on predicting substantial future human health benefits from future reductions in ozone. The technical basis for these predictions, is, by EPA’s own admission, based on new and invalidated models for which “We are unable to properly estimate the true sensitivities or quantitatively assess the uncertainty.” The HERA also project human health benefits by misinterpreting the slopes of descriptive models (curves fit to past ozone levels and public health effects data) as if they were causal models capable of showing policy makers how changing future ozone levels would change future public health. This is a fundamental logical and statistical error, on a par with dividing population-wide car accident rates by population-wide consumption of potatoes, and then concluding, based on the resulting positive ratio (or “slope factor”), that each unit of reduction in potato consumption would bring about a corresponding reduction car accident rates. What is missing, in this example and in EPA’s HERA for ozone based on associational data, is any causal relation between past or future changes in the denominator and corresponding changes in the numerator.
Whether environmental regulations in the United States should be based on the judgments of selected experts or on independently reproducible and verifiable statistical analyses of causation raises several important questions about what relation we want between science and policy-making. Should projections of human health benefits from tighter regulations, and resulting risk management policy decisions, reflect “sound science,” meaning objective and reproducible results, or should the presumed wisdom and insight of selected experts be used to override science-based conclusions when and if the two conflict?
In principle, expert judgments based on qualitative considerations and evaluations of all the evidence would not be in tension with the results of relatively reliable and objective statistical methods for causal analysis. Rather, the latter would inform the former. In practice, EPA’s selected experts express high confidence that ozone-health effects associations are “causal” (presumably meaning entirely causal, with no part due to confounding or modeling biases or coincident historical trends), implying the prudence and health protective nature of EPA’s proposals to revise the ozone NAAQS downward. But objective statistical tests reveal no such causal relation in abundant data from the past several decades. What to do next, in the face of predictions that appear to conflict with past reality, will say much about what role if any, we collectively want science and objective causal analysis to play in shaping crucial environmental and public health risk management policies and regulations.