Using Public Comments to Identify Regulations for Retrospective Review

April 23, 2020

Download this Commentary (PDF)


Each administration since President Carter has issued directives requiring federal agencies to review their existing regulations to make them more effective, more efficient, or to eliminate those that are outdated. Nonetheless, the difficulty of identifying which regulations to evaluate from the existing stock persists as a barrier to agencies in their efforts to implement retrospective review.

A report by the GW Regulatory Studies Center finds that analysis of public comments can help agencies pick which regulations to evaluate. In particular, we find that comments contain information about different forms of regulations (i.e., specific policy instruments such as labeling requirements or performance standards) that agencies could use to derive generalizable criteria (PDF) to guide their reviews. This finding builds on our previous study (PDF) demonstrating that different forms of regulation affect outcomes—such as productivity growth—in substantively different ways. In short, in addition to the feedback about individual regulations that commenters point out as candidates for review, the underlying characteristics of those regulations can be used by agencies to prioritize additional regulations for evaluation.

The Value of Retrospective Review

Retrospective review generates evidence about the effects of regulations after they have been implemented with the goal of either modifying them as needed or verifying that they are working satisfactorily to achieve their intended policy objectives. In addition to improving individual regulations, scholars note that retrospective review results in incremental learning that can help regulators improve their design of future regulations. For instance, reviews could generate more accurate data about compliance costs, distributional effects, or unintended consequences—all of which could improve future agency analysis. In short, retrospective review is necessary for generating evidence about the real-world outcomes of regulation and can improve future policymaking.

Although there is widespread agreement that retrospective review is a key component of evidence-based regulatory policy, assessments of agency performance suggest that it has yet to become systematically implemented anywhere in the world. Experts often attribute this outcome to the difficulty of identifying which regulations should be selected for review. Given that agencies have limited resources (e.g., time, staff), what criteria should agencies use to prioritize regulations from their existing stock as candidates for review?

Public Participation Could Assist Agencies

Agencies have used legislative and executive requirements to guide their identification of regulations. For instance, the Regulatory Flexibility Act requires agencies to review rules with a “significant economic impact on a substantial number of small entities” within ten years after a rule is published. Executive Order 13610 instructed agencies to prioritize paperwork burden reductions. Agencies also rely on their own expertise to determine which rules to review.

Agencies often leverage public participation to assist these efforts, and practitioners and scholars have theorized that public participation can help agencies overcome their identification problem. Nonetheless, few studies have systematically analyzed comments submitted to agencies on the subject of retrospective review to assess the extent to which comments provide useful input or how agencies might use that input to overcome lingering barriers to review.

To address this gap, we analyzed public comments solicited by agencies in response to President Trump’s Executive Orders 13771 and 13777. Agencies asked the public to help them identify regulations “for repeal, replacement, or modification,” pursuant to the administration’s “policy to alleviate unnecessary regulatory burdens.”

Analyzing Public Comments Solicited by Agencies

Our analysis of comments on agriculture-related regulations provided to the U.S. Department of Agriculture, the Food and Drug Administration, EPA supports the claim that public comments can provide information to help agencies identify which regulations to prioritize for review. We found that commenters provided relevant feedback to agencies about the kinds of regulations which our previous study estimated were likely to impose burdens. We also identified the set of forms associated with each regulation explicitly mentioned by commenters to demonstrate how agencies could use this analytical framework to generate broader criteria to identify additional regulations for retrospective review.

For instance, in our sample, the regulation mentioned most often by commenters was EPA’s National Pollutant Discharge Elimination System (NPDES). This provided agencies with feedback about specific regulations to review. However, the NPDES program employs different forms of regulation (i.e., it sets performance standards for water pollutants but also includes permitting and monitoring, reporting, and verification requirements). These are all command-and-control forms of regulation; our prior study found that increases in these forms of regulation were negatively associated with crop yield growth. Across all regulations mentioned by commenters, the top five forms were command-and-control regulations.

We analyzed comments submitted to agencies asking the public to help them identify burdensome regulations. However, agencies could structure future solicitations differently—inviting commenters to submit other kinds of information to assist their reviews. The takeaway for agencies is that empirically meaningful data about forms of regulation can be extracted from comments to provide generalizable criteria useful for overcoming the persistent barrier of identifying regulations as candidates for retrospective review. Additionally, this suggests that other underlying characteristics of the regulations explicitly mentioned by commenters (e.g., subject area, date of last amendment) could be used by agencies to go beyond the one-off identification of regulations by the public.