One (un)remarkable problem?

November 24, 2015

How would you feel if someone called you unremarkable? Insulted, offended or even hurt? But, if the messenger is a doctor, you should be relieved. In the medical world, being unremarkable means you’re fine. This example of an everyday communication gap is presented in a book by Naomi Karten to show that words and phrases can mean different things in different contexts, creating a potential communication gap. Also, different terms can apply to the same concept depending on the context or field of study. The public policy world is not exempt from these two types of misunderstandings. The renewed emphasis on retrospective regulatory review is a case in point.

Progress through “lookback” reviews of regulations

The U.S. Congress and the Executive have implemented different initiatives to evaluate the effectiveness of existing regulations. It has been a goal of the Executive Branch since President Jimmy Carter. The Regulatory Flexibility Act (§610) of 1980 required a periodic effectiveness analysis for rules that have significant economic impact on small entities. President Obama is the latest president to focus on retrospective review; he has committed to increase the application of “lookback” reviews through Executive Orders 13563, 13610 and 13579. Moreover, different U.S. Senate Committees have had several hearings to find ways to improve the regulatory system. Just a few months ago, on September 16, 2015, the Senate Committee on Homeland Security and Governmental Affairs held a hearing titled “A Review of Regulatory Reform Proposals” where Susan Dudley and Sidney Shapiro raised the value of retrospective review. As a result of these policy guidelines, regulatory agencies now post plans for retrospective review on line.

Retrospective review challenges remain

Despite all these developments, some challenges still exist in the U.S. to systematically conduct retrospective review. In fact, a study of the OECD indicates that the United States, together with countries such as Greece, Ireland, Spain, Slovak Republic, and Turkey, never conduct ex post evaluation, never include sunset clauses, and never include automatic evaluation requirements on primary laws (approved by the legislature). Furthermore, as Sofie Miller finds in a new working paper, Learning from Experience: Retrospective Review of Regulations in 2014, “agencies are not designing their rules to facilitate ex post measurement, and are not prospectively planning for retrospective review at the outset of rulemaking.”

There is an ongoing debate on the most effective institutional oversight, procedural requirements and methods needed for a well-functioning retrospective review system. Some of the discussions include whether the review should be conducted by an independent entity, which should be in charge of prioritizing the rules for analysis, among other issues. Irrespective of the model that will be followed it is necessary to bring experts and build capacity within the entities involved. However, the creation of those capacities and expertise need not be built from zero.

Ex post evaluation is still ex post evaluation

As a student of both regulatory policy and program evaluation, I argue that one of the reasons that successful retrospective review in the regulatory arena seems challenging is the language used. The U.S. government uses terms such as “lookback efforts,” “retrospective review" and “review of existing rules.” These may be new terms, but the concepts are not.  Retrospective review is simply a form of ex post evaluation, which according to the OECD “can be the final stage of the regulatory policy cycle, evaluating the extent to which regulations met the goals they were designed for.” The analytical tools for doing this are very familiar to those in the program evaluation field.

Program evaluators strengthen the conversation

We don’t need to start from scratch in the regulatory field. The objective of a retrospective review is the same as that of the long standing and well-established field of program evaluation. We already have a flourishing cadre of program evaluation professionals, offices within agencies, tested methodologies, and even lessons learned. Inviting program evaluation experts to the regulatory reform debate will allow the government as a whole to go forward towards assuring existing regulations are operating as intended. Regulatory officials in the U.S. and countries around the world should take advantage of program evaluators’ expertise and experience to improve implementation of “retrospective reviews.”