Download this document (PDF)
View the Full Peer Review Document
My comments are presented as follows. First, I provide general comments about several broad topics in the revised circular that, in my opinion, warrant further attention and/or modification in the final version. I then respond to the specific questions that have been listed both in the peer reviewer charge and peer reviewer comment form. I conclude with some broad themes to consider in the final version.
1. Choice of the Social Discount Rate
The proposed revisions to Circular A-4 appropriately devote considerable attention to updating the guidance concerning the discount rate to be used in benefit-cost analysis as applied to government regulations. On balance, the 2023 Circular A-4 covers the main points of the literature on choosing the “correct” social discount rate. The following points should, however, be noted.
(1) The specific recommendation to use a default real after-tax discount rate of 1.7% should be reconsidered on several grounds. First, given the uncertainties involved in choosing the discount rate to pick a single value of 1.7% introduces an element of false precision. At the very least, an approximation of 2% (perhaps with some higher alternatives) would seem more defensible. Second, the time frame used to construct the proposed default real social discount rate encompasses a period during which the risk-free Treasury rate was kept intentionally low by quantitative easing and other expansive monetary policy. Thus, while there is a case for proposing a default real discount rate lower than the 2003 Circular A-4 default rate of 7%, 1.7% (or even 2%) may be lower than the longer-run discount rate. Something like a default discount rate of 3% with a lower value of 2% and an upper value of 4% or 5% may be more defensible.
(2) The possible role of the Ramsey approach to discounting is unclear. Are agencies being encouraged to use the Ramsey approach as an alternative to the default based on the risk-free real Treasury rate? Moreover, the actual value of the Ramsey rate depends entirely on assumptions made about its components. Depending on these assumptions it is possible to arrive at a Ramsey rate that is greater than the proposed default rate of 1.7%. For example, the UK government, which has adopted the Ramsey approach recommends use of a 3.5% real discount rate based on the assumptions of a 1.5% pure rate of time preference; a value of the marginal consumption elasticity of 1.0, and a long-run growth rate of 2.0%.
(3) The revised draft of Circular A-4 is correct in noting that when capital market distortions such as taxes, drive a wedge between the real return to capital and the individual discount rate, the conceptually correct approach for discounting is the so-called shadow price of capital approach. The challenge is the practical and consistent implementation of this method. The revised Circular A-4 presents an example based on the RFF Working Paper by Newell, Brest, and Pizer that could be used as a template by agencies. To the extent that OIRA wishes to encourage agencies to consider using the shadow-price approach, it would be useful to provide somewhat more detailed examples of how to implement the shadow price approach, if not in the circular itself, then in a web link. An approach consistent with the shadow price of capital framework can also be found in Mannix (OMB-2022-0014-3906).
(4) Finally, the proposed revision of Circular A-4 presents the case for possible applying a schedule of declining discount rates for policies (mainly environmental) with inter-general benefits. There is some support for this in the peer-reviewed literature cited in the revised Circular A-4. A concern, however, is that the precise level and “shape” of the declining discount rate schedule is likely to be quite sensitive to specific assumptions made about the underlying distribution of “uncertain” discount rates. Indeed, the specific schedule of declining discount rates that is presented in the preamble is offered with relatively little justification.
2. Distributional Effects in Regulatory Impact Analysis
Compared with the 2003 Version of Circular A-4, the proposed 2023 revision devotes considerable attention to incorporating distributional effects of regulation into regulatory impact analysis. Identifying and, where possible, estimating the distributional effects of proposed government regulations is clearly desirable. The question is how best to do so.
The critical first step in any analysis of distributional effects is a determination of the incidence of aggregate benefit and aggregate costs of regulation. Namely, how are both the projected benefits and the projected costs of regulation distributed among the relevant groups affected by the regulation? The word “relevant” is underscored because although a typical assumption may be that what matter is distribution by income group, this need not necessarily be the case, depending on the regulation under consideration. For example, in some cases, the relevant distributional effects may be geographical. Agencies would benefit from more guidance on this matter. Such guidance need not be incorporated in Circular A-4 itself, but perhaps could be provided in a link to a supplementary website focusing on “how to incorporate distributional effects.”
There are formidable conceptual and empirical challenges to identifying and estimating distributional effects. Simply determining who benefits and who bears the cost requires determining what public finance economists refer to as the economic incidence of regulatory benefits and regulatory costs. This may or may not correspond to what might be described as the initial impact of the regulation. Consider the case of a regulation that requires producers to reduce carbon emissions. The costs of complying with such regulations may be paid by producers, but may be shifted backward to workers through lower wages and forward to consumers in the form of higher prices.
Additional guidance to agencies on the basics of incidence analysis may be useful, indeed required, to facilitate some consistency in how distributional effects are analyzed. A possible model may be to adapt the manner in which economic incidence analysis of taxes is undertaken by the U.S. Treasury and the Congressional Budget Office to determining the economic incidence of regulation. Indeed it may be helpful to develop guidelines for undertaking distributional transfer analysis, analogous to benefit transfer analysis often used in environmental benefit-cost analysis.
Once the incidence of regulatory benefits and costs have been identified and estimate, the issue is that of how best to incorporate distribution into regulatory impact analysis. Although the proposed Circular A-4 revision recognizes that there various ways of taking distribution into account, its quasi-endorsement of “distributionally weighted” benefit-cost analysis is problematic in several ways.
(1) The acknowledged purpose of benefit cost analysis is to assess whether a particular policy change, such as a government regulation, enhances economic efficiency. While, from a welfare- analytic perspective, government regulations should be both economically efficient, and have desirable (or at least acceptable) distributional outcomes, the two objectives are distinct. The public interest is best served by presenting separate impacts of regulation on efficiency and equity rather than by attempting to combine the effects in a single weighted benefit-cost measure.
(2) The conceptual premise behind the suggested method of constructing distributional weights is relatively weak. As several public commenters – see Banzaf (OMB-2022-0014-0158} Sullivan (OMB-2022-0014-0029), Kenkel (OMB-2022-0014-3910) and Fraas (OMB-2022-0014-3917) -have noted, the common argument that it is reasonable to assume that the marginal utility of an additional $! of income may decline with income for a single individual is based more subjective views of what seems reasonable, rather than on empirical evidence; and moreover does not necessarily apply across individuals.
The preferable approach would be to incorporate distributional effects in a regulatory impact analysis in much the same way as distributional effects are presented separately from efficiency effects in the analysis of tax policy. Namely show how the benefits and costs are distributed among the relevant groups in addition to and separately from presenting any estimates of the policy’s impact on economic efficiency. The inherently subjective weighting of these separate effects is best left to decision-makers and the political process.
3. Geographical Scope
Guidance in the 2003 version of Circular A-4 states that the “….analysis should focus on benefits and costs that accrue to citizens and residents of the United States.” In the case of a regulation that “is likely to have effects beyond the borders of the United States, these effects should be reported separately.” Thus, while the inclusion of benefits and costs beyond the borders of the United States in regulatory impact analyses was not expressly prohibited in the 2003 version of Circular A-4, the inclusion of benefits and costs experienced by non-U.S. citizens is circumscribed.
The greater willingness to consider global benefits and costs that is found in the 2023 proposed revisions to Circular A-4 reflects a legitimate debate about how to treat “global social benefits and/or costs” particularly in the case of environmental problems whose scope is global rather than national – e.g. Smith (OMB-2022-0014-0079). One way of framing the issue in a manner that is entirely consistent with established benefit-cost principles is as follows. If one accepts the premise that the relevant social benefits should be based on the willingness to pay for environmental improvement of U.S. citizens, the question can be reframed as follows: (1) to what extent do U.S. citizens have a positive willingness to pay for environmental benefits that accrue to citizens in other countries, and if so, (2) How should this willingness to pay be estimated?
Scholarly research suggests that the answer to the first question is “yes,” while the answer to the second question may be that $1 of environmental benefit accruing to citizens of other countries would be valued at less than $1 of benefit accruing to U.S. citizens. This suggests that a conservative approach to incorporating global benefits and costs would be: (a) to include global benefits and costs separately, along with purely domestic benefits and costs, as recommended in the 2003 version Circular A-4, and (b) present a range of values for such global benefits, treating the full magnitude of global benefits and costs as “upper bound” estimates, and applying an appropriate discount to such values to represent the willingness to pay of American citizens. The revised draft is generally consistent with this approach, however the brief reference to situations where “such effects cannot be separated in a practical and reasonably accurate manner, or that the separate presentation of such effects would likely be misleading or confusing in light of the factors detailed above” is likely to confuse agencies and lead to inconsistencies across analyses.
4. Incorporating Insights from Behavioral Economics
The revised circular includes a discussion of the possible role of behavioral economics in regulatory impact analysis. The suggestion to draw upon behavioral economic insights such as through the use of nudges to implement regulations is useful. A potential caveat is that a recent survey of empirical studies (Mertens, et, al. 2021. “The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domains” Proceedings of the National Academy of Sciences) finds that nudges have small to modest effects on outcomes.
Caution, however, should be exercised in broadening the list of possible justifications for regulation to include “behavioral biases.” As Geyer and Viscusi (Journal of Benefit Cost Analysis, 2016) note: The evidence of systematic irrational behavior creates a conflict between two core principles of benefit-cost analysis (BCA): the Kaldor–Hicks principle and the principle of consumer sovereignty. The Kaldor–Hicks principle instructs the analyst to attempt to identify the outcome that maximizes the net benefits to the people subject to the policy options, while the principle of consumer sovereignty instructs the analyst to respect the choices that the people would make in determining what is best for themselves. If consumers are believed to be acting irrationally, then an analyst must choose between incorporating the benefits of a policy that addresses the self-harm done by an individual and respecting consumer sovereignty and thus ignoring such benefits, leading to a violation of the Kaldor–Hicks principle.
To the extent that behavioral biases are offered as a justification for government intervention, and are used as the basis for estimating presumed social benefits of intervention, agencies should be strongly encouraged to adopt something like the Geyer-Viscusi “behavioral transfer” test. Moreover, presentation of estimated benefits in a regulatory impact analysis should separate those benefits arising from correcting “traditional” market failures from market those that are attributable to behavioral biases.
5. Discussion of Uncertainty
Proposed revisions devote more attention to uncertainty than does the 2003 version. At time, however, the discussion of uncertainty seems to conflate two distinct aspects of when uncertainty is relevant. One important issue is how to best account for the unavoidable uncertainties encountered in estimating regulatory impacts. The revisions do provide a brief discussion of what is commonly called sensitivity analysis At minimum undertaking even a simple incremental sensitivity analysis should be required as a strongly recommended best practice in regulatory impact analysis. More sophisticated forms of sensitive analysis based on Monte Carlo simulations are increasingly accessible in excel-based programs such as Crystal Ball, and agencies should be encouraged to adopt these technologies.
The other dimension of uncertainty discussed in the revision pertains to what assumptions are appropriate about how individuals and businesses incorporate uncertainty into their decision. It is not always clear, however, about how different assumptions about uncertainty – e.g. risk neutrality vs, risk aversion – should be incorporated in regulatory impact analyses.
Specific Questions
- Please comment on whether the recommendations in the guidance are supported by the leading theoretical and empirical peer-reviewed academic literature in economics or other relevant disciplines, and if not, please provide alternative recommendations that would be (and citations to support them).
As someone who has taught benefit-cost analysis to hundreds of Master’s and Doctoral students for the past 20+ years, I would give the revised version of Circular A-4 high marks for its coverage of the relevant literature. - Where the guidance reflects assumptions, are they supported by the theoretical and empirical peer-reviewed academic literature in economics, or other relevant disciplines? If unsupported assumptions are identified, are there alternatives you would recommend? Please provide supporting references for both parts of the response—concerns about assumptions, if any, and suggested alternatives.
Several key regulatory analysis inputs discussed in the draft are particularly dependent on underlying assumptions, and the final circular would benefit from more clarity on how different assumptions would affect estimated benefits and costs. As noted above in general comment #2, although several citations are provided to support the case for cresting and using distributional weights, there is considerable disagreement, creating skepticism, among many economists about both the conceptual and the empirical basis for using such weights in benefit-cost analysis. - Does the guidance appropriately recognize and account for potential challenges for implementation (e.g., technical feasibility or constraints on data availability or other resources)?
The revised version of Circular A-4 lays out a fairly challenging agenda for undertaking regulatory impact analysis, including the implementation of distributional analysis, and possible use of the shadow price of capital approach to discounting. The draft revisions do not offer guidance that is detailed enough need to ensure that regulatory analysis is undertaken in a consistent and transparent manner across agencies of the federal government. One option would be to provide such guidance in an expanded version of Circular A-4, but this may not be the best alternative. An alternative would be to create supplemental websites that would be linked within A-4 to provide: (a) “best practice” templates for how to present results of regulatory analyses; (b) guidance for how to develop and present estimates of the distributional effects of regulations; and (c) particular good illustrations of regulatory analyses that embody best practices. An advantage of providing such information via websites is that advances in data availability, as well as empirical approaches to regulatory analysis could be regularly updated for the benefit of the federal regulatory community. - Do you have any other suggestions for improving the completeness, objectivity, and/or transparency of agency regulatory analyses? If so, how might these be incorporated into guidance?
OMB/OIRA might want to consider funding some workshops on the implementation of the revisions discussed both in Circular A-4 and Circular A-94. - What practices might be identified in the guidance to encourage accounting for non-monetized (possibly also non-quantified) effects?
The discussion of this topic in the 2023 revision is fairly thorough. I have two suggestions. First, it would be very useful either in the circular, or a supplemental website to encourage agencies to follow a structure set of steps in undertaking a benefit-cost analysis. A particularly useful list of such steps can be found in Boardman, et. al. Cost-Benefit Analysis: Concepts and Practice. One feature of such a list is that it makes clear that in conceptualizing a benefit cost analysis, all possible benefits and costs – intangible as well as tangible – should be identified and discussed. Second, it might be useful to provide some specific illustrations of common approaches for incorporating non-monetized effects into benefit-cost analysis – e.g. cases in which including an otherwise non-monetized benefit or cost would strengthen the findings; cases in which break-even analysis may provide insight into “how large” a non-monetized effect would need to be in order to affect the outcome. - Do you have suggestions that would improve the clarity and logical presentation of the guidance and/or ease execution of analyses?
I believe a number of my comments above offer suggestions. There is only so much guidance that can be offered in a written document such as Circular A-4. The overall quality and consistency of regulatory impact analyses would be improved by the development of supplemental materials and cases on a separate website, along with the offering of periodic workshops on how to do good and transparent regulatory analysis. - Should the guidance include suggestions of broadly useful data sets? If so, which data sets, and how should this information be presented in the guidance? How should the guidance reflect best practices related to data quality (including timeliness of data)?
Absolutely! An obvious starting point would be https://data.gov/, but some guidance as to which of the many data sets available on this website are likely to be the most useful for different types of regulatory analysis would be important to provide. This hearkens back to comments made above about the desirability of investing public resources in better educating agencies about how to undertake good regulatory analysis. - Do you have any additional recommendations for ensuring that the guidance and associated methodologies are supported by the theoretical and empirical peer-reviewed academic literature in economics, or other relevant disciplines? If so, please provide them here.
I have two additional comments. (1) I am not an expert in the economics of anti-trust, and I am aware that anti-competitive behavior by firms is a form of private market failure. Traditionally, this form of market failure has been dealt with by economic regulation undertaken by independent agencies such as the FTC, and by the anti-trust division of the Justice Department. In contrast, much of the original impetus for OIRA-style regulatory review came in response to the growth of what has been termed social regulation. While the effects of social regulation on market power should certainly be considered, it is not clear whether the revised circular is suggesting that the scope of OIRA regulatory review should be broadened to focus on market power as a source of market failure. In my opinion, moving in this direction would dilute the already-scarce staff resources needed to review social regulations. (2) Circular A-4 is intended to provide broad guidance about regulatory impact analysis generally certainly including, but not limited to environmental regulations. But, as several public commenters have noted, there appears to an emphasis on revising and improving environmental regulations. Somewhat more attention in the document should be given to other regulations, such as in the areas of product safety, occupational safety, etc.
Some Concluding Reactions
Several broad themes emerge from my review.
- Regulatory impact analysis should be transparent to policy officials who will base decisions on it and the public. There are several areas where the 2023 draft unintentionally supports approaches that may reduce that transparency. One key point that is made in the draft is that disaggregation, and clear presentation of the empirical inputs into an estimate conveys important information that collapsing information into a single number (e.g., distributional weights assigned to benefits or costs or exclusive use of global benefits) obfuscates.
- The 2023 draft is arguably too complex or sophisticated at points to be accessible by regulatory analysts, policy makers, and the public. Simple, clear guidance (supported by templates, web-based tools, and examples available separately, as suggested above) could yield better analyses in most cases.
- The draft provides agencies considerable flexibility in some areas—especially on discounting, accounting for how regulatory benefits and costs are distributed, and identifying the problem to be solved. While some flexibility may be appropriate to address different situations, consistency across agencies will be important if the federal government is to ensure its regulatory policies are targeted at the most pressing problems and cost-effective in addressing them. Not only would it be chaotic if every agency approached regulatory impact analysis differently, but the information value of the analysis would also suffer.
- Although it is beyond the scope of this peer review, it would be useful to make sure that principles and approaches for doing benefit-cost analysis that are presented in Circular A-4 are consistent with those discussed in Circular A-94.