Section 1: Economic  Analysis For the purpose of this report, an economic analysis is defined as an  analysis that is intended to inform decision-makers and stakeholders  about the economic effects of an action. Economic effects (hereafter also  called “effects”) commonly include costs, benefits, and/or economic  transfers (for example, transfer payments). Action is defined to include a  government law, rule, regulation, project, policy, or program. An action  may be examined in the context of legislation, regulation, advocacy,  agency operations, or in response to certain events (such as a natural  disaster, for example). An economic analysis may be prospective,  examining an action that could be taken, or retrospective, examining the  outcome of an action that has already been taken. Examples of economic  analyses include:   1.  An economic analysis of the costs of a government program, project,  2.  An economic analysis of the benefits and costs of a government rule  3.  An economic analysis of the impact of a proposed or existing  regulation on regulated entities and consumers.  4.  An economic analysis of an action in response to an event (for  example, an analysis of a federal response to a natural disaster);  5.  A benefit-cost analysis or cost-effectiveness analysis. Section 2: Key  Elements of an  Economic Analysis GAO reviewed handbooks and guidance on economic analysis that have  been issued by various government agencies and institutions and  consulted with experts.  (Appendix 1 details GAO’s objective, scope, and  methodology.) In this Section, GAO synthetizes economic elements and  concepts embodied in this literature for use by GAO in its assessment  methodology for the review of an economic analysis. GAO identifies five key methodological elements to the baseline structure  of an economic analysis. These key elements are:   1.  Objective and scope— the objective and scope of the analysis.    2.  Methodology— the methodology used to examine the economic  effects.   3.  Analysis of effects — the analysis of economic effects.   4.  Transparency— the transparency of the analysis of economic effects.   5.  Documentation— the documentation included in the analysis. These key elements are standard to the structure of analyses, generally  speaking. That is, an analysis is performed to address an objective; the  analysis is scoped to address that objective; the analysis adopts a  methodology, which is used to analyze the economic effects of interest;  and the analysis is transparent and properly documented. The emphasis  on transparency is consistent with the final implementation guidelines of  the Office of Management and Budget (OMB). “The primary benefit of public transparency is not necessarily that errors in analytic results  will be detected, although error correction is clearly valuable. The more important benefit  of transparency is that the public will be able to assess how much of an agency’s analytic  result hinges on the specific analytic choices made by the agency.” Having identified key elements to the structure of an economic analysis  we synthetized for each key element, economic concepts embodied in the  literature that we reviewed. For example, what might we be assessing  under the key element: Objective and Scope? We considered economic  concepts commonly identified across the documents we reviewed, and  incorporated feedback from the experts and agencies with whom  we  consulted. The documents that we reviewed included, among others,  Circulars A-94 and A-4 released by OMB, handbooks for economic  analysis from federal and international agencies, such as the U.S.  Environmental Protection Agency, the U.S. Department of Defense, the  U.S. Department of Transportation, the Organization for Economic Co- operation and Development (OECD), and the United Kingdom’s HM  Treasury. These documents generally outline a methodical structure to an economic  analysis. This methodical structure takes the form of a set of issues or  sequence of steps to address while conducting the analysis. These  issues or steps, in turn, embody economic concepts. For the purpose of  developing our assessment methodology, we synthetized and  categorized these economic concepts with the key elements that we  identified—as listed below. The concepts for each listed key element are  not intended to be exhaustive and do not supplant or alter existing  requirements for economic analysis. Depending on the context in which  an action is examined, GAO’s assessment of a key element could  exclude some concepts, or extend beyond the concepts listed for that  element. In such cases, GAO’s written assessment of the relevant key  element will specify the concepts that were actually considered in the  review process. The five key elements and economic concepts in GAO’s assessment  methodology for an economic analysis are:  1.  Key Element: Objective and Scope—the objective and scope of the  analysis. The economic analysis explains the action examined and includes a  rationale and justification for the action. The analysis states its  objective. The scope of the analysis is designed to address this  objective. Unless otherwise justified, the analysis focuses on  economic effects that accrue to citizens and residents of the United  States, and its time horizon is long enough to encompass the  important economic effects of the action.  2.  Key Element: Methodology—the methodology used to examine the  economic effects. The economic analysis examines the effects of the action by  comparing alternatives, using one of them as the baseline. Unless  otherwise justified, it considers alternatives that represent all relevant  alternatives, including that of no action. The analysis defines an  appropriate baseline. The analysis justifies that the world specified  under each alternative considered (including the baseline) represents  the best assessment of what the world would be like under that  alternative. The analysis identifies the important economic effects for  each alternative considered, their timing, and whether they are direct  or ancillary effects.  3.  Key Element: Analysis of Effects—the analysis of economic effects. Where feasible, the economic analysis quantifies the important  economic effects and monetizes them using the concept of  opportunity cost. The analysis applies the criterion of net present  value, or related outcome measures, to compare these effects across  alternatives. It controls for inflation and uses economically justified  discount rates. Where important economic effects cannot be  quantified, the analysis explains how they affect the comparison of  alternatives. Where the equity and distributional impacts are  important, the full range of these impacts is separately detailed and  quantified, where feasible.  4.  Key Element: Transparency—the transparency of the analysis of  economic effects. The economic analysis describes and justifies the analytical choices,  assumptions, and data used. The analysis assesses how plausible  adjustments to each important analytical choice and assumption affect  the estimates of the economic effects and the results of the  comparison of alternatives. The analysis explains the implications of  the key limitations in the data used. Where feasible, the analysis  adequately quantifies how the statistical variability of the key data  elements underlying the estimates of the economic analysis impacts  these estimates, and the results of the comparison of alternatives.  5.  Key Element: Documentation—the documentation included in the  analysis. The economic analysis is clearly written, with a plain language  summary, clearly labeled tables that describe the data used and  results, and a conclusion that is consistent with these results. The  analysis cites all sources used and documents that it is based on the  best available economic information. The analysis documents that it  complies with a robust quality assurance process and, where  applicable, the Information Quality Act.  The analysis discloses the  use and contributions of contractors and outside consultants. In summary, GAO identifies five key elements, with associated economic  concepts, to the structure of an economic analysis. GAO’s assessment  methodology then examines the extent to which an economic analysis  properly dealt with these key elements. Section 3:  Assessment  Methodology GAO’s assessment methodology has two steps: (1) an assessment of  each individual key element and (2) an overall assessment based on the  assessment of the individual key elements. Below, these two types of  assessment are discussed. While GAO’s assessment methodology  typically considers all five key elements, there may be cases, for example  depending on the scope of an engagement, where it may consider only  certain key elements. In those cases, GAO may not be able to make an  overall assessment. Step 1: Assessing Each  Individual Element The first step in the review process is an assessment of the extent to  which the economic analysis has considered and properly dealt with each  key element. For each element, the outcome of the review is a written  assessment and an assessment score. The written assessment details  the extent to which the analysis considered and properly dealt with the  element. To the extent that important limitations are identified in the  review, the written assessment describes these limitations. This written  assessment informs the assessment score, which is one of three mutually  exclusive scores:  1.  fully met—that is, the economic analysis has considered and properly  dealt with the element;  2.  partly met—that is, the economic analysis has only partly considered  and properly dealt with the element;   3.  not met—that is, the economic analysis has not considered or not  properly dealt with the element. If the outcome is “partly met” or “not met,” the written assessment should  describe the limitations of the analysis. Assessments are made from expertise in economics and with  professional judgment. The guiding principles of a review are objectivity,  integrity, and compliance with generally accepted government auditing  standards (GAGAS). An assessment is contextual—that is, it is  conditional on the evidence underlying the action examined, and the  context in which it takes place. The assessment is also conditional on the  reasonably obtainable information available at the time of the economic  analysis that is being reviewed. A caveat may be added to the  assessment if new information that would affect it, has become available  and is reasonably obtainable since the analysis was made. Should the  review of a key element exclude certain concepts, or extend beyond the  concepts listed for that key element, GAO’s written assessment of the key  element will specify the concepts that were actually considered in the  assessment process. Step 2: Overall  Assessment Once each element has been individually reviewed, an overall  assessment is made of the extent to which the economic analysis  accordingly informs decision-makers and stakeholders about the  economic effects of the action examined. Four outcomes are possible:   1.  The analysis informs decision-makers and stakeholders about the  economic effects of the action examined.  2.  The analysis informs, with caveats, decision-makers and stakeholders  about the economic effects of the action examined. (If this is the  outcome, the review should describe the caveats in writing.)  3.  The analysis needs additional work to inform decision-makers and  stakeholders about the economic effects of the action examined. (If  this is the outcome, the review should describe in writing the important  limitations of the economic analysis.)  4.  The analysis does not inform decision-makers and stakeholders about  the economic effects of the action examined. (If this is the outcome,  the review should describe in writing the deficiencies of the analysis.) An economic analysis that has fully met all the key elements should be  identified as informing decision-makers and stakeholders about the  economic effects of the action examined (this is outcome 1). This  determination is neither an endorsement of the specific findings and  conclusions of the analysis, nor is it a determination that these are  correct. For example, a prospective analysis is predictive of a potential  result, but cannot definitively determine the result. It is a determination  that the analysis is adequately and properly designed, and accordingly, it  can inform the public discourse about the economic effects of the action  examined. The written statements added to outcomes 2–4 should refer to the review  of the individual elements. The difference between outcomes 2 and 3 is a  matter of degree and professional judgment. Generally speaking, the  caveats under outcome 2 will be relatively minor or few, whereas the  important limitations under outcome 3 are likely to be more consequential.  Should the economic analysis suffer from major deficiencies in meeting  the key elements, then outcome 4 may be appropriate. Appendix I: Objective, Scope, and  Methodology Our objective was to identify, for the purpose of developing GAO’s  assessment methodology for the review of economic analysis, key  methodological elements to the structure of an economic analysis that is  intended to inform decision-makers and stakeholders about the economic  effects of a public action. To address this objective, we reviewed existing Circulars issued by the  Office of Management and Budget (OMB), handbooks for economic  analysis issued by federal agencies, international government agencies  and institutions, and established textbooks on economic theory and  benefit-cost analysis. We also solicited feedback from economics experts  (in academia and public policy) and international audit agencies. We identified in our document review, five key elements to the structure  of an economic analysis. These key elements are: (1) objective and  scope; (2) methodology; (3) analysis of effects; (4) transparency; (5)  documentation. These key elements are standard to the structure of  analyses, generally speaking. That is, an analysis is performed to  address an objective; the analysis is scoped to address that objective; the  analysis adopts a methodology, which is used to analyze the economic  effects of interest; and the analysis is transparent and properly  documented. Having identified key elements to the structure of an economic analysis,  we synthetized for each key element economic concepts embodied in the  literature we reviewed. For example, what might we be assessing under  the key element objective and scope? To do so, we looked for economic  concepts commonly identified across the documents we reviewed and  incorporated feedback from the experts and agencies that we consulted  with. We then categorized these economic concepts across the key  elements that we identified. Among the documents we reviewed in our process were the following: OMB Circular A-94, Guidelines and Discount Rates for Benefit-Cost  Analysis of Federal Programs, Revised (Oct. 29, 1992). Circular A-94  provides a checklist of whether an agency has considered and  properly dealt with all the elements for sound benefit-cost and cost- effectiveness analyses. OMB Circular A-4, Regulatory Analysis, (Sept. 17, 2003). Circular A-4,  released in collaboration with the Council of Economic Advisors,  identifies key elements to the structure of economic analyses in  regulatory proceedings. Office of Management and Budget, Office of Information and  Regulatory Affairs, Regulatory Impact Analysis: A Primer  (Washington, D.C.: The White House). The purpose of the primer is to  offer a summary of OMB Circular A-4. Agency-issued handbooks for economic analysis, such as, for  example, those issued by the U.S. Environmental Protection Agency,   the U.S. Department of Defense, the U.S. Department of  Transportation, and the Organization for Economic Co-operation and  Development (OECD). We also reviewed The Green Book: appraisal  and evaluation in central government, issued by HM Treasury,  Government of the United Kingdom, (London; July, 2011). HM  Treasury describes The Green Book as a best practice guide for all  central departments and executive agencies, and covers projects of  all types and size. The guide applies to appraisals—defined as any  analysis used to support a government decision to adopt a new policy,  or to initiate, renew, expand or re-orientate programs or projects that  would result in measurable benefits and/or costs to the public—and  evaluations—defined as retrospective analysis of a policy, program or  project at its completion, conclusion or revision. The National Academies of Sciences, Engineering, and Medicine,  Guidelines for the review of Reports of the National Academies of  Sciences, Engineering, and Medicine. While these guidelines are  specific to the review of reports issued by the National Academies and  outline review criteria that apply across a broad range of disciplines,  not just economics, they provide review criteria for scientific analysis. We conducted our work from June 2017 to April 2018 in accordance with  all sections of GAO’s Quality Assurance Framework that are relevant to  our objectives. The framework requires that we plan and perform the  engagement to obtain sufficient and appropriate evidence to meet our  stated objectives and to discuss any limitations in our work. We believe  that the information and data obtained, and the analysis conducted,  provide a reasonable basis for any findings and conclusions in this  product. Third-party Comments We provided a draft of this product for third-party outside review to  experts at various U.S. and international government agencies and  specialists. Reviewers provided technical comments which we  incorporated as appropriate. Appendix II: GAO Contact and Staff  Acknowledgments GAO Contact Staff  Acknowledgments In addition to the contact named above, Carol Bray, Timothy Carr, Tim  Guinane, Kathleen Scholl, Paola Tena and Elaine Vaurio made key  contributions to this report.
We prepared this report to answer the question: What are key methodological elements of an economic analysis that is intended to inform decision-makers and stakeholders? GAO identifies five key methodological elements to the baseline structure of an economic analysis: Objective and scope, Methodology, Analysis of effects, Transparency and Documentation. GAO's assessment methodology evaluates each key element and provides an overall assessment based on the assessment of the individual key elements.