Background EM’s cleanup sites and areas of cleanup work, EM’s status as a program,  the history of EM’s requirements for operations activities, and  key EM offices and DOE oversight bodies for EM’s cleanup work. EM Cleanup Sites and  Areas of Cleanup Work EM has a headquarters office and 16 sites at which the agency oversees  cleanup work. Figure 1 shows the EM sites where cleanup work remains. EM divides its cleanup work into six work areas. These areas, described  below, sometimes include both operations activities and capital asset  projects:  1.  spent nuclear fuel stabilization and disposition, including safe  shipping, receipt, storage, and disposition of spent nuclear fuel and  heavy water;  2.  nuclear materials stabilization and disposition, including the  management, disposition, safe surveillance, and maintenance of  nuclear materials;  3.  radioactive liquid waste stabilization and disposition, including  treatment, management, and permanent disposal of radioactive liquid  waste stored in storage tanks;  4.  nuclear facility decontamination and decommissioning, including the  deactivation, decontamination, and decommissioning of EM-owned  nuclear, radioactive, and industrial buildings and structures;  5.  solid waste stabilization and disposition, including receipt, treatment,  storage, and disposal of legacy and newly generated low-level  waste, mixed low-level waste, transuranic waste, hazardous waste,  and sanitary waste; and  6.  soil and water remediation, including cleanup of waste regulated  under the Resource Conservation and Recovery Act and the  Comprehensive Environmental Response, Compensation, and  Liability Act. EM’s Status as a Program EM refers to itself as a program, and EM’s organization and mission fit  PMI’s definition of a program. According to PMI, programs include  multiple program components, such as sub-programs (in EM’s case, each  cleanup site is a sub-program) and projects (in EM’s case, the cleanup  work at each site), which are interrelated and managed in a coordinated  way to obtain benefits not available from managing them individually.  According to PMI officials, organizations often use the terms “program”  and “project” interchangeably, but the two terms have different meanings  and apply to different levels of management. Programs are a means of  executing a strategy and achieving organizational goals and objectives. A  program may continue indefinitely. In contrast, a project is a temporary  endeavor undertaken to create a unique product, service, or result.  Projects are executed to improve the efficient implementation of a  program. The relationship between a program and a project is illustrated  in figure 2 below. History of EM’s  Requirements for  Operations Activities In June 2009, EM developed the category of work that EM calls  operations activities to differentiate this work from capital asset projects.  Until then, EM managed all of its cleanup work as projects under Order  413.3B. EM documentation from that time explained that EM decided to  differentiate its cleanup work so that it could quickly make use of an  infusion of $6 billion for EM under the American Recovery and  Reinvestment Act of 2009 (Recovery Act). EM officials stated that EM  could not use the funds quickly at that time if the work had to follow the  project management requirements in Order 413.3B. In 2010, shortly after the initiation of the Recovery Act work, EM decided  to make the approach of managing part of its work as operations activities  permanent. EM officials could not provide any documentation from the  time supporting this decision, which was not consistent with EM findings  from 2009. In particular, according to EM documentation from 2009,  executing all cleanup work under Order 413.3B had served EM well in  defining and controlling the technical scope, project and life-cycle costs,  completion dates, and risks of its cleanup work, and had helped EM  improve its overall performance and become more efficient. EM began  managing operations activities based on a memorandum developed by  EM leadership. In 2012, EM developed the operations activities protocol, which  superseded the 2010 memorandum for managing operations activities.  This protocol stated that although operations activities are not subject to  DOE’s Order 413.3B requirements, EM will apply the appropriate project  management principles from this order using a “graded approach.” We  reviewed the 2012 operations activities protocol in October 2012 and  found that it contained less stringent requirements for operation activities  than Order 413.3B for capital asset projects. We also found that EM did  not have a clear classification policy that set out under what conditions  EM should consider particular cleanup work to be an operations activity or  a capital asset project. In the absence of such a policy, EM classified as  operations activities certain cleanup work that DOE’s Office of Project  Management considered to be capital asset projects. We recommended  that EM provide DOE’s Office of Project Management with information on  EM’s classification decisions. In 2012, DOE agreed with our  recommendation, and EM officials stated in August 2018 that they are  developing guidance. In July 2017, EM developed a cleanup policy that applies to both  operations activities and capital asset projects. For managing capital  asset projects, this policy supplements Order 413.3B. For managing  operations activities, this policy supersedes the 2012 operations activities  protocol. The 2017 cleanup policy states that EM will apply DOE’s  project management principles described in Order 413.3B to its  operations activities in a tailored way. At the time of our review, EM had developed 11 standard operating  policies and procedures that are associated with the 2017 cleanup policy  and that provide guidance on areas such as program performance  reporting, assessing contractors’ performance against contract  requirements, and what officials have approval authority at major steps in  the contract process. However, according to EM officials, the standard  operating policies and procedures are not requirements. Key EM Offices and DOE  Oversight Bodies for EM’s  Cleanup Work The EM program is executed by two main components: EM headquarters,  which serves as the program manager for the EM program, and 16  cleanup sites, which serve as sub-programs. The following EM  headquarters and site officials are key to managing and overseeing EM’s  operations activities, according to the 2017 cleanup policy: The Assistant Secretary for Environmental Management serves  as the head of EM and is responsible for the execution of EM’s  mission. In December 2017, the Assistant Secretary for EM began  reporting to the DOE Undersecretary of Science, who in turn reports  to the DOE Deputy Secretary of Energy. The Assistant Secretary for  Environmental Management, among other things, provides leadership  and develops mission strategies, policy, and guidance for the EM  cleanup program. The Principal Deputy Assistant Secretary for Environmental  Management serves as the EM management official responsible for  operations, including coordination, oversight, and leadership on  scope, cost, and schedule elements. Under the 2017 cleanup policy,  this official has approval authority for contracts equal to or greater  than $200 million. This official is also responsible for conducting  periodic contract reviews for contracts with a total estimated cost  equal to or greater than $200 million. The Associate Principal Deputy Assistant Secretary for Field  Operations provides leadership and develops mission strategies,  policy, and guidance for site operations. This official is responsible for,  among other things, meeting monthly with each site individually to  discuss the status of cleanup work there. The EM Deputy Assistant Secretary for Acquisition and Project  Management is responsible for providing independent oversight and  reports to the Associate Principal Deputy Assistant Secretary for  Corporate Services. Under the 2017 cleanup policy, this official is  responsible for programmatic peer reviews that review cleanup  activities at each site. This official is also responsible for the  implementation of Order 413.3B and review of capital asset projects. At each of the 16 cleanup sites, the EM site manager is responsible  and accountable for management and integration of all EM site-level  activities. Under the 2017 cleanup policy, site managers have  approval authority over contracts under $200 million. The site  manager is also required to conduct periodic contract reviews for  contracts with a total estimated cost of less than $200 million. Outside of EM, two DOE bodies play a role in the oversight of EM’s  capital asset projects, but not of operations activities: DOE’s Office of Project Management has served as DOE’s enterprise  project management organization since July 2015, when the  Secretary of Energy gave it this responsibility as part of an initiative to  improve DOE’s program and project management. As such, DOE  states that this office—as an enterprise project management  organization—is responsible for providing leadership and assistance  in developing and implementing DOE-wide policies, procedures,  programs, and management systems pertaining to project  management, as well as for independently monitoring, assessing, and  reporting on project execution performance. Officials from this office  are experts in project management, especially as it relates to capital  asset projects, and oversee the implementation of DOE’s Order  413.3B. This office also validates project performance baselines— scope, cost, and schedule—for the department’s capital asset  projects, including EM’s. The Project Management Risk Committee reviews and provides  advice on capital asset projects with a total project cost of $100 million  or more. The Risk Committee’s purpose is to assess the risks  associated with projects across DOE and advise DOE senior leaders  on project management, including on cost, schedule, and technical  issues. The committee includes nine senior DOE officials from across  the department, including top project management officials from the  National Nuclear Security Administration, the Office of Science, and  EM. DOE’s EM Program  Manages Most of Its  Multibillion-Dollar  Cleanup Work as  Operations Activities,  Posing Cost and  Schedule Risks DOE’s EM program manages most of its cleanup work as operations  activities, posing cost and schedule risks. These risks stem from EM’s  management of such work using less stringent requirements than for  capital asset projects even though EM spends billions of dollars annually  on operations activities. Site managers have the discretion to classify  cleanup work as operations activities, even if the work has characteristics  of capital asset projects, because DOE and EM have not established  requirements for classifying EM’s cleanup work. In addition, EM has not  addressed concerns raised by DOE project management experts that  some operations activities should be classified as capital asset projects. DOE’s EM Program  Manages Most of Its  Cleanup Work as  Operations Activities,  under Less Stringent  Requirements Than  Capital Asset Projects EM manages its cleanup work under different requirements, depending  on whether it classifies the work as a capital asset project or an  operations activity, with operations activities having less stringent  requirements. EM currently manages most of its work as operations  activities. EM’s work is divided into 77 operations activities and 20 capital  asset projects. In the fiscal year 2019 budget, operations activities  accounted for 77 percent of EM’s approximately $7.2 billion budget— about $5.5 billion—while capital asset projects accounted for 18 percent  of EM’s budget—about $1.3 billion. Figure 3 illustrates how EM  classified and funded its work during fiscal year 2019. For capital asset projects, EM manages the work in accordance with the  requirements in DOE’s Order 413.3B, which is DOE’s project  management order. This order contains numerous, detailed requirements  that describe the steps and project management best practices to follow  throughout the life of a project. The DOE Secretary strengthened this  order in May 2016 by adding more stringent requirements, based in part  on our prior recommendations. Examples of the requirements included  in this order include:    A capital asset project with a total project cost over $50 million must  undergo rigorous reviews outside the project’s management line.  Different types of reviews are to be conducted by an independent  body within the program for capital asset projects over $50 million,  DOE’s Office of Project Management and the Project Management  Risk Committee for capital asset projects over $100 million, and the  Energy Systems Acquisition Advisory Board for capital asset projects  over $750 million. Review and approval are to be received from the  Under Secretary for capital asset projects over $100 million, and the  Deputy Secretary for capital asset projects over $750 million. A capital asset project must complete its original scope of work within  110 percent of the original cost baseline to be considered  successful. The program must conduct a root cause analysis to determine the  underlying contributing causes of cost overruns, schedule delays, and  performance shortcomings, if the program, the project manager or  independent oversight offices realize a capital asset project can no  longer meet its established scope, cost or schedule baseline. Contingency to cover potential risks that might appear during the life  of a project must be included as part of the total project cost estimate  included in the performance baseline. All cost and schedule estimates developed during the life of the  project must follow GAO best practices. For operations activities, EM follows the requirements in its 2017 cleanup  policy, which has fewer, less detailed, and less stringent requirements  than Order 413.3B. For example, in contrast to the more stringent  requirements in Order 413.3B, under EM’s 2017 cleanup policy: The highest level of review an operations activity must receive is by  EM’s top management for contracts equal to or greater than $200  million. For an operations activity to be considered successful, it must be  completed within 110 percent of the current cost and scope  baseline—not the original baseline established at the beginning of  cleanup work. There is no requirement to conduct a root cause analysis for  operations activities. EM does not fund contingency for operations activities. Cost and schedule estimates made before EM authorizes execution of  a contract are to follow GAO best practices, but the policy does not  include a requirement to follow best practices for cost estimates  developed during contract execution. Figure 4 below illustrates how  operations activities are managed under less stringent requirements  than capital asset projects. EM project management officials in charge of developing the 2017  cleanup policy stated that EM intentionally wrote this policy at a high level  because EM planned to develop standard operating policies and  procedures that would establish more detailed steps to implement the  policy. As noted earlier, these standard operating policies and procedures  provide guidance but are not requirements. DOE and EM Have Not  Established Requirements  for Classifying EM’s  Cleanup Work or  Addressed Concerns That  Some Operations  Activities Should Be  Capital Asset Projects Neither DOE nor EM has a policy on how to classify cleanup work as  either operations activities or capital asset projects. According to DOE  Office of Project Management officials, DOE does not have a department- wide policy on how to classify cleanup work. Instead, these officials stated  that DOE’s general management approach is to let its individual  programs, such as EM, decide how to classify their work. EM officials  explained that EM allows each site manager to determine independently  how to classify cleanup work because according to EM’s 2017 cleanup  policy, the site manager is responsible and accountable for the planning  and execution of all site-level activities. DOE project management experts on the Project Management Risk  Committee and in DOE’s Office of Project Management have raised  concerns related to EM’s 2017 cleanup policy and the classification of  cleanup work since 2015. These officials have stated that some current  operations activities should be classified as capital asset projects.  Specifically:  In November 2015, EM approached DOE’s Project Management Risk  Committee with a proposal for a new cleanup policy, which later  became EM’s 2017 cleanup policy. In comments on the proposal, the  committee’s members expressed concerns that the proposed policy  did not address how EM would classify cleanup work, noting that if  programs or sites get to decide on what is a capital asset project and  what is not—which in turn drives the level of DOE oversight—then this  approach was not an appropriate governance model. The committee’s  members also questioned why EM chose not to use the already  available requirements in Order 413.3B. EM did not respond to the  committee’s concerns. Instead, according to the committee’s meeting  minutes, the DOE Undersecretary for Management and Performance,  who at the time oversaw EM, informed the committee in November  2015 that EM was proceeding with drafting its new cleanup policy. In late 2016, DOE’s Office of Project Management officials drafted an  appendix to Order 413.3B that sought to define operations activities  and capital asset projects. Under the classification proposal in the  draft appendix, some of the work now classified as operations  activities would have become capital asset projects and subject to  more stringent requirements. For example, under the appendix, the  cleanup of radioactive liquid waste tanks and solid waste exhumation  and disposition would have been designated as capital asset  projects. However, EM officials informed officials from the DOE  Office of Project Management that EM would continue to develop its  own policy, which it issued in July 2017. This 2017 cleanup policy did  not reclassify any of the operations activities that, in the opinion of  DOE’s Office of Project Management, should be capital asset  projects. Officials from DOE’s Office of Project Management we interviewed said  that continuing to classify and manage most of EM’s cleanup work as  operations activities poses significant risks to DOE. According to these  officials, managing the work this way poses cost and schedule risks for  the following reasons, among others: Because the review of operations activities is conducted entirely  within EM, DOE does not have information on how EM manages  operations activities and cannot hold EM accountable for cost- effective and timely completion of this cleanup work, which represents  a $5.5 billion investment by taxpayers in operations activities in fiscal  year 2019 (see fig. 3). Operations activities are not required to go through a thorough upfront  planning process to determine the scope of work to be completed.  Therefore, these activities are more subject to scope creep, cost  overruns, and schedule delays, which can detract from EM’s  credibility with Congress and other stakeholders. Because EM does not set aside contingency funds to cover risks for  its operations activities—a project management best practice and  requirement under Order 413.3B—if risks are realized, EM must either  reduce or delay scope to later years, which increases costs, causes  schedule delays, and undermines EM’s ability to budget for activities  across the EM program. Officials from DOE’s Office of Project Management stated that EM did not  respond to their concerns that EM’s approach to classification of cleanup  work poses unwarranted cost and schedule risks. Officials in EM told us  they view the role of DOE’s Office of Project Management and the Project  Management Risk Committee as limited to reviewing Order 413.3B  requirements and overseeing capital asset projects. However, since July  2015, DOE’s Office of Project Management has served as DOE’s  enterprise project management organization, with department-wide  responsibilities for overseeing project management. As previously noted,  DOE states that this office is responsible for, among other things,  independently monitoring, assessing, and reporting on project execution  performance. Therefore, review of classification of cleanup work that  constitutes projects is within the scope of the office’s responsibilities. Until  EM works together with DOE’s Office of Project management to (1)  establish requirements for classifying cleanup work as capital asset  projects or operations activities and (2) assess EM’s ongoing operations  activities to determine if they should be reclassified as capital asset  projects based on the newly established requirements, the department  may incur more project management risk of cost increases and schedule  delays than it should for hundreds of billions of dollars of remaining work. EM’s Cleanup Policy  Does Not Follow Most  Selected Program  and Project  Management Leading  Practices EM’s 2017 cleanup policy, which governs the EM program and its  operations activities, does not follow most selected leading practices for  program and project management. More specifically, EM’s 2017  cleanup policy does not follow any of 9 selected program management  leading practices related to scope, cost, schedule performance, and  independent reviews. Further, EM’s 2017 cleanup policy follows 3 of 12  selected project management leading practices related to these areas; it  does not follow the remaining 9. Figure 5 shows the percentage of  selected program and project management leading practices that DOE’s  Office of Environmental Management’s 2017 cleanup policy follows. EM’s Cleanup Policy Does  Not Follow Any of Nine  Selected Leading Program  Management Practices EM’s 2017 cleanup policy does not follow (i.e., does not meet, minimally  meets, or partially meets) the nine leading practices for program  management related to scope, cost, schedule performance, and  independent reviews that we selected based on PMI’s standards. More  specifically, the policy partially met two of the leading practices, minimally  met four others, and did not meet three, as discussed below: Having a program management plan and a roadmap that are  updated regularly. (Minimally meets.) EM’s policy does not require  an overarching program management plan or strategic plan that  encompasses the work at all sites. The policy does require that each  site maintain a life-cycle baseline based on the scope, cost, and  schedule of work, which are components of a program management  plan. However, the requirement is specific to each site and not the  entire EM program. Having a reliable, integrated, comprehensive life-cycle cost  estimate that is updated on a regular basis. (Partially meets.)  EM’s policy requires an integrated life-cycle cost estimate for the  entire EM program but does not state that the cost estimate must be  reliable or updated on a regular basis. Having a reliable, integrated master schedule that is updated on  a regular basis. (Does not meet.) EM’s policy does not require an  integrated master schedule at the program level. Measuring performance against both a program’s life-cycle cost  and integrated master schedule baselines. (Does not meet.)  EM’s policy does not require that EM track and monitor all high-level  program components against a program’s life-cycle cost and  integrated master schedule baselines for the entire EM program. Completing performance reporting and analysis in a way that  provides a clear picture of program performance. (Minimally  meets.) EM’s policy requires performance reporting to the EM  headquarters management level, but it does not require that  performance information be analyzed to give a clear picture of  program performance. Having a lessons learned database. (Partially meets.) EM’s policy  requires that EM collect and disseminate lessons learned, but the  policy does not specify a framework, such as a database, for how the  lessons learned should be collected and shared. Conducting program risk management throughout the life of the  program. (Does not meet.) EM’s policy does not require EM to  conduct risk management throughout the life of the program. Monitoring and controlling the program, including conducting  root cause analyses and developing corrective action plans.  (Minimally meets.) EM’s policy does not have any requirements  related to monitoring and controlling activities at a program level when  there is evidence that the program’s cost or schedule baseline will not  be met. It does require some monitoring and controlling activities at  the site level. Having an independent oversight body that conducts periodic  reviews of the progress of the program in delivering its expected  benefits. (Minimally meets.) EM’s policy does not require any  independent entity outside EM to review the performance of the EM  program as a whole in delivering its expected benefits. The policy  requires EM’s Office of Project Management to conduct a periodic  Programmatic Peer Review of cleanup work at each site, but this  review is not independent of EM. EM officials stated that even though EM’s policy does not follow these  program management leading practices, EM officials may take some  actions that address these leading practices. For example, to address the  leading practice of having a lessons learned database, EM officials  explained that EM’s Office of Project Management generates and  distributes across EM a monthly lessons-learned bulletin on a topic of its  choosing, and these lessons learned are uploaded on a site accessible to  everyone within EM. They also explained that officials across EM could  enter lessons learned in a DOE-wide lessons-learned database managed  by DOE’s Office of Environment, Health, Safety, and Security. In addition,  to address the leading practice of monitoring and controlling the program,  including conducting root cause analyses and developing corrective  action plans, the new Assistant Secretary for Environmental Management  requested the development of a root cause analysis and a corrective  action plan for the EM program in August 2018. To address the  Assistant Secretary’s request, EM officials stated that in November 2018  they identified nine improvement areas for the EM program, for which  they are developing corrective measures. However, when we reviewed the actions EM officials cited they took to  address the selected leading practices, we found that they fell short of  following leading practices. For example, the lessons learned listed in the  bulletins we reviewed were related only to capital asset projects, and the  database cited by EM officials is not used often by EM; it contains a total  of six entries on EM-related issues from 2005 to 2017. In addition, EM  officials stated they do not apply key practices that can be used to identify  and apply lessons learned. Further, EM officials in charge of developing  a root cause analysis and a corrective action plan stated that EM does  not have a process for doing so and that EM has not prepared such an  analysis or plan since 2011. They also stated that EM does not intend to  publish this document and that EM will not develop a root cause analysis  to show the problems these corrective measures are supposed to  address. The selected leading practices help ensure that a program achieves its  goals and intended benefits and that it optimizes scope, cost, and  schedule performance, and independent review of performance. Without  documenting such leading practices in policy, EM officials may not be  aware of expectations to carry them out and may not do so consistently.  Under federal standards for internal control, management should design  control activities to achieve objectives and respond to risks, such as by  clearly documenting internal control in management directives,  administrative policies, or operating manuals. Furthermore, these  standards state that management periodically reviews policies,  procedures, and related control activities for continued relevance and  effectiveness in achieving the entity’s objectives or addressing related  risks. Until EM reviews and revises its cleanup policy to include program  management leading practices related to scope, cost, schedule  performance, and independent review, the EM program is at risk of  continued uncontrolled changes to the program’s scope, exceeding its  cost estimate and schedule, failing to meet its programmatic goals, and  increasing DOE’s environmental liabilities. EM’s Cleanup Policy Does  Not Follow Most Selected  Project Management  Leading Practices EM’s 2017 cleanup policy, which applies to operations activities, follows  (i.e., substantially or fully meets) 3 and does not follow (i.e., does not  meet, minimally meets, or partially meets) 9 of the 12 leading practices for  project management related to scope, cost, schedule performance, and  independent reviews that we selected based on PMI’s standards.  Specifically, the policy follows these three selected leading practices: Establishing a performance baseline and tracking it from the  beginning to the end of the project. (Substantially meets.) EM’s  policy requires that a contractor must establish a cost baseline and  complete key performance measures within 110 percent of the  approved, current cost baseline. The policy also requires that  managers in charge of the work be responsible for successfully  executing work within the approved performance baseline. Conducting monitoring and controlling activities to measure  performance at regular intervals. (Fully meets.) EM’s policy  requires periodic project reviews from various levels, from the federal  cleanup director in charge of the operations activity and site manager,  all the way to EM senior leadership. Using an EVM system that is independently certified and  continuously monitored to assess project performance.  (Substantially meets.) EM’s policy requires the implementation at the  contract level of a work control system, either an EVM system or an  approved alternative. EM guidance suggests that the EVM system  be surveilled regularly, although EM does not require the EVM system  to be independently certified. The policy did not follow the other 9 selected project management leading  practices; specifically, it partially met 5, while the remaining 4 were  minimally or not met, as explained below: Establishing a project execution plan with policies and  procedures to manage and control project planning. (Does not  meet.) EM’s policy does not require a plan to establish policies and  procedures to manage and control project planning. Clearly and completely defining the scope of a project so that its  performance can be measured. (Partially meets.) EM’s policy  requires that the scope be defined for a segment—typically a 5- to 10- year contract—at the beginning of the work. However, EM’s policy  also states that the segment’s scope may be reduced to free up  funding to cover risks. When risks occur and the scope is reduced, the  segment’s performance may not be accurately and fully measured. Developing a cost estimate using GAO best practices. (Partially  meets.) EM’s policy requires that EM follow our best practices for cost  estimating prior to starting the execution of a segment. However, once  the contractor begins executing the segment, the policy does not  require EM to follow our best practices, even when independent cost  estimates are developed during a baseline change process. Developing and maintaining an integrated master schedule using  GAO best practices. (Minimally meets.) EM’s policy requires that  the contract specify the schedule for the segment, which could be an  input to an overall integrated master schedule for that segment. The  policy does not require that an integrated master schedule be  developed and maintained in accordance with GAO best practices. Conducting risk assessments throughout the life cycle of the  project; prioritizing risks in a risk register; developing risk  mitigation strategies; and determining the appropriate amount of  contingency. (Minimally meets.) EM’s policy does not require a risk  management plan for projects. In addition, the policy states that EM  will not fund contingency to cover risks that may occur for operations  activities. Capturing lessons learned throughout the continuum of a project  in a database and disseminating them among projects. (Partially  meets.) EM’s policy requires the EM Deputy Assistant Secretary for  Acquisition and Project Management to collect and disseminate  lessons learned, but the policy does not specify that this process  should be done throughout the continuum of a project or that lessons  learned should be disseminated among operations activities. Developing a root cause analysis and corrective action plan to  identify and address the underlying causes of cost overruns,  schedule delays, and performance shortcomings when a cost or  schedule overrun occurs. (Does not meet.) The policy does not  contain any information on the steps that EM will take, such as  developing a root cause analysis and corrective action plan, once  management becomes aware that a cost or schedule overrun is  probable for an operations activity. Conducting a variety of independent reviews throughout the life  of a project, including at key decision points, and on multiple  aspects of the project, such as the mission need, cost, earned- value management system, and baseline review. (Partially meets.)  EM’s policy requires reviews of segments conducted or organized by  EM’s Office of Project Management. However, there are no  requirements for any independent reviews conducted by DOE offices  or other entities outside EM. Establishing project-reporting systems/databases to provide a  clear picture of project performance to management and to keep  the contractor accountable. (Partially meets.) EM’s policy  established a requirement that performance information be reported in  the Integrated Planning, Accountability, and Budgeting System  database for each operations activity. However, EM’s policy does  not address how this performance information will provide a clear  picture of performance and how it will be used to keep the contractor  accountable. Our findings on the inclusion of project management leading practices in  EM’s 2017 cleanup policy are consistent with concerns raised by DOE’s  Project Management Risk Committee. According to meeting minutes from  December 2015, the committee expressed concerns that EM’s proposed  cleanup policy (adopted in July 2017) appeared to run counter to the  Secretary’s initiative to apply best practices to oversight of project  management. In committee meeting minutes from November 2015, the  committee expressed concern with the level of rigor that would be applied  to independent cost analysis, project reviews, general oversight, and risk  mitigation under the new cleanup policy. According to PMI, effective  project management is key to implementing an organization’s strategy,  and has a dramatic impact on the bottom line; organizations that invest in  proven project management practices—such as these selected leading  practices—continue to experience greater success than their  underperforming counterparts. In addition, under federal standards for internal control, management  periodically reviews policies, procedures, and related control activities for  continued relevance and effectiveness in achieving the entity’s objectives  or addressing related risks. Until EM reviews and revises its policy to  include project management leading practices related to scope, cost, and  schedule performance, and independent reviews, EM’s operations  activities are at risk of scope creep or uncontrolled changes to scope,  exceeding their initial budget and schedule, and failing to meet their  goals. EM’s Performance  Measures for  Operations Activities  Do Not Provide a  Clear Picture of  Overall Performance EM uses three tools to measure the overall performance of operations  activities, but these tools do not provide a clear picture of overall  performance. These tools are earned value management, performance  metrics, and milestones, according to EM documentation and officials.  However, EM has not followed best practices for its contractors’ EVM  systems; EM’s performance metrics do not link performance to cost; and  EM postpones milestones when they are at risk of missing them and does  not consistently track or report those milestone changes over time. Figure  6 summarizes our findings on these three performance measures and  how they affect EM’s ability to effectively manage the cleanup effort. EM Relies on Three Tools  to Measure Performance  of Its Operations Activities To measure the overall performance of its operations activities, EM relies  primarily on EVM data, supplemented by program-wide performance  metrics and cleanup milestones, according to EM documentation and  officials. EVM is a management tool used to measure the value of work  accomplished in a given period and compare it with the planned value of  work scheduled for the same period and with the actual cost of the work  accomplished. EVM data can alert project managers to potential  problems sooner than expenditures alone can. The use of EVM as a  management tool is considered a best practice for conducting cost and  schedule performance analysis for projects. EM’s 2017 cleanup policy requires that contractors use an EVM system  or an approved alternative for monitoring and controlling work at the  contract level. We reviewed all 20 EM contracts covering operations  activities and found that EM requires its contractors to maintain EVM  systems for 17 of all 20 contracts. EM paid contractors for maintaining  these systems and providing EVM reports to EM. For example, EM has  paid one contractor $1 million annually to maintain its EVM system, and  EM has paid contractors anywhere from $10,000 to $235,000 annually to  receive their EVM reports, according to EM responses to our information  request. EVM by itself may not be sufficient to measure the progress of operations  activities, according to EM’s 2012 operations activities protocol. The  second tool EM uses to measure performance is performance metrics.  EM developed 17 program-wide performance metrics for its cleanup  work. The goal of these metrics is to measure progress toward  completing the scope of work for the contract and the entire life of an  operations activity. EM headquarters collects information from the sites  monthly to measure how each activity has performed against a goal set at  the beginning of each year. Examples of EM’s performance metrics  include  the number of cleanup sites being eliminated,  the cubic meters of transuranic waste being disposed of,  the number of containers of high-level waste packaged for final  disposition, and  the number of closed radioactive liquid waste tanks. The EM cleanup sites set targets for these metrics annually. According to  EM officials, many operations activities have one or more of these  performance metrics associated with them, but some do not. Appendix II  contains the full list of EM’s performance metrics. The third tool EM uses to measure performance are cleanup milestones.  Cleanup milestones represent deadlines for various cleanup-related  activities derived from agreements DOE enters into with its regulators,  including the Environmental Protection Agency and states. There are  many different types of milestones, including enforceable and planning  milestones. Generally, an enforceable milestone has a fixed, mandatory  due date that is subject to the availability of appropriated funds while a  planning milestone is not enforceable and usually represents a  placeholder for shorter term work. EM collects program-wide performance information from the three  performance measures tools in a centralized database known as the  Integrated Planning, Accountability, and Budgeting System. These  performance data are used by EM to manage its program and to provide  information to DOE management, Congress, and other stakeholders.  According to DOE’s Office of Inspector General and EM officials, this  database was developed as a program management tool to provide  information to EM headquarters officials, to ensure effective overall program  performance; DOE’s Chief Financial Officer, for inclusion in DOE-wide reports; Congress and taxpayers, to identify the remaining environmental  cleanup liability and to provide transparency regarding contractor  performance; and  stakeholders, to make sure the work reported is accurate, timely,  complete, and in accordance with agreements. EM Has Not Ensured That  EVM Systems Are  Comprehensive, Provide  Reliable Data, or Are Used  by Leadership for  Decision-Making EM relies on contractors’ EVM systems to measure the performance of its  contractors’ operations activities, but EM has not followed (i.e., has not  met, has minimally met, or has partially met) best practices to ensure that  these systems are (1) comprehensive, (2) provide reliable data, and (3)  are used by EM leadership for decision-making—which are the three  characteristics of a reliable EVM system. Moreover, EM has allowed the  contractors to categorize a large portion of their work in a way that limits  the usefulness of the EVM data. EM Has Not Followed Best  Practices for Its Contractors’  EVM Systems Our analysis of EM contractors’ EVM systems for operations activities  found that EM has not followed (i.e., has not met, has minimally met, or  has partially met) best practices, as discussed below. As a result, EM has  not ensured that these systems are: (1) comprehensive, (2) provide  reliable data, and (3) used by EM leadership for decision-making—which  are the three characteristics of a reliable EVM system. (See app. III for  more specific information on EM’s performance on each best practice  considered and app. IV for information on how each contract followed  each best practice.) Comprehensive: Best practices to ensure EVM systems are  comprehensive are: (1) requiring the contractor’s EVM systems be  certified to meet guidelines established by the Earned Value  Management Systems EIA-748-D Intent Guide; (2) conducting an  integrated baseline review to ensure that all work is accurately  captured in the performance measurement baseline; and (3)  performing regular surveillance to ensure the contractors continue to  maintain their EVM systems in a way to meet the EIA-748-D  guidelines. We found that 17 out of 20 contractors’ EVM systems  were certified to be compliant with the EIA-748-D guidelines, but of  these 17, 4 contractors had self-certified their EVM systems.  However, only about half of the EVM systems met the best practices  for conducting integrated baseline reviews and performing ongoing  surveillance. Among those, many of the reviews were not rigorous  enough to ensure that the performance measurement baseline  captured all of the work. In November 2017, EM issued a standard  operating policy and procedure, which suggests that EVM systems be  surveilled regularly. However, we discovered that EM officials were  not performing thorough surveillance reviews to ensure that EVM  systems were in alignment with the EIA-748-D guidelines and that the  data being reported by the EVM systems were reliable. Provide reliable data: Best practices to ensure that the contractors’  EVM systems provide reliable data are (1) the EVM data do not  contain any anomalies and (2) estimates at completion— the  expected total cost of completing all work based on the contractor’s  performance to date—are realistic. The EVM data for contracts  covering operations activities contained numerous, unexplained  anomalies in all the months we reviewed, including missing or  negative values for some of the completed work to date. Negative  values should occur rarely, if ever, in EVM reporting because they  imply the undoing of previously scheduled or performed work. In  addition, we found problems with the estimate at completion listed in  all 20 contractors’ EVM systems. More specifically, we found (1) many  instances where the actual costs exceeded the estimates at  completion even though there was still a lot of work remaining; (2)  several occasions where the estimates at completion were less than  half of the original budget at the beginning of the project; and (3)  several contractors reported estimates at completion of zero dollars  when their original budgets were for hundreds of millions of dollars.  These problems indicated that the EVM systems were not being  updated in a timely manner or were not well monitored since the  estimate at completion values were too optimistic and highly unlikely. Used by EM leadership for decision-making. Best practices to  ensure that the data from the contractors’ EVM systems are used by  EM leadership for decision-making are: (1) reviewing EVM data,  including cost and schedule variances, on a regular basis; (2)  ensuring that EM management use EVM data to develop corrective  action plans; and (3) ensuring that the performance measurement  baseline is updated to reflect changes. We reviewed monthly  reports EM sites present to EM headquarters management for review.  We found that none of the sites adequately reported EVM variances  to EM headquarters management; they were all missing some EVM  information such as trend data or the estimate at completion. In  addition, many of the sites’ monthly reports did not include corrective  action plans for addressing variances, if any, between planned and  actual performance. We also reviewed monthly reports that the EM  Office of Project Management started to present to EM headquarters  senior leadership in October 2017, and found that these reports  included most of the EVM indicators for all 15 contracts on which EM  Office of Project Management reported. However, EM Office of  Project Management officials stated that they have only started  suggesting corrective action to EM headquarters senior leadership  since early 2018; it is too soon to tell how EM headquarters senior  leadership is using this information to determine which contracts need  the most attention and which corrective actions management will  develop and take. Moreover, this monthly report uses unreliable EVM  data, as we found in the prior characteristic. Finally, regarding the  third best practice, EM provided evidence that 17 out of 20 contractors  had a formal process in place for updating the budget baseline.  However, the extent to which contractors followed their processes  was questionable given the problems we found with the estimates at  completion, as discussed in the prior characteristic. Even though EM requires most of its contractors for operations activities  to maintain EVM systems and pays them for doing so, EM’s 2017 policy  generally does not require that EVM systems be maintained and used in  a way that follow EVM best practices. Until EM updates its cleanup  policy to require that EVM systems be maintained and used in a way that  follow EVM best practices, EM leadership may not have access to reliable  performance data to make informed decisions in managing its cleanup  work and to provide to Congress and other stakeholders on billions of  dollars’ worth of cleanup work every year. Much of the Cleanup Work Is  Categorized in a Way That  Limits the Usefulness of the  EVM Data Compounding the limitations with the EVM systems currently in place, EM  has categorized a large portion of its work in a way that limits the  usefulness of the EVM data. Specifically, a sizable amount of the work is  categorized as level of effort for all 14 contracts for which we could  identify the percentage of the level-of-effort work (in dollars). Work that  is categorized as level of effort does not have defined deliverables or  physical products. Progress for level-of-effort work is measured by the  passage of time, but is not measured against a scheduled amount, so no schedule variance occurs. The effectiveness of EVM systems, which  are designed to measure performance against cost and schedule targets,  will be limited if there is a high amount of level-of-effort work, according to  our best practices. Thus, according to best practices, categorizing work  as level of effort should be minimized to the extent possible if EVM is  being used to measure performance, and contracts with level-of-effort  work over 15 percent should be subject to additional scrutiny. As  shown in figure 7 below, the range for EM’s contracts on operations  activities is between 36 and 83 percent. (We used letters for each  contract, rather than identifying the site or contractor). According to EM officials, at least half of the level-of-effort work  conducted under the cleanup contracts consists of recurring activities  necessary to maintain the sites, which EM refers to as “minimum safety”  work. According to EM officials, examples of such work include physical  security, health and radiation protection and services, or critical facility  and infrastructure maintenance for safe conditions. These officials said  that minimum safety work makes up 30 to 60 percent of individual sites’  budgets, for a total of at least $2.7 billion, or 42 percent, of EM’s $6.4  billion fiscal year 2018 budget. The Assistant Secretary for EM noted in  September 2018 that much of DOE’s environmental cost liability has to do  with the management of the minimum safety work. The Assistant  Secretary also noted that significant potential cost savings could result  from reducing minimum safety work and planned to start an initiative in  fiscal year 2019 to examine how EM can reduce this work. EM officials agreed that some of the contractor’s work currently  categorized as level of effort could in fact be measured discretely.  According to an ANSI guideline, only work not measurable or for which  measurement is impractical may be categorized as level of effort. EM  officials we interviewed stated that EM relies on its contractors to  categorize work as discrete or as level of effort, and EM approves these  decisions during the integrated baseline review. According to EM officials,  there is no EM policy or guidance on what circumstances justify  categorizing work as level of effort. Federal standards for internal controls  state that management should design control activities to achieve  objectives and respond to risks, such as by clearly documenting internal  control in management directives, administrative policies, or operating  manuals. Until EM develops a policy that ensures that work is  categorized as level of effort only in appropriate, specified circumstances,  such as when work is not measurable or when measurement is  impractical, it may not have reliable performance data to help it achieve  its objective of reducing risks and costs associated with billions of dollars’  worth of cleanup work every year. Performance Metrics and  Milestones for EM  Cleanup Work Do Not  Provide a Clear Picture of  Performance We found that EM’s 17 performance metrics for its cleanup work measure  the scope of work accomplished in a specific year but do not link that  work to the cost of completing it. For example, EM reported in the  Integrated Planning, Accountability, and Budgeting System database  eliminating 72,000 gallons of radioactive liquid waste out of a target of  342,000 gallons for fiscal year 2017 at the Savannah River Site, and  disposing of 1,734 cubic meters of low-level waste out of a target of 360  cubic meters at the Idaho site. However, in neither case did EM indicate  how much that work cost to accomplish. According to officials from DOE’s  Office of Project Management, the scope of work accomplished is not a  good indicator of performance by itself because it does not allow the  project manager to know whether EM received good value from the  contractor. In contrast, EVM systems allow managers to measure the  value of work accomplished in a given period. As discussed above, EM  collects EVM data, but EM’s performance metrics do not link to the EVM  data. According to federal standards for internal control, management  should use quality information to achieve an entity’s objectives and the  quality information must be complete, among other things. In EM’s  case, its objective, as stated in its mission, includes completing its  cleanup work in a way that reduces associated risks and costs. By  integrating reliable EVM data into EM’s performance metrics for  operations activities, EM could provide a clearer picture of performance  and better indicate whether EM is achieving its objective of reducing risks  and costs. With regard to cleanup milestones, we found in February 2019 that EM  has hundreds of milestones, but the exact number cannot be determined  because of inconsistencies in tracking and defining milestones between  sites and EM headquarters, and sites have the discretion to send updated  milestone data to EM headquarters when they choose. As a result,  some sites track milestones differently than EM headquarters does. We  also found that EM does not consistently define or track met, missed, or  postponed cleanup-related milestones at selected sites, and EM’s  milestone reporting to Congress is inconsistent. EM sites renegotiate  milestone dates with their regulators before they are missed, and EM  does not track the history of these changes. This is because once  milestones are changed, sites are not required to maintain or track the  original milestone dates. As a result, the new milestones become the new agreed-upon time frame, essentially resetting the deadline. Further, in its  report to Congress on enforceable milestones’ status, EM reports the  most recently renegotiated milestone dates with no indication of whether  or how often those milestones have been missed or postponed. Thus, the  EM program is unable to use milestone data to provide a clear, reliable  picture of its performance. Furthermore, EM officials at headquarters and  selected sites said they had not conducted root cause analyses on  missed or postponed milestones. Thus, EM cannot address systemic  problems and consider them when renegotiating milestones with  regulators. In addition, without such analysis, EM and its cleanup  regulators lack information to set more realistic and achievable  milestones. As a result, future milestones are likely to continue to be  pushed back, further delaying the cleanup work and likely increasing  cleanup costs. In this same report, we recommended, among other  things, that EM should establish a standard definition of milestones  across the cleanup sites, track changes to the milestones, report annually  to Congress on the status of its milestones, and conduct root cause  analyses of performance shortcomings that lead to missed or postponed  milestones. Conclusions DOE’s EM program has the challenging mission of safely cleaning up  radioactive waste, spent nuclear fuel, and environmental contamination  from 50 years of federal nuclear weapons production and energy  research, while working to reduce associated risks and costs within the  established regulatory framework. Since its mission began in 1989, EM  has spent more than $164 billion on its cleanup work, and it faces future  cleanup costs of more than $377 billion—the federal government’s single  largest environmental liability. To improve management of projects undertaken within the department,  including EM, DOE established its Office of Project Management and  strengthened project management requirements in Order 413.3B for  managing capital asset projects. However, since 2009, when EM created  a new category of cleanup work called operations activities, EM has  opted not to apply DOE’s project management requirements to almost 80  percent of its cleanup work. From fiscal years 2011 to 2018, EM’s  environmental liability increased by about $214 billion. DOE’s Office of  Project Management officials have raised concerns about how EM  classifies this work. Until EM works together with DOE’s Office of Project  management (1) to establish requirements for classifying cleanup work as  capital asset projects or operations activities and (2) to assess EM’s  ongoing operations activities to determine if they should be reclassified as  capital asset projects based on the newly established requirements, the  department may incur more project management risk of cost increases  and schedule delays than it should for hundreds of billions of dollars of  remaining work. In July 2017, EM released a new cleanup policy containing requirements  for managing its program and its operations activities, but this policy does  not follow most of the selected program and project management leading  practices we identified related to management of scope, cost, and  schedule performance, and independent review of performance. Until EM  reviews and revises its cleanup policy to include program and project  management leading practices related to scope, cost, schedule  performance, and independent reviews, the EM program is at risk of  uncontrolled changes to scope, exceeding its cost estimates and  schedule, failing to meet its goals, and increasing DOE’s environmental  liabilities. The new Assistant Secretary for the Office of Environmental Management  has acknowledged the importance of improving EM’s performance in  addressing the department’s large and growing environmental liabilities.  However, the three tools that EM uses to measure its overall program  performance and contractors’ performance on operations activities— earned value management, performance metrics, and cleanup  milestones—do not provide a clear, reliable picture of performance for EM  leadership, Congress, and other stakeholders. In particular, EM’s EVM  systems for operations activities are not comprehensive, do not provide  reliable data, and are not used by EM leadership to measure overall  performance of the EM program. Furthermore, a large portion of the work  performed by contractors is categorized as level of effort, limiting the  usefulness of the EVM data. In addition, EM’s performance metrics are  not linked to the costs of the work performed. Until EM updates its  cleanup policy to require that EVM systems be maintained and used in a  way that follows EVM best practices, EM leadership may not have access  to reliable performance data to make informed decisions in managing its  cleanup work and to provide to Congress and other stakeholders on  billions of dollars’ worth of cleanup work every year. Moreover, until EM  develops a policy that ensures that work is categorized as level of effort  only in appropriate, specified circumstances, such as when work is not  measurable or when measurement is impractical, it may not have reliable  performance data to help it achieve its objective of reducing risks and  costs associated with billions of dollars’ worth of cleanup work every year.  Finally, by integrating reliable EVM data into EM’s performance metrics  for operations activities, EM could provide a clearer picture of  performance and better indicate whether EM is achieving its objective of  reducing risks and costs. Recommendations for  Executive Action We are making the following seven recommendations to DOE: The Secretary of Energy should direct the Director of the Office of  Project Management and the Assistant Secretary of the Office of  Environmental Management to work together to establish  requirements for classifying cleanup work as capital asset projects or  operations activities. (Recommendation 1) The Secretary of Energy should direct the Director of the Office of  Project Management and the Assistant Secretary of the Office of  Environmental Management to work together to asses EM’s ongoing  operations activities to determine if they should be reclassified as  capital asset projects based on the newly established requirements.  (Recommendation 2) The Secretary of Energy should direct the Assistant Secretary of the  Office of Environmental Management to review and revise EM’s 2017  cleanup policy to include program management leading practices  related to scope, cost, schedule performance, and independent  reviews. (Recommendation 3) The Secretary of Energy should direct the Assistant Secretary of the  Office of Environmental Management to review and revise EM’s 2017  cleanup policy to include project management leading practices  related to scope, cost, schedule performance, and independent  reviews. (Recommendation 4) The Secretary of Energy should direct the Assistant Secretary of the  Office of Environmental Management to update its cleanup policy to  require that EVM systems be maintained and used in a way that  follows EVM best practices. (Recommendation 5) The Secretary of Energy should direct the Assistant Secretary of the  Office of Environmental Management to develop a policy to ensure  that work is categorized as level of effort only in appropriate, specified  circumstances, such as when work is not measurable or when  measurement is impractical. (Recommendation 6) The Secretary of Energy should direct the Assistant Secretary of the  Office of Environmental Management to integrate EVM data into EM’s  performance metrics for operations activities. (Recommendation 7) Agency Comments  and Our Evaluation We provided DOE with a draft of this report for its review and comment. In  its written comments, reproduced in appendix V, DOE generally agreed  with the findings in the report and its recommendations and described  actions that it intends to take in response to our recommendations. More  specifically, of the seven recommendations, DOE concurred with four and  partially concurred with three. DOE partially concurred with our recommendations that the Director  of the Office of Project Management and the Assistant Secretary for  the Office of Environmental Management (EM) work together to (1)  establish requirements for classifying cleanup work as capital asset  projects or operations activities, and (2) assess EM’s ongoing  operations activities to determine if they should be reclassified as  capital asset projects based on the newly established requirements.  DOE stated that the department commits (1) to reviewing its  methodology for categorizing work and revising it, as appropriate, as  well as (2) to determining the appropriate application of any revisions  to the work classification methodology to new and existing work. DOE  also stated that the Assistant Secretary for EM is ultimately  responsible for the proper classification of work and will consult with  the Office of Project Management. We appreciate DOE’s commitment  to addressing these two recommendations. As we stated in our report,  in July 2015, the Secretary of Energy gave DOE’s Office of Project  Management responsibility to serve as DOE’s enterprise project  management organization. As such, DOE states that this office is  responsible for providing leadership and assistance in developing and  implementing DOE-wide policies, procedures, programs, and  management systems pertaining to project management, as well as  for independently monitoring, assessing, and reporting on project  execution performance. Officials from this office are experts in project  management, especially as it relates to capital asset projects. Given  (1) the high-risk posed by EM’s cleanup work and the high  environmental liability, which may continue to grow; (2) the difference  in the stringency of requirements between managing and overseeing  operations activities and capital asset projects; and (3) the concerns  raised by DOE top project management experts that some current  operations activities should be classified as capital asset projects, we  encourage the Secretary to direct EM not only to consult with DOE’s  Office of Project Management but to take advantage of the office’s  role and expertise and direct EM to work with this office to come to an  agreement about proper classification requirements and classification  of current and future cleanup work. It is in DOE’s interest to ensure its  cleanup work is classified and managed appropriately, regardless of  which office is ultimately responsible for the proper classification of  work. DOE concurred with our recommendations to review and revise EM’s  2017 cleanup policy to include program and project management  leading practices related to scope, cost, schedule performance, and  independent reviews and to require that EVM systems be maintained  and used in a way that follows EVM best practices. DOE also  concurred with our recommendation to develop a policy to ensure that  work is categorized as level of effort only in appropriate, specified  circumstances, such as when work is not measurable or when  measurement is impractical. DOE also partially concurred with our  recommendation to integrate EVM data into EM’s performance  metrics for operations activities. For all these recommendations, DOE  stated that EM is already in the process of reviewing the EM cleanup  policy for necessary updates, revisions, and modifications. DOE  further stated that EM will consider and incorporate changes relative  to these recommendations, as appropriate, during this process, and  EM will also consider any necessary changes to related guidance or  policies and procedures. DOE also provided technical comments, which we incorporated in our  report as appropriate. As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 14 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretary of Energy, and other interested  parties. In addition, the report will be available at no charge on the GAO  website at http://www.gao.gov. If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made significant contributions to this  report are listed in appendix VI. Appendix I: Objectives, Scope, and  Methodology Our report examined: (1) how the EM program manages its cleanup work,  (2) the extent to which EM’s cleanup policy follows selected program and  project management leading practices, and (3) how EM measures the  overall performance of its operations activities. To examine how the EM program manages its cleanup work, we  reviewed various DOE documents, including DOE’s Order 413.3B, EM’s  2012 operations activities protocol, EM’s 2017 cleanup policy, standard  operating policies and procedures associated with this cleanup policy,  EM’s mission and functions document, EM’s draft 45-day review  documentation, meeting minutes from the Project Management Risk  Committee, draft appendix to Order 413.3B developed by DOE’s Office of  Project Management, and documents received from cleanup sites. We  also interviewed DOE officials from the Office of Project Management and  members of the Project Management Risk Committee, and EM officials  from headquarters, such as the Associate Principal Deputy Assistant  Secretary for Field Operations, the Deputy Assistant Secretary for  Acquisition and Project Management, officials from EM’s Office of Project  Management, Office of Budget and Planning, Office of Program Planning,  officials in charge of managing the Integrated Planning, Accountability,  and Budgeting System database that collects monthly performance  information from the sites, and officials from 5 of EM’s 16 cleanup sites.  (We contacted all sites and interviewed 5 sites over the phone that  responded to our request for an interview.) We then decided to conduct  site visits. We visited two of these sites—Savannah River and Idaho— because they are among the sites with the highest number of operations  activities and the most diverse types of and highest-cost cleanup work  remaining. Our findings from these 5 sites are not generalizable to all EM  sites, but they help explain the delineation of roles between the site  managers and EM headquarters in managing and classifying cleanup  work. We also attended an EM internal training session in which EM  headquarters officials introduced the 2017 cleanup policy to officials at  the Hanford site and attended EM cleanup public conferences. Moreover,  we reviewed the role of DOE’s Office of Project Management in EM’s  cleanup work. More specifically, we examined whether this office played a  role in the development of EM’s 2017 cleanup policy and classification of  EM’s cleanup work, consistent with its designation as DOE’s enterprise  project management organization. To assess the reliability of EM’s fiscal  year 2019 budget data, we requested information about EM’s Financial  Integration System module of the Integrated Planning, Accountability, and  Budgeting System database, from which these data were provided.  Based on the responses from officials in charge of this database, we  determined the data to be sufficiently reliable for our purposes. To examine the extent to which EM’s cleanup policy follows selected  program and project management leading practices, we selected two sets  of criteria for program and project management leading practices using  leading practices from the Project Management Institute, which are  generally recognized as the top leading practices for program and project  management. To select program management leading practices, we first  reviewed the Project Management Institute’s The Standard for Program  Management—Third Edition (2013). We identified 9 program  management leading practices based on PMI’s standards related to a  program’s management of scope, cost, schedule performance, and  independent review of performance. To select project management  leading practices, we first identified 12 project management leading  practices listed in DOE’s Order 413.3B related to a project’s management  of scope, cost, schedule performance, and independent review of  performance. We then compared these 12 project management leading  practices to PMI’s A Guide to the Project Management Body of  Knowledge–Fifth Edition, which includes PMI’s standards for project  management, to make sure these leading practices align with PMI’s standards for project management. To select these leading practices, (1)  two GAO analysts separately examined the PMI and DOE  documentation, then, (2) a GAO specialist independent of the team  producing this report reviewed the leading practices we selected. All three  GAO staff agreed on these selected leading practices. To validate our  selection of program and project management leading practices, we  shared these selected leading practices with PMI representatives and  incorporated their feedback, as appropriate. PMI representatives agreed  with the program and project management leading practices that we  selected. We then compared EM’s 2017 cleanup policy and the 11 associated  standard operating policies and procedures developed by EM by the time  of our analysis (by May 2018) with the 9 program management and 12  project management leading practices we selected. We included these  standard operating policies and procedures in our analysis because EM  officials stated that EM intentionally wrote this policy at a high level  because EM planned to develop standard operating policies and  procedures that would establish more detailed steps to implement the  policy. We analyzed the extent to which the policy and the 11 standard  operating policies and procedures follow these leading practices. We also  interviewed EM headquarters and site officials to learn more about the  2017 cleanup policy. We used a 5-point scoring system to determine the  extent to which EM’s cleanup policy follows selected program and project  management leading practices. We used the following 5-point scoring  system: “fully met” means that complete evidence was provided that  satisfied the leading practice; “substantially met” means that evidence  was provided that satisfied a large portion of the leading practice;  “partially met” means that evidence was provided that satisfied about half  of the leading practice; “minimally met” means that evidence was  provided that satisfied a small portion of the leading practice; and “did not  meet” means that no evidence was provided that satisfied the leading practice. If the score for each leading practice was “fully met” or  “substantially met,” we concluded that EM’s cleanup policy and its  associated standard operating policies and procedures followed the  leading practice. In contrast, if the score was “partially met,” “minimally  met,” or “not met,” we concluded that EM’s policy did not follow the  leading practice. To determine this score, two GAO analysts separately  examined EM’s policy document and then agreed on a final score for  each of the leading practices. To examine how EM measures the performance of its operations  activities, we analyzed EM’s use of the three measures of performance  that EM policy identified: earned value management (EVM); performance  metrics; and cleanup milestones. To evaluate EM’s EVM systems, we  compared EM’s use of EVM with 8 of the 10 best practices for earned  value management found in our Cost Estimating and Assessment Guide,  which draws best practices from federal cost-estimating organizations  and industry. Specifically, we reviewed the use of EVM systems in the  21 contracts EM uses to execute its operations activities and compared  this review’s results with EVM best practices. To gather this information,  we submitted a data collection instrument to all 16 sites to ascertain  whether or not they follow these best practices for each contract  containing operations activities. We also requested documentation, such  as EVM system certification information or surveillance reports,  supporting their answers. We relied mainly on the sites’ responses but,  when available, also reviewed the documentation we received to check  the sites’ answers for accuracy and completeness. To determine whether information on EVM is reported to EM senior  leadership, we also reviewed (1) monthly progress reports EM sites  presented to EM headquarters management that ranged from April 2017  to April 2018 depending on the site and (2) monthly reports that EM Office  of Project Management presents to EM headquarters senior leadership;  specifically the April 2018 Cleanup Program Monthly Performance and  the EM Segment Activity Portfolio Summary, or “Quad Chart,” reports,  which were the most recent reports available at the time of this analysis. In addition, as part of our analysis, we analyzed EM headquarters’ EVM  data on operations activities from October 2016 through September 2017  (the most recent data available at the time of our review) to determine  whether or not the EVM data were reliable. We checked for data  anomalies, such as missing or negative values for each of those months.  We also reviewed DOE and EM documents—such as monthly progress  reports submitted by the 16 sites to EM headquarters for review or the  monthly reviews prepared by an EM headquarters office for senior  management—to see what EVM data senior management used for  decision-making. To provide a score for our analysis, we used the following 5-point scoring  system to score the answer for each contract for each best practice: “fully  met” means that complete evidence was provided that satisfied the best  practice; “substantially met” means that evidence was provided that  satisfied a large portion of the best practice; “partially met” means that  evidence was provided that satisfied about half of the best practice;  “minimally met” means that evidence was provided that satisfied a small  portion of the best practice; and, “did not meet” means that no evidence  was provided that satisfied the best practice. For each best practice, we  color-coded the assessment at the contract level. Contracts that fully met  or substantially met the criteria were coded green, those that partially met  the criteria were coded yellow, and those that did not or minimally meet  the criteria were coded red. We then assigned a score for each color: 1  for red, 3 for yellow, and 5 for green. We determined the overall score for  each best practice by taking the average across the 20 contracts we  reviewed. After scoring each best practice individually, we then used  these scores to develop an average score for the three EVM  characteristics: whether EM has ensured that these EVM systems are (1)  comprehensive, (2) provide reliable data, and (3) are used by EM  leadership for decision-making. To examine EM’s use of performance metrics data, we reviewed annual  performance metrics collected by EM headquarters for every operations  activity from 2010 to 2017. We chose this period because 2010 is the  time when EM started classifying work as operations activities while 2017  was the most recent available data at the time of our analysis. We  reviewed relevant documentation, and interviewed agency officials  knowledgeable about those data, among other things. Specifically, we  interviewed DOE and EM officials at headquarters and from the five  cleanup sites (including in-person interviews at the Savannah River and  Idaho sites). We also reviewed our prior work in GAO-19-207 related to  EM’s cleanup agreements and milestones. We conducted this performance audit from April 2017 to February 2019,  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Appendix II: EM’s Program-wide  Performance Metrics Presented to  Congress, as of the end of Fiscal Year 2017 Appendix II: EM’s Program-wide Performance  Metrics Presented to Congress, as of the end  of Fiscal Year 2017 The information in this table is from DOE’s fiscal year 2019 budget request, which was the most  recent request presented to Congress. DOE, Department of Energy: FY 2019 Congressional Budget  Request for Environmental Management, DOE/CF-0142, Vol. 5 (Washington, D.C.: March 2018). Appendix III: GAO Assessment of How Earned  Value Management Systems Used for EM’s  Operations Activities Met Best Practices GAO assessment of individual best practice   Substantially met. Seventeen out of 20 contracts we  reviewed had a certified EVM system, of which 4 self- certified. EM officials reported that the remaining three  contracts were not certified or were not required to be  certified. Partially met. Thirteen out of the 20 contracts we  reviewed had conducted or planned to conduct an  integrated baseline review to ensure that the  performance measurement baseline provides reliable  cost and schedule data for managing the program and  projecting accurate estimates at completion. However,  many of these reviews were not rigorous enough to  ensure that the performance measurement baseline  captured all of the work.  Not assessed. Partially met. Eleven out of the 20 contracts fully met  this best practice, and contractors performed self- assessments or conducted annual reviews for 5  additional contracts. However, EM field and  headquarters officials were not performing thorough  reviews to check whether the EVM systems were in  alignment with the EIA-748-D guidelines to ensure that  the data being reported by the systems were reliable. Partially met. The EVM data for operations activities  contracts contained numerous, unexplained anomalies  in all the months we reviewed—including missing or  negative values for some of the completed work to date.  Having anomalies in the EVM data occurring each  month can cause potential distortions resulting in  inaccurate projections of estimates at completion.   Not assessed. Characteristic /  Score Does EM’s use of   EVM systems follow  characteristic? GAO assessment of individual best practice   Minimally met. We found problems with the estimate at  completion in all of the 20 contracts we analyzed. For  example, we found instances where the estimates at  completion were either (1) less than half the original  budget, (2) higher than expected, or 3) zero when the  original budget was for hundreds of millions of dollars.  These problems indicated that the EVM systems were  not being updated in a timely manner or were not well  monitored since the estimate at completion values were  too optimistic and highly unlikely. Partially met. We reviewed two sources of information  on earned value management reporting to EM senior  leadership for this best practice. 1) When reviewing the  monthly reports EM sites present to EM headquarters  management, we found that none of the sites  adequately reported EVM data. 2) When reviewing the  new monthly report format that EM’s Office of Project  Management presents to EM headquarters senior  leadership since October 2017, we found that EM  reported on the performance of 15 out of the 20  contracts. We found that these reports included most of  EVM indicators for all 15 contracts on which EM Office  of Project Management reported. However, this monthly  report uses unreliable EVM data, as we found in the  prior characteristic. Partially met. We reviewed two sources of information  on earned value management reporting to EM senior  leadership for this best practice. 1) When reviewing the  monthly reports EM sites present to EM headquarters  management, we found that they contained corrective  action plans for only 3 contracts. 2) When reviewing the  new monthly reports that EM’s Office of Project  Management present to EM headquarters senior  leadership since October 2017, EM Office of Project  Management officials stated that they have only started  suggesting corrective action to EM headquarters senior  leadership since early 2018; it is too soon to tell how  EM headquarters senior leadership is using this  information to determine which contracts need the most  attention and which corrective actions management will  develop and take. Substantially met. EM provided evidence that 17 out of  20 contractors had a formal process in place for  updating the budget baseline. However, the extent to  which contractors followed their processes was  questionable given the problems we found with the  estimates at completion, as discussed in the prior  characteristic above. Appendix IV: EM’s Earned Value  Management Systems Used by Contracts  Containing Operations Activities B Not reviewed  Assessment for each best practice: Not met—provided no evidence that satisfies any of the best  practice; Minimally met—provided evidence that satisfies a small portion of the best practice; Partially  met—provided evidence that satisfies about half of the best practice; Substantially met—provided  evidence that satisfies a large portion of the best practice; and Met – provided complete evidence that  satisfies the entire best practice. We determined the overall score for each best practice by taking the  average across the 20 contracts we reviewed. We did not evaluate the following two best practices: (1) the schedule reflects the work breakdown  structure, the logical sequencing of activities, and the necessary resources and (2) EVM data are  consistent among various reporting formats. We excluded these two best practices because we  examined the use of EVM by contractors at a higher program level and did not conduct in-depth  analysis of each contractor’s EVM system. EM uses 21 contracts for its operations activities. We reviewed the use of EVM systems in 20 of  these contracts because one contract (contract K) is a fixed price contract, which does not require the  use of EVM. Appendix V: Comments from the Department  of Energy Appendix VI: GAO Contact and Staff  Acknowledgments GAO Contact Staff  Acknowledgments In addition to the contact named above, Nico Sloss (Assistant Director),  Cristian Ion (Analyst in Charge), Nathan Anderson, Margot Bolon, Jenny  Chow, Jennifer Echard, Juan Garay, Cindy Gilbert, Katherine Nicole  Laubacher, Cynthia Norris, Karen Richey, Dan C. Royer, Kiki  Theodoropoulos, and David Wishard made key contributions to this  report.
EM's mission is to complete the cleanup of nuclear waste at 16 DOE sites and to work to reduce risks and costs within its established regulatory framework. In December 2018, DOE reported that it faced an estimated $494 billion in future environmental cleanup costs—a liability that roughly tripled during the previous 20 years. GAO was asked to examine EM's operations activities. This report examines, among other objectives, (1) how EM manages its cleanup work and (2) the extent to which EM's cleanup policy follows selected leading practices for program and project management. To do this work, GAO reviewed agency documents and interviewed DOE project management experts and EM officials. GAO compared EM's policy with selected leading practices endorsed by the Project Management Institute for program and project management related to scope, cost, schedule, and independent review. The Department of Energy's (DOE) Office of Environmental Management (EM) manages most of its cleanup of nuclear waste (77 percent of its fiscal year 2019 budget) under a category that EM refers to as operations activities, using less stringent requirements than a category of work, known as capital asset projects. (See figure) Capital asset projects—which involve the acquisition of land and other assets, including through environmental remediation—must undergo a series of reviews by independent experts and DOE's senior leadership. In contrast, operations activities are not reviewed outside of EM. EM's policy defines operations activities as reoccurring facility or environmental operations, as well as activities that are project-like, with defined start and end dates. EM cleanup site managers have discretion on how to classify cleanup work because DOE and EM have not established classification requirements. Since 2015, experts in DOE's Office of Project Management have raised concerns that some operations activities should be classified as capital asset projects, and that managing them under less stringent requirements poses cost and schedule risks. For example, the experts stated the cleanup of tanks of radioactive liquid waste should be designated as capital asset projects. However, these experts also stated that EM did not respond to their concerns, even though the office has department-wide responsibilities for overseeing project management. Until EM works with DOE's Office of Project Management to establish requirements for classifying cleanup work, the department may incur more cost and schedule risks than it should. EM's cleanup policy does not follow any of 9 selected program management leading practices or 9 of 12 selected project management leading practices. For example, EM's 2017 cleanup policy does not follow the program management leading practice of conducting risk management throughout the life of a program or the project management leading practice of requiring independent reviews of operations activities. These leading practices help ensure that a program optimizes scope, cost, and schedule performance and that it achieves its goals and intended benefits. Until EM revises its cleanup policy to follow leading practices, EM's operations activities are at risk of uncontrolled changes to scope, exceeding initial budget and schedule, and failing to meet their original goals.