Background Overview of Personnel  Security Clearance  Process ODNI estimates that, as of October 1, 2015, approximately 4.2 million  government and contractor employees were eligible to hold a security  clearance. Personnel security clearances are required for access to  certain national security information. National security information may be  classified at one of three levels: confidential, secret, or top secret. The  level of classification denotes the degree of protection required for  information and the amount of damage that unauthorized disclosure could  reasonably be expected to cause to national security. Specifically,  unauthorized disclosure could reasonably be expected to cause (1)  “damage,” in the case of confidential information; (2) “serious damage,” in  the case of secret information; and (3) “exceptionally grave damage,” in  the case of top secret information. According to the Office of Personnel  Management (OPM) Federal Investigations Notice 16-02, tier 3  investigations are required for eligibility for access to secret and  confidential information, or for noncritical sensitive positions, or “L”  access. OPM Federal Investigations Notice 16-07 indicates that tier 5  investigations are required for eligibility for access to top secret or  sensitive compartmented information, or for critical sensitive or special  sensitive positions, or “Q” access. Once an executive branch agency determines that a position requires a  certain level of access to classified information, the employee in that  position completes a questionnaire for national security positions, which  the requesting agency sends to an investigative service provider. NBIB— the bureau within OPM with responsibility for conducting personnel  background investigations—conducts background investigations for most  of the federal government; however, some agencies have authority  delegated to them to conduct their own investigations. The investigative  service provider then conducts a background investigation and submits  an investigative report to the requesting agency. Adjudicators from the  requesting agency use the information from the investigative report to  determine whether to grant or deny the employee eligibility for a security  clearance by considering guidelines in 13 specific areas that address (1)  conduct that could raise security concerns and (2) factors that could allay  those security concerns and permit granting a clearance. Individuals  granted security clearances are investigated periodically—for as long as  they remain in a position requiring access to classified information—to  ensure their continued eligibility. The 2012 Federal Investigative  Standards changed the frequency of periodic reinvestigations for certain  clearance holders. Continuous Evaluation Is  Intended to Supplement  the Personnel Security  Clearance Process According to Executive Order 13467, as amended, continuous evaluation  is a vetting process to review the background of an individual who has  been determined to be eligible for access to classified information or to  hold a sensitive position at any time during the period of eligibility.  Continuous evaluation is intended to fill the gap that exists between  periodic reinvestigations in which issues relevant to an individual’s  continued eligibility for a security clearance may go unreported or  unknown. For example, while the Federal Investigative Standards have  allowed for periodic reinvestigations to be conducted at any time following  the completion of the previous investigation or reinvestigation, agencies  have not been required to conduct them more frequently than every 5  years, at most, depending on the clearance level and investigative  standards in effect. Like periodic reinvestigations, the purpose of  continuous evaluation is to assist agencies in evaluating an individual’s  continued eligibility for access to classified information. Continuous evaluation involves automated record checks conducted on a  more frequent basis, whereas periodic reinvestigations are conducted  less frequently and may include, among other things, subject and  reference interviews. The types of records checked as part of continuous  evaluation are the same as those checked for other personnel security  purposes. Security-relevant information discovered in the course of  continuous evaluation is to be investigated and adjudicated under the  existing standards. According to ODNI, implementation of continuous  evaluation will not alter clearance holders’ existing rights or  responsibilities and it will incorporate protections for privacy and civil  liberties. Continuous Evaluation Is a  Key Initiative of the  Personnel Security  Clearance Reform Effort The enactment of the Intelligence Reform and Terrorism Prevention Act  of 2004 initiated a reform effort including goals and requirements for  improving the personnel security clearance process government-wide.  In June 2008, Executive Order 13467 established the PAC as the  government-wide governance structure responsible for driving the  implementation of and overseeing security and suitability reform efforts.  The PAC presently has four principal members: the Deputy Director for  Management of OMB; the Director of National Intelligence, who is the  Security Executive Agent; the Director of OPM, who is the Suitability  Executive Agent; and the Under Secretary of Defense for Intelligence.  The Executive Order also designated the Deputy Director for  Management of OMB as the chair of the PAC. Among other things, the  PAC is to work with agencies to implement continuous performance  improvement programs, policies, and procedures; establish annual goals  and progress metrics; and prepare annual reports on results. It is also to  develop and continuously reevaluate and revise outcome-based metrics  that measure the quality, efficiency, and effectiveness of the vetting  enterprise. In April 2014, the PAC established the Program  Management Office to implement security clearance reforms. This office  includes subject-matter experts with knowledge of personnel security  clearances and suitability determinations from OMB, ODNI, OPM, DOD,  the Department of Homeland Security, the Department of Justice, the  Department of the Treasury, and the Federal Bureau of Investigation. In March 2014, OMB established Insider Threat and Security Clearance  Reform as a government-wide, cross-agency priority goal to improve  interagency coordination and implementation within the area of personnel  security clearances. Through this goal, the PAC and executive-branch  agencies are to work to improve oversight to ensure that investigations  and adjudications meet government-wide quality standards. Included  among the goal’s key milestones are implementing a continuous  evaluation policy for the executive branch that regularly assesses trusted  insiders who have been granted, or are eligible for, access to classified  national security information, and overseeing the establishment of  continuous evaluation capabilities. ODNI is identified as the lead agency  for achieving both of these milestones. In addition, continuous evaluation  is identified as a key initiative in the PAC’s strategic framework for fiscal  years 2017 through 2021 as part of an effort to modernize the vetting  process. While the PAC is responsible for driving the implementation of and  overseeing the overall government-wide reform effort, individual agencies  are responsible for various aspects of the effort. For example, as the  Security Executive Agent, ODNI is responsible for developing and issuing  uniform and consistent policies and procedures to ensure the effective,  efficient, timely, and secure completion of investigations, polygraphs, and  adjudications relating to determinations of eligibility for access to  classified information or eligibility to hold a sensitive position. In  addition, Executive Order 12968, as amended, indicates that ODNI is  responsible for setting the standards for continuous evaluation of those  individuals who have access to classified information. According to  ODNI, under these Executive Orders, it has responsibility for and  oversight of continuous evaluation, as it is an investigative activity that  supports eligibility determinations. As such, ODNI established a program  office within the National Counterintelligence and Security Center to,  among other things, establish policy, guidance, and standards for the  implementation of continuous evaluation across the executive branch. DOD Has Conducted  Research on Continuous  Evaluation for More Than  a Decade DOD has been piloting aspects of continuous evaluation for more than a  decade—with pilot tests of automated record checks conducted as early  as 2002. Specifically, PERSEREC has conducted several studies dating  back to 2001 that have informed and evaluated DOD’s continuous  evaluation pilots, including the utility of and costs associated with various  data sources. These studies have focused on the technical capability to  conduct automated record checks from over 40 government and  commercial databases, the value and utility of automated record checks  in tier 5 investigations, and investigative alternatives to the traditional  periodic reinvestigation, among other things. The studies have also  included recommendations to further improve DOD’s continuous  evaluation program, as well as areas for future research. PERSEREC  noted that it undertook these studies to identify ways to make the  personnel security system more efficient, fair, and effective. According to  PERSEREC, starting in 2004 with the formation of the government-wide  security clearance reform effort, it began to plan for a broader application  of its research beyond the department. Using this body of knowledge, DOD has incrementally improved its  automated record check capabilities and therefore its ability to implement  a continuous evaluation program, which it did in 2014 at the  recommendation of the Secretary of Defense. Specifically, following the  September 2013 shooting at the Washington Navy Yard, the Secretary of  Defense directed concurrent internal and independent reviews to identify  and recommend actions to address any gaps or deficiencies in DOD  programs, policies, and procedures regarding, among other things, the  granting and renewing of security clearances for department and  contractor personnel. In March 2014, the Secretary of Defense identified  four key recommendations based on the findings and recommendations  from those reviews. One of those recommendations was to implement  continuous evaluation to provide automated record checks of personnel  with access to DOD facilities or classified information. In addition, DOD  Instruction 5200.02, which was also issued in March 2014, states that all  personnel in national security positions shall be subject to continuous  evaluation. Consistent with the recommendation and the DOD  Instruction, the department implemented a continuous evaluation pilot in  October 2014, the details of which are discussed later in the report. ODNI Has Taken an  Initial Step to  Implement  Continuous  Evaluation across the  Executive Branch but  Has Not Determined  Key Program Aspects  or How it Will Monitor  and Measure  Performance In October 2016, ODNI took an initial step to implement continuous  evaluation across the executive branch in a phased approach, but as of  May 2017, it had not yet formalized the program in policy. The seven  agencies we spoke with have been limited in their abilities to plan for the  implementation of continuous evaluation, including developing estimated  costs, in accordance with ODNI’s phased approach. This is due, in part,  to the fact that ODNI has not yet determined key aspects of the program,  such as when the future phases of implementation will occur or what they  will entail, and none of the agencies has completed implementation plans.  Further, ODNI lacks plans for monitoring and measuring the performance  of continuous evaluation across the executive branch. ODNI Has Taken an Initial  Step to Implement  Continuous Evaluation  across Executive Branch  Agencies, but Has Not Yet  Formalized the Program in  Policy ODNI has taken an initial step to implement continuous evaluation across  all executive branch agencies in a phased approach, but it has not yet  formalized the program in policy. Specifically, in October 2016, ODNI  initiated the first phase of continuous evaluation and outlined  requirements for this phase in interim guidance distributed to  implementing agencies in December 2016. For the first phase of  implementation, executive branch agencies are to conduct certain  continuous evaluation record checks against a portion of their national  security population. Specific details of the requirements were omitted  from this report because the information is sensitive. According to OPM  Federal Investigations Notice 17-03, the first phase of continuous  evaluation is to be implemented by the end of fiscal year 2017. These  checks are conducted in addition to any initial investigations or periodic  reinvestigations occurring in fiscal year 2017. ODNI provided agencies  with prioritization guidance to help them select individuals for continuous  evaluation. Nearly 80 executive branch agencies are subject to the  requirements for this first phase of implementation. ODNI has taken steps to establish the executive branch-wide continuous  evaluation program in coordination with key stakeholders. For example, in  June 2013, ODNI established a Continuous Evaluation Working Group— consisting of 12 core voting member agencies—as a mechanism to  effectively coordinate continuous evaluation implementation among  executive branch departments and agencies. According to the group’s  charter, it meets on at least a quarterly basis and is responsible for  coordinating the development of continuous evaluation standards,  policies, and procedures, among other things. Since January 2015, ODNI  has also issued interim guidance to executive branch agencies that are  subject to its continuous evaluation requirements informing them about  the purpose of continuous evaluation and providing them with some  details of the program. Further, to inform the establishment of the  executive branch-wide program, ODNI itself began a 1-year continuous  evaluation pilot in September 2016, according to ODNI officials. Specific  details of ODNI’s pilot were omitted from this report because the  information is sensitive. In addition to developing standards for continuous evaluation and its  oversight role, ODNI is also developing a system that agencies can use to  conduct continuous evaluation. According to ODNI, its system is under  development and will be available to all executive branch agencies with a  full suite of continuous evaluation data sources. Agencies may opt to: (1)  use ODNI’s system, (2) develop their own technical solution, (3) partner  with another agency to fulfill their continuous evaluation requirements, or  (4) some combination of the above options. ODNI asked agencies in  December 2016 to provide a preliminary determination as to how they will  satisfy future automated records checks requirements to allow ODNI’s  continuous evaluation program to adequately plan for system enrollee  volume and data usage. Specific details regarding the response of  executive branch agencies to this request were omitted from this report  because the information is sensitive. Some executive branch agencies  stated the following: Department of Justice and State officials stated that they plan to use  ODNI’s system once its development is complete; DOD officials stated that they plan to use their own internal system  that they are developing to conduct continuous evaluation, but that  they may use ODNI’s system to conduct certain checks; and Department of Homeland Security officials noted that they plan to use  a combination of existing internal agency systems and ODNI’s  system. Standards for Internal Control in the Federal Government states that  management should externally communicate the necessary information to  achieve an entity’s objectives. Effective information and communication  are vital for enabling an entity to achieve its objectives, which can be  accomplished through written guidance. While ODNI has provided some details of the program to implementing  executive branch agencies through interim guidance, it has not yet  formalized the continuous evaluation program through a Security  Executive Agent Directive. Specifically, in May 2017, ODNI officials stated  that ODNI had not yet issued a Security Executive Agent Directive for  continuous evaluation, but that a draft directive was undergoing  interagency coordination. ODNI officials stated that the directive will  contain a definition of continuous evaluation that is consistent with, but  expands upon, the definition contained in the relevant Executive Order.  These officials stated that the expanded definition will help to clarify  continuous evaluation and ensure that agencies have a common  understanding of the program. In addition, ODNI officials stated that they  have developed draft implementation guidelines, which they plan to issue  after the directive is finalized. ODNI officials stated that the interim  guidance will remain in effect until the Security Executive Agent Directive  or follow-on interim guidance is issued. DOD’s continuous evaluation program—which it began in October 2014,  in advance of implementation of continuous evaluation executive branch- wide by ODNI—identified, in a requirements document for its continuous  evaluation IT system, that the most important gap in the development of  the department’s program was the lack of a national or DOD-level policy.  Specifically, the requirements document notes the lack of a policy that  fully describes the continuous evaluation process or purpose, or the end  uses of data. The requirements document further notes that there are  multiple definitions of continuous evaluation and, due to the lack of policy, there is not a common lexicon of terms used in the continuous evaluation  program, thereby creating an additional gap. While ODNI reports that the policy is under review, it has not prioritized  the implementation of continuous evaluation and, as a result, has missed  numerous milestones in issuing the policy since 2014. Specifically, the  original Insider Threat and Security Clearance Reform cross-agency  priority goal milestone for ODNI to issue a continuous evaluation policy  was July 2014. This milestone was not attained, and it was adjusted to  September 2016, a milestone that was also missed. The current  milestone for issuing the policy is October 2017. Additionally, ODNI has  missed other milestones for implementing a continuous evaluation  program, as discussed later in the report. Furthermore, ODNI has initiated the first phase of continuous evaluation  without a government-wide issued policy or an expanded definition of  continuous evaluation. As a result, agencies may develop inconsistent  approaches to implementing continuous evaluation. For example, DOD  officials stated that DOD has developed its own path for continuous  evaluation from ODNI’s limited guidance and that in the absence of a  government-wide policy, DOD is developing its own internal guidance. As  a result, the approach to continuous evaluation taken by DOD—the  executive branch agency with the majority of security clearance holders— may differ from that of other executive branch agencies once fully  implemented. Ultimately, such inconsistent approaches to continuous  evaluation could affect reciprocity among agencies—another key  objective of government-wide security clearance reform efforts. Without  issuing a Security Executive Agent Directive in advance of the next phase  of implementation—the timeframe for which ODNI has not yet  determined—that includes, among other things, an expanded definition of  continuous evaluation, agencies may develop inconsistent approaches to  continuous evaluation, resulting in an uneven and perhaps ineffective  implementation across the federal government. ODNI Has Not Yet  Determined Key Aspects  of the Continuous  Evaluation Program, and  Executive Branch  Agencies Have Been  Limited in Their Ability to  Plan for Implementation ODNI has not yet determined key aspects of its continuous evaluation  program, which has limited the ability of executive branch agencies to  plan for implementation in accordance with ODNI’s phased approach. For  example, while ODNI has initiated the first phase of continuous evaluation  in coordination with implementing executive branch agencies, it has not  yet determined what the future phases of implementation will entail, or  when they will occur. Specifically, ODNI officials stated that they have not  set any further timeframes for implementing continuous evaluation or  determined agency requirements for future phases. Moreover, the timeframes for the implementation of continuous evaluation  across the executive branch have been extended over time. For example,  the original milestone set by the government-wide reform effort for  implementing continuous evaluation was the 4th quarter of fiscal year  2010, and it was not attained. The PAC subsequently set an Insider  Threat and Security Clearance Reform cross-agency priority goal  milestone for developing an initial continuous evaluation capability for the  most sensitive top secret clearance holders by September 2014—which  was extended to December 2014—and a milestone for implementing the  capability for additional clearance holders by December 2016. These  milestones were also missed. As of May 2017, continuous evaluation had  not yet been fully implemented, and ODNI had not set a milestone for  when it would occur. Although ODNI is one of the goal leaders for the Insider Threat and  Security Clearance Reform cross-agency priority goal, a senior ODNI  official stated that the milestones were arbitrarily set, and that  implementing continuous evaluation has proven to be challenging as a  result of several technical and legal issues that need to be resolved.  Further, ODNI officials highlighted the complexities associated with  developing a whole-of-government continuous evaluation program and  noted that a number of challenges have come to light as they have been  developing the program, which have contributed to missed milestones.  However, ODNI has not prioritized the setting of internal milestones for  the future phases of implementation that it considers to be reasonable.  ODNI officials stated that because continuous evaluation is a new  initiative, no realistic timeline for full implementation will be set until the  initial results of implementation are analyzed and technical capabilities  have matured. Further, they stated that although they are unable to  develop a timeline for full implementation at this time, they are actively  working to implement the program. In addition, as previously discussed,  ODNI’s milestone for issuing a continuous evaluation policy has also  been adjusted over time. Figure 1 shows the adjusted executive branch  milestones for issuing a continuous evaluation policy and implementing a  continuous evaluation program, including developing a technical  capability. The uncertainty regarding the requirements and timeframes for the future  phases of the program has affected the ability of executive branch  agencies to plan to implement continuous evaluation and estimate the  associated costs. First, although OPM Federal Investigations Notice 17- 03 notes that the first phase of continuous evaluation is to be  implemented by the end of fiscal year 2017, none of the seven executive  branch agencies we spoke with has completed an agency-specific  implementation plan. While some agencies, such as DOD and State— both of which have established continuous evaluation programs in  advance of implementation across the executive branch—have  developed concepts of operations or standard operating procedures for  continuous evaluation, all seven agencies we spoke with stated that they  are waiting for additional information from ODNI before completing their  implementation plans. Department of Homeland Security officials stated  that they are waiting for ODNI to define and schedule the future phases of  implementation and to finish developing its continuous evaluation IT  system, because there could be unknown policy implications that would  affect the Department’s planning efforts. In August 2017, ODNI officials  described plans to distribute information to executive branch agencies  regarding continuous evaluation requirements for fiscal year 2018.  Specific details of these plans were omitted from this report because the  information is sensitive. Second, six of the seven agencies we spoke with noted challenges  associated with estimating the costs of implementation. For example,  while the Federal Bureau of Investigation has developed some cost  estimates for implementing continuous evaluation, officials noted that it is  challenging to estimate the full costs of the program until they receive  additional information from ODNI, such as the requirements for future  phases of implementation, as well as information about record check,  technology, and personnel requirements. DOD officials stated that the  number of individuals enrolled in continuous evaluation directly relates to  the amount of agency resources required, for example, to validate,  respond to, and adjudicate alerts. Two agencies we spoke with stated  that they had not yet taken any steps to estimate costs because they are  waiting for additional information from ODNI. In August 2017, ODNI  officials stated that they plan to leverage an upcoming OMB budget data  request, administered through the PAC, to obtain agency funding  estimates for expenses related to conducting continuous evaluation from  fiscal years 2017 through 2019. We have previously identified weaknesses associated with estimating the  costs of personnel security clearance reform. Specifically, in April 2015  we found, among other things, that long-term costs of implementing the  2012 Federal Investigative Standards—including the implementation of  continuous evaluation—were not addressed in personnel security  clearance background investigation reform planning documentation.  Further, we found that OMB did not have current and detailed cost- estimate information from executive-branch agencies, because it did not  begin to solicit the information from the agencies until almost 2 years after  the updated standards were approved. As such, we recommended in  April 2015, among other things, that the Deputy Director for Management  of OMB, in the capacity as Chair of the PAC, develop long-term funding  estimates for changes to the federal government’s investigation practices  resulting from the implementation of the standards, including but not  limited to costs related to: (1) information technology adjustments to  enable government-wide data sharing; (2) implementation of continuous  evaluation of clearance holders; and (3) additional personnel resources  for twice-as-frequent reinvestigations. OMB concurred with the  recommendation. However, as of October 2017, this recommendation  remained open. We continue to believe that this recommendation is  valid and should be implemented. In addition, the seven executive branch agencies we spoke with identified  other areas related to agency expectations for which they need  information from ODNI. For example, officials from the Department of  Justice; the Bureau of Alcohol, Tobacco, Firearms, and Explosives; and  the Federal Bureau of Investigation stated that while they would like to  use ODNI’s IT system to conduct all or at least some of the record checks  that will be required, they will need to develop an interface with ODNI’s  system to do so. However, these officials stated that they were unaware  of ODNI’s technical requirements for that interface. These officials further  stated that without information related to the technical requirements, they  are unable to sufficiently plan or budget for continuous evaluation. ODNI  officials stated that although ODNI’s IT system remains under  development, information on technical interface requirements is available  to all stakeholders and that they meet with agencies to discuss agency- specific IT requirements. According to ODNI, several executive branch  agencies have expressed an interest in using ODNI’s IT system to  conduct at least some, if not all, of the checks that will be required once  continuous evaluation is fully implemented. ODNI officials acknowledged  that agencies will need to develop an interface to use the system, and  that agencies will be responsible for the associated costs. The Project Management Institute’s Guide to the Project Management  Body of Knowledge (PMBOK® Guide) provides guidelines for managing  individual projects, including developing a project management plan—in  advance of executing the project—that describes how the project will be  executed, monitored, and controlled. The plan should include, among  other things, project schedules and stakeholder roles and responsibilities.  The guide notes that updates may be made to the project management  plan as changes may occur as the project progresses. ODNI officials managing the continuous evaluation program stated that  they have not developed a project management plan for the  implementation of continuous evaluation, to include an implementation  schedule, because they are still in the planning stage. However, ODNI  has already started to implement the program. Without a plan that, among  other things, identifies reasonable milestones for the future phases of  implementation, ODNI does not have a schedule against which it can  track its progress or to which it is accountable. Further, without a plan for  implementing continuous evaluation executive branch-wide that includes  a schedule and agency requirements for future implementation phases,  full implementation—which has been delayed for almost 7 years—may be  further delayed. While a phased approach to implementation provides agencies time to  adapt their personnel security clearance programs to changing  requirements, without an implementation plan outlining ODNI’s  expectations of agencies’ roles and responsibilities, agencies are unable  to sufficiently plan for the implementation of continuous evaluation,  including identifying required resources and estimating potential costs.  Further, without clearly defining expectations for agencies—including  information such as the planned requirements for future phases of  implementation—continuous evaluation may not be fully implemented  across the executive branch. Incomplete implementation could potentially  prevent the federal government from identifying security-relevant  information in a timely manner, thereby exposing it to further national  security risks, such as unauthorized disclosures of classified information.  Limited planning, both by ODNI and at the agency level, ultimately puts  the success of the continuous evaluation program—a key aspect of the  security clearance reform effort—at risk. ODNI Lacks Plans for  Monitoring and Measuring  Continuous Evaluation  Program Performance ODNI lacks a plan to monitor and measure the performance of continuous  evaluation across executive branch agencies. Specifically, ODNI officials  stated that ODNI has not developed a plan to monitor or assess the  performance of continuous evaluation across the executive branch,  including for the first phase of implementation, which is underway. ODNI  officials stated that, ideally, agencies will report that they have met the  fiscal year 2017 requirements for the first phase of implementation, and  that ODNI will follow up with agencies that do not report. The officials  added that, in the long term, ODNI would like to incorporate continuous  evaluation into its Security Executive Agent National Assessment  Program, through which it conducts oversight of the security clearance  process at executive branch agencies, but that continuous evaluation is  not currently included in the oversight program. As previously discussed,  according to Executive Order 13467, ODNI, as the Security Executive  Agent, is to direct the oversight of investigations, reinvestigations,  adjudications, and, as applicable, polygraphs for individuals’ eligibility for  access to classified information, or eligibility to hold a sensitive position  made by any agency. Similarly, Executive Order 12968, as amended,  indicates that ODNI is responsible for determining standards for  continuous evaluation. According to ODNI, its authorities under the  Executive Orders include responsibility for and oversight of continuous  evaluation as it is an investigative activity that supports eligibility  determinations. Standards for Internal Control in the Federal Government emphasizes the  importance of assessing performance over time, noting that ongoing  monitoring should be built into operations, performed continually, and  responsive to change. The PMBOK® Guide also states that project  management includes monitoring and controlling work to meet  performance objectives. Without developing a plan to monitor continuous evaluation—including  assessing continuous evaluation at various phases of implementation— ODNI cannot ensure that continuous evaluation is being conducted  consistently across the executive branch, and it may experience  challenges in identifying any needed modifications to the program.  Further, ODNI cannot ensure that continuous evaluation is effectively  meeting its critical purpose of filling the information gap between  investigative cycles to identify risks to national security. Additionally, we reported in 2012 that federal agencies engaging in large  projects can use performance measures to determine how well they are  achieving their goals and to identify any areas for improvement.  Reporting on these measures can help key decision makers within  agencies, as well as stakeholders, to obtain feedback for improving both  policy and operational effectiveness. Moreover, performance measures  need to provide managers and other stakeholders with timely, action- oriented information in a format that helps them make decisions that  improve program performance. Throughout our body of work on leading  performance management practices we have identified several attributes  of successful performance measures, which include, among other things,  measures that are clear, quantifiable, and objective, and that are linked to  measurable goals. However, ODNI has not developed and distributed to executive branch  agencies performance measures to assess the effectiveness of  continuous evaluation once it is implemented executive branch-wide.  ODNI officials stated that they would like to collect metrics in order to  determine the potential effects of continuous evaluation, in particular on  agency resources. Although these officials stated that they have had  some discussions with DOD about the types of metrics it might want to  collect, such as the number of false positives and the resources required  to address the workload, ODNI has not prioritized the development of  performance measures. In February 2017, ODNI officials stated that they  had not developed—or distributed to DOD or other agencies conducting  continuous evaluation—any performance measures for continuous  evaluation. These officials stated that once continuous evaluation has  matured, ODNI plans to identify appropriate measures and determine a  mechanism to collect and analyze them. In August 2017, ODNI officials  stated that they had developed a draft list of metrics for fiscal year 2017. Once the metrics are finalized, these officials stated that they would issue  guidance to agencies requesting them to report these metrics to ODNI. However, since ODNI initiated the first phase of continuous evaluation in  October 2016, without developing and distributing performance measures  to executive branch agencies, it is unclear whether agencies are  positioned to collect and report the information to ODNI for fiscal year  2017. Developing performance measures before the program fully  matures could help it to identify potential program modifications needed  prior to the next phase of implementation, as well as prior to full  implementation. Further, without developing clear, quantifiable, and  objective performance measures that are linked to measurable goals for  agencies to track, and without determining a process and schedule for  agencies to regularly report those measures, ODNI cannot ensure that  the first phase of the program it has already initiated is effective or  achieving similar results at all agencies, which could ultimately affect  reciprocity. DOD and State Have  Designed, Piloted,  and Evaluated  Continuous  Evaluation to Varying  Extents DOD and State have designed, piloted, and evaluated continuous  evaluation, although their respective approaches have varied in scope,  size, and duration—with DOD’s pilot involving the most record checks,  the largest population, and the longest duration. As previously discussed,  DOD’s efforts to design, pilot, and evaluate continuous evaluation have  been ongoing for more than a decade, and they pre-date efforts at ODNI  to develop and implement an executive branch-wide continuous  evaluation program. According to ODNI officials, as of February 2017,  DOD and State were the only agencies, other than ODNI, that had piloted  continuous evaluation. ODNI officials stated that DOD and State’s pilots  were conducted at the discretion of those agencies, and that while ODNI  did not oversee them, the results of the pilots have helped inform ODNI’s  development of an executive branch-wide program. These pilots were  ongoing prior to ODNI’s December 2016 interim guidance outlining the  fiscal year 2017 continuous evaluation requirements for executive branch  agencies, and as a result, both DOD and State have taken different  approaches to developing their programs. DOD’s Continuous  Evaluation Pilot In October 2014, consistent with the Secretary of Defense’s March 2014  recommendation to implement continuous evaluation and DOD  Instruction 5200.02, DOD initiated a continuous evaluation pilot that  included approximately 100,000 military, civilian, and contractor clearance  holders, using a limited set of trusted commercial and government data  sources. DOD has conducted this pilot in a phased approach, increasing  the number of cleared individuals enrolled over time, in accordance with  enrollment milestones set as part of the Insider Threat and Security  Clearance Reform cross-agency priority goal. Specifically, the department  expanded enrollment to 225,000 DOD clearance holders in December  2015 and 500,000 in December 2016, and it plans to increase the  enrolled population to 1 million by the end of calendar year 2017. The  department has also set an internal goal to enroll all clearance holders  department-wide by the end of fiscal year 2021. DOD has developed its own continuous evaluation IT system—which is  called Mirador, and is separate from the IT system that ODNI is  developing—to conduct automated record checks of commercial and  government data sources on the enrolled population, with the goal of near  real-time identification of adverse information to be considered in the  evaluation of an individual’s continued eligibility for access to classified  information. DOD officials developing the system stated that while they  are currently using Mirador to conduct automated record checks for  continuous evaluation, the system remains under development, and they  are integrating additional data sources and user requirements as those  are identified. As of February 2017, the department had implemented  seven data sources in Mirador, which provide information about  suspicious financial and criminal activity, among other things. Another  nine sources were undergoing testing or were otherwise in progress. The  department expects Mirador to reach initial operating capacity in fiscal  year 2018. DOD officials stated that aspects of Mirador are still manual,  such as enrolling individuals, but that they plan to take steps to automate  them. DOD officials stated that, depending on the data source, they run record  checks on enrolled individuals daily, monthly, quarterly, or annually.  According to DOD officials, if a record check results in an alert, such as  for criminal activity, Mirador forwards the alert to DOD’s continuous  evaluation validation cell—within the Defense Security Service, which  manages the department’s continuous evaluation program—to ensure  that: (1) the alert applies to the correct individual; (2) the issue was not  previously known; and (3) the issue is adjudicatively relevant. DOD  officials stated that if an analyst determines that an alert is valid— meaning that all three of the above statements are believed to be true— then the analyst generates a report and forwards it to the individual’s  designated security manager. Alerts are prioritized for analyst review  according to business rules designed around the severity of the alert, and  according to DOD officials, all alerts are reviewed by a supervisor  following an analyst’s initial determination. The officials stated that  currently, if additional investigative work is required based on the alert,  the results of that investigation are forwarded to an adjudicator to make a  determination as to whether the alert affects the individual’s continued  eligibility for a security clearance. The officials added that the due process  safeguards in place for periodic reinvestigations are also in place for  continuous evaluation. Figure 2 provides an overview of DOD’s  continuous evaluation process. DOD has collected and analyzed metrics on the results of its current pilot.  For example, according to DOD data, as of February 2017, continuous  evaluation had identified 12,400 alerts. Of those alerts, 2,064—pertaining  to 1,816 individuals—were determined to be valid, meaning that they  were adjudicatively relevant and not previously known. According to  DOD, action has been completed on 1,307 of those cases. Specifically,  859 cases were closed with a favorable decision, but context was added  to the individuals’ records; in 375 cases the subject separated and/or no  longer needed access; and 62 cases involved a clearance revocation,  condition, or warning. For DOD’s secret-eligible population, continuous  evaluation helped to identify risk, on average, 6 years 7 months sooner  than the traditional 10-year periodic reinvestigation model, and 1 year 5  months earlier for the top secret-eligible population, which is to be  reinvestigated every 5 years. DOD officials stated that these metrics are  presently tracked manually by the Consolidated Adjudications Facility,  and they identified a need to automate the process, going forward. In addition, DOD officials stated that they have shared the results of the  pilot and lessons learned with ODNI through the Continuous Evaluation  Working Group. For example, DOD identified lessons learned related to  identifying the right data sources, eliminating duplicate alerts, the  frequency of record checks, methods for achieving identity resolution, and  the need for operational access to reporting data. Most recently, DOD  issued Department of Defense Manual 5200.02 in April 2017, which  includes continuous evaluation among the responsibilities and procedures  of the DOD Personnel Security Program. State’s Continuous  Evaluation Pilot State began its continuous evaluation pilot in January 2015 to evaluate  the coverage and reliability of public records information, using a public  records service provider. Specifically, it compared information received  from public record checks, such as criminal and financial activity, against  information contained in personnel security files for approximately 8,600  personnel. State found, among other things, that while public records can  provide coverage beyond the traditional scope of investigations, the  quality of the information varies, and not all jurisdictions participate. State  continued its pilot in 2016 and expanded the enrolled population to  include its entire tier 5 population. Additionally, the focus of the pilot  shifted from evaluating the usefulness of public records information to  evaluating the alerts received. State officials stated that the results of the  public record checks are reviewed by the department’s continuous  evaluation team, which determines whether the information is new,  accurate, and relevant, and if so, whether it needs further review and  investigation. These officials stated that because State has authority to  conduct its own investigations, it is easy to conduct investigative follow- up. According to officials, minor issues, such as traffic violations, are  added to personnel files for consideration during the individual’s next  periodic reinvestigation. According to State officials, as of March 2017, they had not revoked any  clearances as a result of the identification of derogatory information  through continuous evaluation. As of April 2017, State had invested  approximately $2.4 million in its continuous evaluation pilot for contract  costs and personnel to administer the program, and, according to State  officials, ODNI provided approximately one-third of that funding. State  officials stated that because ODNI provided funding, State has voluntarily  shared some lessons learned with ODNI, although it was not tasked to do  so. Some details of State’s pilot were omitted because the information is  sensitive. Number of Agencies  Meeting Periodic  Reinvestigation  Timeliness Goals  Decreased from  Fiscal Years 2012- 2016, and Potential  Continuous  Evaluation Effects Are  Unknown The number of executive branch agencies meeting established timeliness  goals for completing periodic reinvestigations decreased from fiscal years  2012 through 2016. Additionally, while executive branch agencies have  already initiated the first phase of continuous evaluation, the potential  effects of continuous evaluation on periodic reinvestigations and agency  resources are unknown, as they have not been assessed. Executive Branch  Agencies Meeting  Established Timeliness  Goals for Completing  Periodic Reinvestigations  Decreased from Fiscal  Years 2012 through 2016 Our analysis of timeliness data for specific executive branch agencies  showed that the percent of agencies meeting timeliness goals decreased  from fiscal year 2012 through 2016. As part of the Insider Threat and  Security Clearance Reform cross-agency priority goal, since the second  quarter of fiscal year 2014, the PAC has reported quarterly on agency  timeliness. Among other things, the PAC reports on the average number  of days taken, for the executive branch as a whole, to complete the end- to-end process for periodic reinvestigations, as compared with the  following goals for the fastest 90 percent of periodic reinvestigations: 15  days to initiate a case, 150 days to conduct the investigation, and 30 days  to adjudicate—totaling 195 days to complete the end-to-end processing of  the periodic reinvestigation. For fiscal year 2016, the PAC reported that the executive branch as a  whole: did not meet the goal of conducting the investigative portion of  periodic reinvestigations within 150 days for the fastest 90 percent of  cases for any quarter. The average number of days ranged from 175  days to 192 days. did not meet the goal of completing periodic reinvestigations—the  end-to-end goal—within 195 days for any quarter of fiscal year 2016.  The average ranged from 209 days to 227 days. Our analysis of timeliness data for specific executive branch agencies  showed that the percent of agencies that reported meeting timeliness  goals decreased from fiscal year 2012 through 2016. Specifically, while 84 percent of the executive branch agencies met the 150-day  investigative goal for at least three of four quarters for the fastest 90  percent of periodic reinvestigations in fiscal year 2012, only 18  percent of the agencies met the investigative goal in fiscal year  2016. while 84 percent of the executive branch agencies met the end-to-end  processing goal of 195 days for at least three of four quarters for the  fastest 90 percent of periodic reinvestigations in fiscal year 2012, only  22 percent of the agencies completed their fastest 90 percent of  periodic reinvestigations within 195 days for at least three of four  quarters in fiscal year 2016. Of the agencies we reviewed, we found that agencies which use NBIB as  their investigative service provider and agencies with delegated authority  to conduct their own investigations both experienced challenges in  meeting established timeliness goals for periodic reinvestigations in fiscal  years 2015 and 2016. For example, 50 percent of the agencies with  delegated authority completed investigations for at least three of four  quarters for the fastest 90 percent of periodic reinvestigations within 150  days in fiscal year 2015, and 44 percent of agencies with delegated  authority met the timeliness goal in fiscal year 2016. Of the executive  branch agencies for which we obtained timeliness data from ODNI and  which use NBIB as their investigative service provider, NBIB completed  the investigative portion within 150 days for 0 percent of the agencies in  fiscal year 2015, and completed it within that timeframe for 6 percent of  the agencies in fiscal year 2016 for at least three of four quarters for the  fastest 90 percent of reinvestigations. Of the executive branch agencies  we reviewed, 67 percent met the adjudication timeliness goal of 30 days  in fiscal year 2016 for at least three of four quarters for the fastest 90  percent of reinvestigations. Specific details on the timeliness of individual  executive branch agencies’ periodic reinvestigations were omitted from  this report because the information is sensitive. According to NBIB officials, as of June 2017, NBIB’s investigation backlog  totaled approximately 673,000 cases—about 183,000 of which were  periodic reinvestigations for both tier 3 and tier 5 clearances. NBIB cited  the September 2014 decision to not exercise the option of one of its  investigative fieldwork contracts—which led to a loss in capacity and an  increase in the program’s contract costs—and difficulties attracting and  retaining investigative resources as two main challenges to timeliness.  NBIB officials stated that they are taking steps to address the backlog for  background investigations, including periodic reinvestigations. These  steps include hiring additional federal and contract investigators,  implementing a number of workload management initiatives, and  conducting a business process reengineering review to identify potential  process efficiencies. Additionally, executive branch agencies noted the  increased requirements stemming from the 2012 Federal Investigative  Standards, such as continuous evaluation and more frequent periodic  reinvestigations for certain clearance holders, as additional challenges to  meeting timeliness goals. In 2008, the Joint Security and Suitability Reform Team issued Security  and Suitability Process Reform, a report to the President that, among  other things, includes OMB-issued interim government-wide processing  goals for security clearances for calendar year 2008. The calendar year  2008 government-wide goal for the fastest 90 percent of periodic  reinvestigations is the same as the goal currently in place: 195 days to  complete the end-to-end processing of the periodic reinvestigation. The  report states that OMB issued the interim goal to assist agencies in  projecting workload and resource requirements. However, the timeliness goals on which the PAC currently reports for  periodic reinvestigations are the same as those identified by OMB as  interim goals for calendar year 2008. Unlike initial investigations, for  which timeliness objectives are established by statute, the 195-day goal  for the end-to-end timeliness of periodic reinvestigations was an interim  goal set by OMB for calendar year 2008. The 2008 report to the  President does not detail how the goals were developed or what data, if  any, were used to establish them. ODNI officials initially stated that they  did not know how the 195-day goal was developed or where it was  documented, and did not know whether subsequent, finalized goals were  ever established, but they later provided a copy of the 2008 report. A  senior NBIB official stated that OMB’s interim calendar year 2008  timeliness goals were developed based on the average timeliness of the  fastest 90 percent of periodic reinvestigations at that time. Since the  establishment of OMB’s interim goals, the executive branch has  measured periodic reinvestigation timeliness against those goals, and it  has not conducted an evidence-based review to ensure that 195 days— and the associated goals of the different phases of periodic  reinvestigations—are realistic goals for periodic reinvestigations. Standards for Internal Control in the Federal Government states that  management evaluates and, if necessary, revises defined objectives so  that they are consistent with requirements and expectations. Without  conducting an evidence-based review of the goals, the executive branch  will continue to compare the timeliness of its periodic reinvestigations  against goals that it established almost a decade ago and that may no  longer be appropriate. Further, without ensuring that 195 days, along with  the associated goals of the different phases of periodic reinvestigations,  are appropriate goals, agencies may not be adequately planning for the  amount of time and resources actually required to conduct periodic  reinvestigations, and, as a result, they may experience further timeliness  delays. Moreover, if an agency does not plan for sufficient time to conduct  periodic reinvestigations, it may allow individuals to retain access to  sensitive documents when it has not yet confirmed those individuals’  continued eligibility, which could have potential repercussions for national  security. Potential Effects of  Continuous Evaluation on  Periodic Reinvestigations  Are Unknown The potential effects of continuous evaluation on periodic  reinvestigations, such as possible changes to their frequency or scope,  remain unknown. In addition, the executive branch’s plans for replacing  periodic reinvestigations with continuous evaluation have evolved over  time. For example, the 2008 Security and Suitability Process Reform  report to the President outlined plans to replace the periodic  reinvestigation model with continuous evaluation, conducting continuous  evaluation annually or at least once every 5 years, depending on an  individual’s security clearance level. The report identified a June 2009  milestone to develop an implementation plan to transition from periodic  reinvestigations to continuous evaluation, and as previously discussed,  an estimated operational date of the fourth quarter of fiscal year 2010  (see figure 1). The purpose of the change was to reveal security-relevant  information earlier and to provide increased scrutiny on populations that  could potentially represent risk to the government because they already  have access to classified information. However, ODNI documentation  states that continuous evaluation supplements and enhances, but does  not replace, established personnel security processes. Executive branch agencies have expressed varying views about potential  changes to the periodic reinvestigation model. For example, DOD officials  stated that with workload and funding issues, they see no alternative but  to replace periodic reinvestigations for certain clearance holders with  continuous evaluation, as the record checks conducted are the same for  both processes. In addition, DOD officials stated that they believe  continuous evaluation will not only result in the more timely identification  of security-relevant information, but will also help to change individuals’  behaviors—for example, that individuals will be more likely to self-report  such information once they are enrolled in the program. DOD officials  also noted that if changes are not made to the periodic reinvestigation  process, the investigation backlog will persist, because continuous  evaluation alerts will continue to add to the investigative workload. In addition, in September 2016, PERSEREC issued a report on a study it  conducted on the effectiveness, timeliness, and cost of various  automated record checks-based investigative strategies as compared  with traditional periodic reinvestigations. The analysis found that some of  the automated record checks strategies were effective, improved the  timeliness of issue detection, and lowered costs. However, DOD officials  noted that because ODNI is the Security Executive Agent, it must  approve the change to the investigative process. These officials stated  that they hope to influence this change by demonstrating the  effectiveness of continuous evaluation at DOD. Additionally, NBIB officials  stated that continuous evaluation will increase their workload and costs,  since it is an additional layer to the personnel security clearance process.  Accordingly, they hope that ODNI will identify efficiencies that can be  made to the process. Further, PAC Program Management Office officials  stated that there may be changes to the periodic reinvestigation model in  the future, but that any changes to the model will be determined by data  and will be made under the authority of ODNI and OPM as the Security  Executive Agent and the Suitability Executive Agent, respectively. Other agencies, such as State, do not share DOD’s view. For example,  State officials stated that although a reduction in costs would result from  replacing periodic reinvestigations with continuous evaluation, they have  concerns that relevant information, such as state and local law  enforcement records that are not yet automated, would be missed if they  did not conduct periodic reinvestigations. Similarly, officials from the  Department of Justice and the Department of Homeland Security stated  that they do not intend to replace periodic reinvestigations, and that  continuous evaluation is to be a supplement to the personnel security  clearance process. However, officials from all three of these agencies  stated that it may be possible to change the frequency or scope of  periodic reinvestigations at some point in the future. ODNI officials stated  that, at this time, they have no intention of replacing periodic  reinvestigations with continuous evaluation, and that the Security  Executive Agent Directive for continuous evaluation, once issued, will  clarify that continuous evaluation is intended to supplement and not  replace periodic reinvestigations. In May 2017, ODNI officials stated that ODNI is not opposed to further  improving the security clearance process, and that once continuous  evaluation is operational, it plans to determine the efficiencies and  mitigation of risks associated with the approach. Specifically, these  officials stated that once continuous evaluation is further implemented  and ODNI has gathered sufficient data—which they estimated would take  about a year from May 2017—they can perform analysis and research to  determine whether any changes are needed to the periodic  reinvestigation model. While executive branch agencies have different views about potential  changes to the periodic reinvestigation process, officials from five of the  seven executive branch agencies we spoke with identified the potential  expenditure of increased resources, such as workload and costs, as a  risk associated with the implementation of continuous evaluation.  Specifically, all five agencies stated that continuous evaluation will  increase their workloads—and therefore costs—if no other changes are  made to the personnel security process. For example, DOD officials  noted that adjudicator workloads will increase as new investigative  leads—identified through continuous evaluation—require adjudication.  Senior DOD officials stated that DOD cannot afford to conduct both  continuous evaluation and periodic reinvestigations. Specifically, DOD  estimates that implementing the 2012 Federal Investigative Standards  requirement to conduct more frequent periodic reinvestigations for certain  clearance holders will cost approximately $1.8 billion for fiscal years 2018  through 2022. In addition, State officials stated that they anticipate that  continuous evaluation will increase their personnel security workload  because alerts will have to be validated, and potentially investigated, and  then adjudicated. Standards for Internal Control in the Federal Government states that  management should identify, analyze, and respond to risks related to  achieving defined objectives. Risk assessment is the identification and  analysis of risks related to achieving defined objectives to form a basis for  designing risk responses. In addition, the PMBOK® Guide states that  entities should perform a quantitative risk analysis to numerically analyze  the effect of identified risks on overall project objectives. The key benefit  of this process is that it produces quantitative risk information to support  decision-making in order to reduce project uncertainty. Although executive branch agencies have identified increased resources  as a risk associated with implementing continuous evaluation, and ODNI  has acknowledged that risk, ODNI, in coordination with the PAC, has not  assessed the potential effects of continuous evaluation on an agency’s  resources. Further, ODNI has not developed a plan, in consultation with  implementing agencies, to address such effects, to include modifying the  scope or frequency of periodic reinvestigations or replacing periodic  reinvestigations for certain clearance holders. While ODNI is  implementing continuous evaluation in a phased approach, having a plan  in place to address the increased workload once continuous evaluation is  fully implemented is critical to ensuring the sustainability and  effectiveness of executive branch agencies’ personnel security programs.  Further, without assessing the potential impacts on agency resources and  developing a plan to address them—once ODNI has further defined the  program—implementing continuous evaluation could further increase the  periodic reinvestigation backlog and agency costs. With delays in  determining continued eligibility, executive branch agencies are assuming  greater risk, which runs counter to the purpose of continuous evaluation. Conclusions Continuous evaluation has been a key and long-standing initiative of  security clearance reform efforts, intended to assist agencies in the timely  identification of security-relevant information that may affect an  individual’s continued eligibility for access to classified information.  However, ODNI has not demonstrated the leadership necessary to make  continuous evaluation a priority. Accordingly, the program’s  implementation has been delayed for almost 7 years. Although ODNI has  taken an initial step to implement it in a phased approach, it has not yet  formalized the program in policy or provided an expanded definition of  continuous evaluation to implementing agencies. In addition, ODNI has  not yet determined key aspects of the program, including future phases of  implementation and agency requirements. Key executive branch  agencies have deemed information about the future phases necessary to  plan for the implementation of continuous evaluation and to estimate  potential costs. The absence of this information has limited their ability to  prepare for the next phases of implementation. This could further delay  the full implementation of continuous evaluation executive branch-wide  and result in inconsistencies among agencies’ approaches. Specifically,  in the absence of ODNI policy and comprehensive guidance, DOD and  State continue to develop their current continuous evaluation programs.  The ultimate effects of such inconsistencies could negatively affect  reciprocity—another key government-wide security clearance reform  effort. Although ODNI is to have oversight of continuous evaluation, it has not  incorporated it into its oversight program or developed a plan to ensure  that agencies implement it. Without a Security Executive Agent Directive  for continuous evaluation that provides an expanded definition of  continuous evaluation and relevant terms to help ensure consistent use; a  plan for implementing continuous evaluation across the executive branch,  that includes future phases of implementation and expectations for  agencies; and a plan for monitoring program performance throughout the  implementation process, as well as performance measures by which to  track and report progress, ODNI is not well-positioned to ensure the  success and effectiveness of the continuous evaluation initiative. Further,  ODNI does not know whether it is meeting the critical purpose of filling the  information gap between investigative cycles to identify risks to national  security. Executive branch timeliness in completing periodic reinvestigations has  declined over the past five years. Further, the executive branch does not  know whether the timeliness goals—set nearly a decade ago—are still  relevant and appropriate, given changes to the personnel security  clearance process. Without conducting an evidence-based review to  ensure that goals for the timely completion of periodic reinvestigations are  appropriate, executive branch agencies may not be planning sufficient  time and resources to complete periodic reinvestigations and therefore  may be challenged to ensure the continued eligibility of the entire national  security workforce. Finally, executive branch agencies have identified  increased resources, such as workload and costs, as a challenge to  implementing continuous evaluation. However, the executive branch has  not determined the potential expected effects of continuous evaluation on  periodic reinvestigations, and agencies have varying views about what, if  any, additional changes should be made to the personnel security  clearance process. Without an assessment of the potential effects of  continuous evaluation and a plan to address those effects—once ODNI  has further defined the program—agencies may not be able to effectively  integrate continuous evaluation into their personnel security clearance  programs, which in turn could lead to further delays in the clearance  process. Recommendations for  Executive Action We are making the following six recommendations to ODNI: The Director of National Intelligence should issue a Security Executive  Agent Directive for continuous evaluation to formalize the program,  which includes, among other things, an expanded definition of  continuous evaluation in advance of the next phase of  implementation. (Recommendation 1) The Director of National Intelligence should, in coordination with the  Continuous Evaluation Working Group, develop an implementation  plan for continuous evaluation across the executive branch that  includes a schedule with timeframes and expectations for agencies,  such as the requirements (e.g., the size of the enrolled population in  continuous evaluation) for future phases of implementation.  (Recommendation 2) The Director of National Intelligence should develop a plan for  monitoring continuous evaluation performance, to include assessing  continuous evaluation at various phases of implementation.  (Recommendation 3) The Director of National Intelligence should develop performance  measures for continuous evaluation that agencies must track and  determine a process and schedule for agencies to regularly report  those measures to ODNI. At minimum, these performance measures  should be clear, quantifiable, objective, and linked to measurable  goals. (Recommendation 4) The Director of National Intelligence should, in coordination with the  Deputy Director for Management of the Office of Management and  Budget in the capacity as Chair of the Security, Suitability, and  Credentialing Performance Accountability Council, conduct an  evidence-based review of the timeliness goal of 195 days for  completing the fastest 90 percent of periodic reinvestigations and the  associated goals for the different phases of periodic reinvestigations,  and adjust the goal if appropriate, taking into consideration available  resources, the additional workload of continuous evaluation, and the  risks associated with individuals retaining access to classified  information without determining their continued eligibility.  (Recommendation 5) The Director of National Intelligence should, once ODNI has further  defined the continuous evaluation program, to include issuing a  Security Executive Agent Directive and developing an implementation  plan, in coordination with the Deputy Director for Management of the  Office of Management and Budget in the capacity as Chair of the  Security, Suitability, and Credentialing Performance Accountability  Council, assess the potential effects of continuous evaluation on  agency resources and develop a plan, in consultation with  implementing agencies, to address those effects, such as modifying  the scope of periodic reinvestigations, changing the frequency of  periodic reinvestigations, or replacing periodic reinvestigations for  certain clearance holders. (Recommendation 6) Agency Comments  and Our Evaluation We provided a draft of this report to ODNI, DOD, OMB, State, NBIB, the  Department of Justice, and the Department of Homeland Security for  review and comment. Written comments from ODNI are reprinted in their  entirety in appendix I. DOD, OMB, NBIB, and the Department of  Homeland Security did not provide comments. ODNI, State, and the  Department of Justice provided technical comments, which we  incorporated in the report as appropriate. In its written comments, ODNI stated that it generally concurred, with  comments, with our six recommendations. However, ODNI stated that it  did not concur with aspects of our overall conclusions and provided  observations in four specific areas. We continue to believe that our  conclusions are valid, as discussed below. First, ODNI disagreed with our conclusion that it has not demonstrated  the leadership necessary to make continuous evaluation a priority. ODNI  noted that it has taken recent actions to better prioritize the  implementation of continuous evaluation. While these recent steps are  positive and may help position ODNI for success, historically ODNI has  not demonstrated the leadership necessary to make the implementation  of a continuous evaluation program a priority. Specifically, while ODNI  refers to continuous evaluation as a new initiative, the original milestone  for implementing the program was the fourth quarter of fiscal year 2010,  which was not attained. Since then, as discussed in the report, a number  of revised milestones for implementing the program have been missed.  For example, the PAC, of which ODNI is a principal member,  subsequently set a milestone for developing an initial continuous  evaluation capability for other clearance holders by September 2014— which was extended to December 2014—and a milestone for  implementing the capability for other clearance holders by December  2016. These milestones were also missed. As of August 2017,  continuous evaluation has not yet been fully implemented, and ODNI has  not set a milestone for when full implementation would occur. As such, we  recommended specific actions that are needed to better position ODNI for  success, including issuing a Security Executive Agent Directive for  continuous evaluation, developing plans for implementing the program  and monitoring its performance, and developing performance measures. Second, ODNI disagreed with our conclusion that it has not yet  determined key aspects of the continuous evaluation program, including  future phases of implementation and agency requirements. ODNI stated  that the Security Executive Agent Directive for continuous evaluation is  undergoing interagency coordination and that it has provided executive  branch agencies with interim guidance until that process is completed,  which we acknowledge in the report. While ODNI has provided interim  guidance for continuous evaluation, it only details the requirements for  fiscal year 2017 and not for the future phases of implementation. In  August 2017, after receiving a draft of our report, ODNI officials stated  that they planned to provide additional guidance to agencies clarifying  that the requirements for fiscal year 2018 will be the same as those for  fiscal year 2017. While this correspondence, once issued, will help  agencies with their immediate program planning, ODNI officials stated  that they have not yet determined the requirements for fiscal year 2019 or  beyond, which limits agencies’ abilities to plan beyond the next fiscal year  for the future phases of implementation. Additionally, ODNI stated that the technical development milestones of  the Continuous Evaluation System it is developing are well-established,  tracked, and shared with stakeholders. As discussed in the report,  according to ODNI officials, they have established technical milestones  for the development of ODNI’s Continuous Evaluation System. While this  is an important step in implementing the program, ODNI has not  developed similar programmatic milestones for the overall implementation  of the program, such as when future phases of implementation will occur,  to include full implementation. As discussed in the report, this has limited  the ability of executive branch agencies to plan for implementation in  accordance with ODNI’s phased approach. As a result, full  implementation—which has been delayed for almost 7 years—may be  further delayed. Third, ODNI did not agree with our conclusion that although it is to have  oversight of continuous evaluation, it has not incorporated it into its  oversight program or developed a plan to ensure agencies implement it.  In its response, ODNI identified its intention to take certain actions and  future mechanisms that could position it to monitor continuous evaluation.  Specifically, ODNI stated that continuous evaluation metrics will be  collected and analyzed when the initial phase of continuous evaluation  implementation ends on September 30, 2017. Additionally, ODNI stated  that it will leverage a pending OMB budget data request and that its  Security Executive Agent National Assessments Program will be  responsible for analysis and oversight of agency implementation and  operation of continuous evaluation. However, as we note in the report, ODNI has not developed and  distributed plans to monitor or assess the performance of continuous  evaluation across the executive branch, including for the first phase of  implementation. As we note in our report, ODNI officials stated that ODNI  did not oversee the pilots that were conducted by DOD and State, as they  were performed at the discretion of those agencies. State officials noted  that while they have shared lessons learned on their continuous  evaluation pilot, they were not tasked to do so. While ODNI stated in its  written comments that it has specific expertise in researching, measuring,  analyzing, and monitoring personnel security performance across the  executive branch, it has not yet demonstrated these actions with regard to  continuous evaluation. For example, DOD—the executive branch agency  with the majority of security clearance holders—has conducted research  on continuous evaluation since 2001, piloted its program since October  2014, and plans to increase the number of personnel enrolled in the  program to 1 million by the end of calendar year 2017. However, ODNI, in  the capacity as the Security Executive Agent, has not overseen DOD’s  pilot. Moreover, as discussed in the report, as of August 2017—10 months into  fiscal year 2017—ODNI has not yet developed and distributed to  executive branch agencies continuous evaluation performance measures.  At the end of our review, in August 2017, ODNI officials stated that they  have developed a draft list of metrics for continuous evaluation for fiscal  year 2017 and that once the metrics are finalized, they will issue guidance  to executive branch agencies requesting them to report these metrics to  ODNI. While metrics can help to establish a baseline and inform aspects  of a program’s status—and ODNI’s development of draft metrics is a  positive step—performance measures are linked to a goal and inform how  well an agency is doing against that goal. As ODNI has not developed  and distributed performance measures that are clear, quantifiable, and  objective, and that are linked to measurable goals prior to initiating, or  earlier in the first phase of implementation, executive branch agencies  may not be positioned to collect and report these metrics at the end of the  fiscal year. Additionally, as discussed in the report, according to ODNI  officials, while they would like to incorporate continuous evaluation into  their Security Executive Agent National Assessments Program, it is not  currently part of the program. While ODNI has identified steps that could  position it to monitor continuous evaluation in the future, it has not yet  implemented mechanisms to monitor and measure program performance. Fourth, ODNI did not agree with our conclusions that it is not well- positioned to ensure the success and effectiveness of the continuous  evaluation initiative, and that it does not know if it is meeting the critical  purpose of filling the information gap between investigative cycles to  identify risks to national security. However, in its written comments, ODNI  stated that successful implementation of continuous evaluation across the  executive branch requires formal Security Executive Agent policy  guidance, implementation and technical guidance and milestones,  performance measures, and a monitoring program, which we  recommended in the report. ODNI states that it is well-postured to  achieve these goals, and refers to its intention to apply Security Executive  Agent National Assessments Program best practices as a mechanism to  use to monitor and ensure compliance. Although this action could be a  step in better positioning ODNI as continuous evaluation implementation  further proceeds, as noted above and in our report, ODNI has not yet  finalized, distributed, and implemented these and other actions to ensure  that it is currently positioned to ensure success, even while it has initiated  the first phase of continuous evaluation implementation. As noted in our report, although ODNI has taken steps to implement  continuous evaluation in a phased approach, executive branch efforts to  implement continuous evaluation have been a long-standing component  of overall security clearance reform. The actions ODNI intends to take as  it further implements continuous evaluation, as well as the mechanisms it  identified, may better position it and the implementing agencies for  success. However, given the challenges that the executive branch has  faced in implementing continuous evaluation thus far and the continued  delays it has faced, without a fully defined program in place, we believe  that our conclusions remain valid. Finally, in its written comments, ODNI suggested a revision to our sixth  recommendation. Specifically, ODNI suggested adding an explicit  timeframe for completing the action. We believe that ODNI is best  positioned to set an appropriate timeframe for completion based on its  familiarity with the progress of the program and, as such, did not  incorporate this change in our report. We agree with ODNI that  establishing such a timeframe is a positive step. We are sending copies of this report to the appropriate congressional  committees, the Director of National Intelligence, the Secretary of  Defense, the Director of OMB, the Secretary of State, the Secretary of  Homeland Security, the Director of OPM, the Director of NBIB, the  Attorney General, the Director of the Federal Bureau of Intelligence, and  the Director of the Bureau of Alcohol, Tobacco, Firearms, and Explosives.  In addition, this report will also be available at no charge on the GAO  website at http://www.gao.gov. If you or your members of your staff have any questions regarding this  report, please contact me at (202) 512-3604 or farrellb@gao.gov. GAO  staff who made significant contributions to this report are listed in  appendix II. Appendix I: Comments from the Office of the  Director of National Intelligence Appendix II: GAO Contact and Staff  Acknowledgments GAO Contact Staff  Acknowledgments In addition to the contact named above, Kimberly C. Seay (Assistant  Director), Chris Businsky, Molly Callaghan, Jenny Chanley, Dawn  Godfrey, Saida Hussain, James Krustapentus, Michael Shaughnessy,  Rachel R. Stoiko, John Van Schaik, Cheryl Weissman, and Jina Yu made  significant contributions to this report. Related GAO Products High-Risk Series: Progress on Many High-Risk Areas, While Substantial  Efforts Needed on Others. GAO-17-317. Washington, D.C.: February 15,  2017. Personnel Security Clearances: Funding Estimates and Government- Wide Metrics Are Needed to Implement Long-Standing Reform Efforts.  GAO-15-179SU. Washington, D.C.: April 23, 2015. Personnel Security Clearances: Additional Guidance and Oversight  Needed at DHS and DOD to Ensure Consistent Application of Revocation  Process. GAO-14-640. Washington, D.C.: September 8, 2014. Personnel Security Clearances: Actions Needed to Ensure Quality of  Background Investigations and Resulting Decisions. GAO-14-138T.  Washington, D.C.: February 11, 2014. Personnel Security Clearances: Opportunities Exist to Improve Quality  Throughout the Process. GAO-14-186T. Washington, D.C.: November  13, 2013. Personnel Security Clearances: Full Development and Implementation of  Metrics Needed to Measure Quality of Process. GAO-14-157T.  Washington, D.C.: October 31, 2013. Personnel Security Clearances: Further Actions Needed to Improve the  Process and Realize Efficiencies. GAO-13-728T. Washington, D.C.: June  20, 2013. Managing for Results: Agencies Should More Fully Develop Priority Goals  under the GPRA Modernization Act. GAO-13-174. Washington, D.C.:  April 19, 2013. Security Clearances: Agencies Need Clearly Defined Policy for  Determining Civilian Position Requirements. GAO-12-800. Washington,  D.C.: July 12, 2012. Personnel Security Clearances: Continuing Leadership and Attention Can  Enhance Momentum Gained from Reform Effort. GAO-12-815T.  Washington, D.C.: June 21, 2012.  2012 Annual Report: Opportunities to Reduce Duplication, Overlap and  Fragmentation, Achieve Savings, and Enhance Revenue.  GAO-12-342SP. Washington, D.C.: February 28, 2012. Background Investigations: Office of Personnel Management Needs to  Improve Transparency of Its Pricing and Seek Cost Savings.  GAO-12-197. Washington, D.C.: February 28, 2012. GAO’s 2011 High-Risk Series: An Update. GAO-11-394T. Washington,  D.C.: February 17, 2011. High-Risk Series: An Update. GAO-11-278. Washington, D.C.: February  16, 2011. Personnel Security Clearances: Overall Progress Has Been Made to  Reform the Governmentwide Security Clearance Process. GAO-11-232T.  Washington, D.C.: December 1, 2010. Personnel Security Clearances: Progress Has Been Made to Improve  Timeliness but Continued Oversight Is Needed to Sustain Momentum.  GAO-11-65. Washington, D.C.: November 19, 2010. DOD Personnel Clearances: Preliminary Observations on DOD’s  Progress on Addressing Timeliness and Quality Issues. GAO-11-185T.  Washington, D.C.: November 16, 2010. Personnel Security Clearances: An Outcome-Focused Strategy and  Comprehensive Reporting of Timeliness and Quality Would Provide  Greater Visibility over the Clearance Process. GAO-10-117T.  Washington, D.C.: October 1, 2009. Personnel Security Clearances: Progress Has Been Made to Reduce  Delays but Further Actions Are Needed to Enhance Quality and Sustain  Reform Efforts. GAO-09-684T. Washington, D.C.: September 15, 2009. Personnel Security Clearances: An Outcome-Focused Strategy Is  Needed to Guide Implementation of the Reformed Clearance Process.  GAO-09-488. Washington, D.C.: May 19, 2009. DOD Personnel Clearances: Comprehensive Timeliness Reporting,  Complete Clearance Documentation, and Quality Measures Are Needed  to Further Improve the Clearance Process. GAO-09-400. Washington,  D.C.: May 19, 2009. High-Risk Series: An Update. GAO-09-271. Washington, D.C.: January  2009. Personnel Security Clearances: Preliminary Observations on Joint  Reform Efforts to Improve the Governmentwide Clearance Eligibility  Process. GAO-08-1050T. Washington, D.C.: July 30, 2008. Personnel Clearances: Key Factors for Reforming the Security Clearance  Process. GAO-08-776T. Washington, D.C.: May 22, 2008. Employee Security: Implementation of Identification Cards and DOD’s  Personnel Security Clearance Program Need Improvement.  GAO-08-551T. Washington, D.C.: April 9, 2008. Personnel Clearances: Key Factors to Consider in Efforts to Reform  Security Clearance Processes. GAO-08-352T. Washington, D.C.:  February 27, 2008. DOD Personnel Clearances: DOD Faces Multiple Challenges in Its Efforts  to Improve Clearance Processes for Industry Personnel. GAO-08-470T.  Washington, D.C.: February 13, 2008. DOD Personnel Clearances: Improved Annual Reporting Would Enable  More Informed Congressional Oversight. GAO-08-350. Washington, D.C.:  February 13, 2008. DOD Personnel Clearances: Delays and Inadequate Documentation  Found for Industry Personnel. GAO-07-842T. Washington, D.C.: May 17,  2007. High-Risk Series: An Update. GAO-07-310. Washington, D.C.: January  2007. DOD Personnel Clearances: Additional OMB Actions Are Needed to  Improve the Security Clearance Process. GAO-06-1070. Washington,  D.C.: September 28, 2006. DOD Personnel Clearances: Some Progress Has Been Made but Hurdles  Remain to Overcome the Challenges That Led to GAO’s High-Risk  Designation. GAO-05-842T. Washington, D.C.: June 28, 2005. High-Risk Series: An Update. GAO-05-207. Washington, D.C.: January  2005.
Continuous evaluation is a key executive branch initiative to more frequently identify and assess security-relevant information, such as criminal activity. Implementing a continuous evaluation program has been a long-standing goal, with implementation milestones as early as 2010 and DOD pilots dating back to the early 2000's. GAO was asked to review efforts to implement continuous evaluation. This report assesses the extent to which (1) ODNI has implemented an executive branch-wide program and developed plans to monitor and measure its performance; (2) DOD and other agencies have designed, piloted, and evaluated continuous evaluation and (3) agencies completed timely periodic reinvestigations from fiscal years 2012-2016, and the potential effects of continuous evaluation on reinvestigations. GAO reviewed documentation, analyzed timeliness data, and interviewed officials from ODNI and other agencies. This is a public version of a sensitive report that is being issued concurrently. Information that ODNI and State deemed sensitive has been omitted. In October 2016, the Office of the Director of National Intelligence (ODNI) took an initial step to implement continuous evaluation—a process to review the background of clearance holders and individuals in sensitive positions at any time during the eligibility period—across the executive branch, but it has not yet determined key aspects of the program, and it lacks plans for implementing, monitoring, and measuring program performance. For the first phase, agencies are to conduct certain continuous evaluation record checks against a portion of their national security population by the end of fiscal year 2017. However, ODNI has not formalized its policy on what continuous evaluation encompasses, determined what the future phases will entail or when they will occur, or developed an implementation plan. According to all seven agencies GAO interviewed, this uncertainty has affected their ability to plan for the program and estimate its costs. Without a continuous evaluation policy and a fully developed plan, full implementation—which has been delayed since 2010—may be further delayed. Moreover, ODNI lacks a plan to monitor and measure program performance, including for the first phase, which is underway. Without developing such a plan, ODNI cannot ensure that the program is being implemented consistently across the executive branch or that it is effectively identifying risks to national security. The Department of Defense (DOD) and the Department of State (State) have designed, piloted, and evaluated continuous evaluation. Their approaches have varied in scope, size, and duration, as they pre-date ODNI's efforts to implement continuous evaluation executive branch-wide. DOD's pilot involves the most record checks and the largest population. DOD had 500,000 employees enrolled in December 2016, and it plans to enroll 1 million by the end of calendar year 2017 and all clearance holders by the end of fiscal year 2021. Executive branch agencies meeting established timeliness goals for completing periodic reinvestigations decreased from fiscal years 2012 through 2016, and the potential effects of continuous evaluation, including on reinvestigations and resources, are unknown. While 84 percent of the executive branch agencies reviewed by GAO reported meeting the executive branch's 195-day timeliness goal for at least three of four quarters in fiscal year 2012, only 22 percent did so in fiscal year 2016. Also, a 2008 report outlined a plan to replace reinvestigations with continuous evaluation, but ODNI documentation indicates that this is no longer the intent. While agencies expressed varying views about changes to reinvestigations—such as modifying their scope—officials from five agencies stated that the continuous evaluation program will increase their workloads and costs if no other changes are made to the requirements. DOD officials said they cannot afford to conduct both continuous evaluation and reinvestigations, as DOD estimates that more frequent reinvestigations for certain clearance holders will cost $1.8 billion for fiscal years 2018 through 2022. Although agencies have identified increased resources as a risk of the program, ODNI has not assessed the program's potential effects on agency resources. Without assessing the potential effects once ODNI has further defined the program, implementing continuous evaluation could lead to further delays and backlogs in reinvestigations, and could increase agency costs.