Background NIH Institutes and Centers  and Biomedical Research NIH, which had total budgetary resources of $32 billion in fiscal year  2016, is comprised of the Office of the Director and 27 institutes and  centers that focus on specific diseases, particular organs, or stages in life,  such as childhood or old age. As the central office at NIH, the Office of  the Director establishes agency policy and is responsible for overseeing  the institutes and centers to ensure that they operate in accordance with  NIH’s policies. The institutes and centers accomplish their missions  primarily through extramural research programs. Most extramural  research funding is provided for investigator-initiated research projects for  which researchers, through their institutions, submit applications in  response to NIH announcements. In addition to these announcements,  the institutes and centers may issue more narrowly scoped solicitations,  through request for proposals, for research targeting specific areas. All extramural research project applications are to follow NIH’s process of  peer review, which includes two sequential levels of review. The first level  involves non-governmental experts assessing the scientific merit of the  proposed applications and assigning them a priority score. The second  level involves advisory councils at the institute or center associated with  the grant application, that, in addition to scientific merit, consider the  institutes’ and centers’ missions and strategic plan goals and public  health needs. Advisory councils review grant applications and their  scores, and, based on this review, make recommendations about which  grant applications should be awarded funding. The director of each  institute or center makes the final extramural funding decisions. NIH investigators also conduct research through NIH’s intramural  research program. These efforts accounted for approximately 10 percent  of NIH’s total budgetary resources of $32 billion in fiscal year 2016. NIH  employs about 3,600 investigators working in its own laboratories and  clinics. In addition, this research relies on another 6,000 investigators at  various stages of research training who come to NIH for a few years to  work as non-employee trainees, including about 2,500 who are  postdoctoral fellows. According to NIH officials, intramural investigators  are generally not allowed to apply for extramural or private grants,  because their salaries are funded with the agency’s appropriations. Career Path of  Independent Extramural  Investigators The career path to become an independent extramural investigator  generally consists of students completing graduate level education (i.e.,  research doctorate or clinical doctorate), postdoctoral research, or  medical residency. When postdoctoral research is completed, the  researcher will generally seek opportunities to become an investigator at  a medical research center or as a faculty member at a university and  begin the process of obtaining academic tenure—that is, a full-time,  permanent faculty position. Once the postdoctoral researcher becomes a  faculty member, he or she can generally begin applying for large NIH  research project grants. Some researchers may become affiliated with  other types of research institutions and also apply for grants. Investigators in medical research centers and university faculty are  generally dependent on external funding to cover the cost of their  research. Although biomedical investigators may be funded by other  federal agencies—such as the National Science Foundation—and  nonfederal sources, studies have shown that NIH is the most likely source  of government funding for biomedical research. NIH Grants NIH’s research support for extramural investigators includes research  project grants, fellowships, training grants, and career development  grants. Some of the main funding mechanisms provided to institutions by  NIH that fund investigators beginning their research careers include the  following extramural grants: Large grants. NIH awards large renewable research project grants:  R01 and R01-equivalent (R01e) grants. According to NIH, in fiscal  year 2016, the average size of large grants was typically in excess of  $460,000 total. R01and R01e grants are NIH’s most common type of  grant, according to NIH. They are generally the largest type of grant  available to investigators beginning their careers and, for purposes of  this report, are therefore referred to as “large” grants. Large grants  provide 3 to 5 years of financial support for discrete, specified  research projects. According to NIH, it is generally expected that  within that period a project can be completed, results published, and  sufficient time will remain for the investigator to prepare a subsequent  application for a renewal or new award before funding ends. Smaller grants. While some non-R01 equivalent (non-R01e) grants  may match or exceed the amount of some R01e grants, they are  generally of a lesser amount and, for purposes of this report, are  therefore referred to as “smaller” grants. According to NIH, in fiscal  year 2016, smaller grants were, on average, amounts that ranged  from about $61,000 to about $1.1 million total. These grants provide  limited funding for a relatively short period of time to support a variety  of exploratory or developmental projects, including pilot or feasibility  studies, collection of preliminary data, and secondary analysis of  existing data. Career development grants. Also known as K-series grants, these  grants are intended to provide mentored research opportunities and  career enhancement experiences to support investigators or  postdoctoral fellows at various stages of their research careers.  NIH’s data show that in fiscal year 2016, career development grants  were, on average, about $178,000 total. Extramural Investigator  Career Status NIH generally classifies the career status of an extramural investigator  based on whether the investigator has received a large NIH research  grant. NIH considers early career investigators to be those who meet the  definition of early stage and intermediate stage investigators. NIH also  recognizes established and “other” investigators among those who apply  for research grants. Table 1 lists NIH extramural investigators’ career  stages and descriptions of these stages. According to NIH, it generally takes an early stage investigator up to 2  years to develop a successful application for a large grant and receive  funding. Typically, investigators devote between 6 months to 1 year to  write their first large NIH grant application. Most of these grants, with a  funding period of over 3 years, require significant preliminary data to  support the proposed hypothesis contained in the application. In addition,  the median average time elapsed for applicants to learn whether they  have been awarded a grant is 270 days, or 9 months. According to NIH,  because most investigators beginning their careers do not receive large  NIH research grants on their first attempt, these investigators might apply  for smaller grants. They may also apply for career development grants  that are intended to provide mentored research or training opportunities. Concerns Regarding the  Stability and Diversity of  the Biomedical Research  Workforce According to research by the National Academies of Sciences,  Engineering, and Medicine, and others, the biomedical research  workforce is growing older at a rate that is disproportionate to the general  American labor force. Some stakeholders in the scientific community  have voiced concerns that large NIH research grants that can launch  early career investigators are often being awarded to established  investigators rather than early stage and intermediate stage investigators.  For example, a recent National Academies report pointed out that  between 1998 and 2003, the NIH budget grew from $13 billion to $27  billion, but the percentage of grants awarded to investigators who were in  the early stages of their careers steadily declined. Many in the field have  reported on the need to support investigators who are researching varied  biomedical issues in order to maximize the number of new discoveries.  Further, stakeholders within the scientific research community have  reported on the uncertain path that investigators may encounter early in  their careers and the prospect that they will ultimately pursue other career  options. Several reports have found that certain racial and ethnic groups are  underrepresented in the biomedical research workforce and in science.  These reports have also provided data on gender workforce disparities.  For example, a 2011 publication by the National Academies of Sciences,  Engineering, and Medicine showed that, in 2006, underrepresented  minorities made up about 29 percent of the U.S. population, but, in 2007,  were awarded about 5 percent of science and engineering doctorates.  Other studies have shown significant research funding disparities for  investigators from underrepresented groups that apply to NIH for large  research grants, such as R01 grants. In 2011, NIH funded a study that  examined the association between grant recipients and the applicants’  race and ethnicity. The study found that R01 applicants that self-identified  as African American were 13 percentage points less likely than white  applicants to receive these grants. After controlling for other variables— including educational background, training, previous research grants, and  publication record—African American applicants were 10 percentage  points less likely to be awarded such a grant than a white applicant.  Further, while women comprise about half of the postdoctoral graduates  for the biological sciences in the United States, studies have shown a  disparity in the number of female investigators in senior science research  positions at universities. This disparity may result in a smaller number of  female investigators among NIH grant applicants and may further  contribute to their underrepresentation in certain facets of science.  However, we previously reported that once female investigators apply for  NIH grants, their likelihood of receiving NIH grants is the same as their  male counterparts. NIH Has Promoted  Efforts to Support  Early and  Intermediate Stage  Investigators, but  Those Who Have Not  Yet Received a Large  NIH Research Grant  Remain Less  Competitive NIH Has Promoted  Programs and Policies to  Support Early and  Intermediate Stage  Extramural Investigators,  but It Is Too Early to  Assess Its Most Recent  Initiative Over the last 10 years, NIH has introduced programs and policies to  support extramural investigators competing for their first large NIH  research grant that leads to research independence. NIH developed  certain programs to fund extramural researchers with the goal of  stabilizing the biomedical research workforce. These targeted programs  were intended to promote support for extramural investigators that had  not yet received a large NIH research grant. The various programs  include both large and smaller research grants, career development  grants, and student loan repayments. Of particular note are the NIH  Director’s New Innovator Award, which is intended to support  investigators beginning their research careers with reviewer-determined  highly novel research; and the Director’s Early Independence Award,  which is intended to support reviewer-determined exceptional  investigators who wish to pursue independent research directly, forgoing  the traditional postdoctoral training period. In addition, the Pathway to  Independence Award provides investigators beginning their research  careers with a mentored research experience, which may lead to  independent research positions. Some institutes and centers have established their own programs to  support investigators beginning their research careers. For example, a  subset of the National Institute of General Medical Sciences’ “Maximizing  Investigators’ Research Award program” targets funding for laboratories  led by an early stage investigator. In addition, the National Institute of  Arthritis and Musculoskeletal and Skin Diseases’ “Supplements to  Advance Research from Projects to Programs,” supports intermediate  stage investigators by providing supplemental funding to existing  research projects to encourage broader innovation and exploration of  high-risk ideas. In addition, NIH’s LRP is designed to help recruit and retain highly  qualified individuals into biomedical research careers. This program  provides student loan repayments in return for a commitment to engage  in NIH mission-relevant and certain statutorily-defined approved  research. We examined the funding rates of early stage and  intermediate stage extramural and intramural investigators who applied  for both initial and renewal LRP payments. LRP payments to extramural investigators: The LRP funding  rate (awardees/applicants) for extramural investigators applying  for total (both initial and renewal) payments between fiscal years  2013 through 2017 was about 50 percent. During this period,  8,186 extramural investigators applied for initial LRP payments  and 3,206 received them; 5,131 extramural investigators applied  for renewal payments and 3,426 received them. Therefore, the  funding rates were 39 percent for initial applicants and 67 percent  for renewal applicants. Early stage and intermediate stage  investigators had similar funding rates in receiving LRP payments  during the 5-year period, though there was some variation each  year. Early stage and intermediate stage investigators seeking  initial LRP payments had funding rates of about 40 percent and 35  percent, respectively. Both of these categories of investigators  seeking renewal LRP payments had a funding rate of 67 percent. LRP payments to intramural investigators: The LRP funding  rate (awardees/applicants) for intramural investigators applying for total (both initial and renewal) LRP payments from fiscal years  2013 through 2017 was about 87 percent; 397 intramural  investigators applied for both initial and renewal LRP payments,  and NIH funded 345 of the applicants. The funding rate for  applicants seeking initial LRP payments during this 5-year period  was about 83 percent, whereas the funding rate for those applying  to renewal LRP payments was 90 percent. NIH also implemented policies to improve opportunities for early and  intermediate stage extramural investigators. For example, to address the  concerns about established investigators receiving a disproportionate  share of research funds, NIH established its Early Stage Investigator  Priority Policy in 2008. The policy specified that early stage investigator  status would be considered a factor when applications were being  selected for award. Studies have shown that under the Early Stage  Investigator Priority Policy, grants being awarded to early stage  investigators stopped declining and remained flat for several years. They  also showed that the field of biomedical research continued to be very  competitive for early stage investigators. However, some have expressed concern that these accomplishments are  not sufficient. For example, according to a recent report by the National  Academies of Sciences, Engineering, and Medicine, a variety of steps  have been taken over the years to address the challenges facing early  and intermediate stage investigators, but these efforts have not resolved  the underlying problems that make it difficult for them to establish their  careers. More recently, the Cures Act required that NIH implement the NGRI,  which the agency established in August 2017. NIH’s Office of the  Director, which oversees the initiative and its implementation, directed the  NIH institutes and centers to reprioritize large NIH research grant support  for early stage and intermediate stage investigators. The policy’s stated  goal for fiscal year 2017 was to increase the number of large NIH  research grants provided to both early stage investigators and  intermediate stage investigators by 200 grants each compared to the  number that were awarded in fiscal year 2016. These 400 grants would  redirect approximately $210 million from NIH’s base budget to support  additional early career investigators in the first year of NGRI’s  implementation. However, with only one month to implement the policy,  NIH did not meet this goal. From fiscal year 2016 to fiscal year 2017, the  number of large NIH research grants awarded increased by 57 for early  stage investigators and decreased by 2 for intermediate stage  investigators. Similarly, the goal to increase funding for the additional 400  grants was not met; funding increased by about $107 million during this  period. Given that this initiative is in the early stages and its goals were  set late in fiscal year 2017, it is too early to fully assess the impact of this  effort. According to NIH officials, the agency is in the process of reevaluating  which investigators should be the focus of the NGRI initiative and may  revise the program to include investigators whose careers are more  advanced. NIH officials stated that the NGRI policy’s intention to direct  more research funding to early stage investigators will remain in place.  However, NIH’s NGRI Working Group no longer designates intermediate  stage investigators—or what it calls “early established investigators”—as  a distinct group. NIH’s current definition—that of being within 10 years of  receiving a first large NIH research grant as an early stage investigator— includes investigators who could have completed their graduate level  education (i.e., research doctorate or clinical doctorate), postdoctoral  research, or medical residency between 15 and 20 years ago. According  to NIH officials, NIH’s working group is considering broadening this  definition even further. It is concerned that intermediate stage  investigators, facing increasing pressure to secure additional sources of  research funding to prevent the closure of their laboratories if their first  large NIH research grant is not renewed, could lose all NIH support and  become likely to leave the biomedical research workforce. Therefore, the  working group is considering a different approach for all established  investigators, with a focus on all meritorious investigators (regardless of  career stage) who are doing high quality research, yet are still at risk for  losing all NIH funding. Specifically, NIH officials said the working group  plans to reevaluate ways that it can provide additional, prioritized support  to these investigators in order to further their career trajectories. The  working group may recommend to NIH that the NGRI be expanded to  also target support for certain investigators whose careers are in more  advanced stages, rather than just those in the early stages of their  careers. In addition, NIH has not yet implemented the expansion of its LRP as  directed by the Cures Act. The Cures Act amended the LRP by  increasing the eligible annual loan repayment amount from a maximum of  $35,000 to a maximum of $50,000. The act also gave the NIH Director  the discretion to amend the research categories that are eligible for  intramural or extramural loan repayment based on emerging scientific  priorities or workforce needs. The agency has established a working  group to provide recommendations to the NIH Director regarding any  suggested structural changes and associated timelines for  implementation. NIH officials told us that they are awaiting  recommendations from this working group on how to use the agency’s  new authorities. They said that they expect to implement program  changes to the LRP, as permitted by the Cures Act, by fiscal year 2020. Investigators Who Had  Received at Least One  Large NIH Grant Had  Higher Funding Rates for  All Grant Types Compared  to Those Who Had Not Our analysis shows that intermediate stage investigators are more  successful at competing for grants than early stage investigators. Our  examination of the trends of NIH grant data showed that the applicant  funding rates (awardees/applicants) for investigators who had previously  received an initial large NIH research grant was greater than the applicant  funding rates for investigators who had never received such a grant. We  analyzed 5 years of grant data to determine an overall perspective of  funding rates from fiscal years 2013 through 2017. We found that  intermediate stage and established investigators—groups comprised of  investigators who had already received their first large grant award—had  greater applicant funding rates for all three grant types compared to early  stage and other investigators. For example, we found that in fiscal year  2017, the most recent year for which data were available, intermediate  stage investigators had funding rates that were comparable to those of  established investigators. Investigators that had not yet been awarded  their first large NIH research grant—early stage investigators and other  investigators—were not as successful when competing for large NIH  research grants, small grants, or career development grants. (See table  2.) We also found that over time—from fiscal years 2013 through 2017— intermediate stage investigators and established investigators had greater  applicant funding rates for all three grant types compared to early stage  and other investigators. Of the investigators that had not yet been  awarded their first large NIH research grant, early stage investigators  were more successful in competing for NIH grants than the other  investigators that were outside of the 10-year period of having completed  their graduate level education (i.e., research doctorate or clinical  doctorate), postdoctoral research, or medical residency. For instance, we  found that early stage investigator funding rates ranged from about 5 to  11 percentage points lower than intermediate stage or established  investigators for each of the five fiscal years examined. Similarly, other  investigator funding rates ranged from about 12 to 14 percentage points  lower than intermediate stage or established investigators for each of the  five fiscal years examined. (See fig. 1.) Finally, we found that during this 5-year period, two of the four extramural  investigator groups were more likely to receive large, small, and career  development grants than the other two groups. Specifically, investigators  beginning their research careers—the early stage and intermediate stage  investigators—were more likely to receive these grants. Although early  stage investigators were more likely than intermediate stage investigators  to apply for smaller research grants (about 4,500 applicants compared to  about 2,000 applicants, respectively) and career development grants  (about 2,000 applicants compared to about 50 applicants, respectively),  intermediate stage investigators were still more successful in competing  for these grants, as well as the large NIH research grants. For more  information on the trends in the number of grants awarded to early stage  and intermediate stage investigators, by award type, for fiscal years 2013  through 2017, see appendix I. NIH Has Taken Steps  to Support a Diverse  Scientific Workforce,  but Disparities Persist  and Its Diversity  Efforts Have Not  Been Fully Evaluated NIH Established Working  Groups and Programs to  Support Investigators from  Underrepresented Groups Over the last 7 years, NIH established advisory groups and other  programs to determine how best to support extramural and intramural  investigators from underrepresented groups.  NIH’s Working Group on  Diversity in the Biomedical Research Workforce was established in  response to the 2011 NIH study that examined the association between  R01 grant recipients and the applicants’ race and ethnicity. NIH directed the group to provide recommendations to improve retention of  underrepresented minorities, the disabled, and scientists from  disadvantaged backgrounds. In June 2012, the working group issued 13  recommendations, which, we found that NIH uses as the foundation of  some NIH-wide efforts to diversify the extramural and intramural  biomedical research workforce. Other advisory groups that have  examined or are currently examining related topics include the following: NIH Working Group on Women in Biomedical Careers was  established in 2007 in response to a report from the National  Academies of Sciences, Engineering, and Medicine on barriers  women in biomedical science experience in advancing their careers.  It produced a workshop and report in 2008 on best practices for  sustaining the careers of women in biomedical research; Addressing Gender Inequality in the NIH Intramural Research  Program Action Task Force was established in 2016 in response to  data showing women are underrepresented in top NIH research  positions. It produced recommendations in 2017 aimed at ensuring  that female and male investigators have equal opportunities in the  intramural research program at NIH, among other things; and African-American/Black R01 Funding Disparities Working Group was  established in response to the 2011 NIH study that found a funding  disparity between blacks and whites applying for R01 grants. This  group analyzed data on the funding rates of applicants that self- identify as African American or black compared to other racial  groups. NIH has acted on some of the advisory groups’ recommendations. For  example, in response to recommendations made by the Diversity in the  Biomedical Research Workforce advisory group, the agency hired a Chief  Officer of Scientific Workforce Diversity in 2014; implemented the three- tiered Diversity Program Consortium, which includes the Building  Infrastructure Leading to Diversity program, the National Research  Mentoring Network, and the Coordination and Evaluation Center; and  established a permanent advisory group on diversity. NIH also  developed a “toolkit” that includes training modules to educate intramural  investigator search committee members on biases that can lead to a less  diverse workforce, among other things. In fiscal year 2017, NIH created  an Equity Committee to address recommendations made by the  Addressing Gender Inequality in the NIH Intramural Research Program  Action Task Force to further examine concerns about parity between  male and female intramural investigators and other diversity issues. Other NIH-wide policies and programs may also help to attract, retain,  and develop investigators from underrepresented groups. The 24 NIH  institutes and centers that fund research and the Office of the Director  provide funds for its investigators, called research supplements, to recruit  graduate students, postdoctoral fellows, and others from  underrepresented racial and ethnic groups, as well as those with  disabilities and from economically disadvantaged backgrounds. These  funds provide graduate students, postdoctoral fellows, and others an  opportunity to conduct research and be mentored by an investigator  supported by the specific NIH institute or center or office. Some  stakeholders we interviewed said that the agency’s LRP also may help to  retain investigators from underrepresented groups, noting that the student  loan debt for African American or black graduate students is higher than  that of white graduate students. Physicians from a professional  organization we interviewed said that the LRP helps to attract physician  scientists from underrepresented groups into research careers.  Physicians we interviewed stressed the importance of the LRP to attract  physician scientists into research careers, because these scientists often  have significant medical school debt. Our analysis of extramural LRP data  showed that, in 2017, African Americans or black, non-Hispanics had a  funding rate of about 34 percent for receiving an LRP payment. White,  non-Hispanic applicants had a funding rate for receiving an LRP payment  of about 52 percent. More recently, the National Academies of  Sciences, Engineering, and Medicine recommended that NIH make the  LRP available to all individuals pursuing biomedical physician-scientist  researcher careers, regardless of their research area or clinical specialty.  They also suggested NIH increase the monetary value of loan repayment  to reflect the debt burden of current medical trainees. Some  stakeholders said that NIH’s family friendly policies, such as  reimbursement for child care expenses and parental leave, may also help  address work-life balance issues for female investigators that may  otherwise forego some research duties to care for young children. Additionally, many—at least 17 of 27—of NIH’s institutes and centers  have established their own policies and programs to attract, retain, and  develop investigators from underrepresented groups. For example, the  National Cancer Institute initiated the Continuing Umbrella of Research  Experiences program to provide training and career development  opportunities to enhance and increase diversity in the cancer research  workforce. This program offers research opportunities and development  to future and current scientists from underrepresented groups from middle  school students to investigators who have yet to achieve research  independence. NIH Research Funding  and Workforce Data  Shows that Disparities  Persist for  Underrepresented Groups Although NIH has implemented numerous diversity-related efforts, our  analysis of NIH research grant funding and intramural workforce data  from fiscal years 2013 through 2017 shows that some disparities persist  for investigators from underrepresented racial and ethnic groups, and for  female investigators. NIH Research Grant  Applicants Our analysis of NIH data shows that investigators from underrepresented  racial and ethnic groups comprise a small percentage of applicants. For  example, in fiscal year 2017, applicants from underrepresented racial  groups—that is, American Indian or Alaskan Native, African American or  black, and Native Hawaiians and Pacific Islanders—were 0.2 percent, 1.8  percent, and 0.1 percent, respectively, of all applicants for large NIH  research grants. Applicants from underrepresented ethnic groups— Hispanics or Latinos— comprised 4.3 percent of the applicants for large  NIH research grants. (See table 3.) In contrast, white applicants were  about 64 percent of all applicants for large NIH grants in fiscal year 2017.  Investigators from underrepresented racial and ethnic groups also  comprise a smaller number of applicants than other groups for smaller  NIH grants and career development grants. Among grant applicants from underrepresented racial groups, African  American or black applicants were consistently the largest group  represented. For example, in 2017, among underrepresented racial  groups, African American or black applicants were named as  investigators on about 88 percent of applications for large NIH research  grants, about 89 percent of applications for smaller NIH grants, and about 92 percent of career development grant applications. Hispanics and  Latinos were about 5 percent of applicants for smaller NIH grants and  about 6 percent of applicants for career development grants in 2017. According to data published by the National Science Foundation in 2017,  women represent slightly more than half of all doctorates in biological  sciences. However, from 2013 through 2017, women represented less  than one-quarter of all tenured NIH intramural investigators. For  example, in 2017, 191, (23 percent) of NIH’s 822 intramural tenured  investigators were women. In addition, in 2017, 79, (37 percent), of NIH’s  211 tenure-track intramural investigators were women. Further, in fiscal  years 2013 through 2017, nearly one-third of all extramural investigators  that applied for large grants were women. (See table 4.) Nearly one-third  of all applicants for smaller research grants, and close to half of all  applicants for NIH career development grants, were women. (See app. II  for information on the number of smaller and career development grant  applicants by racial and ethnic groups and gender.) Stakeholders from 8 of the 12 entities we interviewed suggested potential  reasons why the number of NIH research grant applicants among  underrepresented racial and ethnic groups and for women may be limited.  Attrition of biomedical science doctoral students and early career  investigators from these groups is one explanation. Some stakeholders  said that, while in graduate school, students from these groups may be  discouraged from pursuing a biomedical research career as a result of  implicit bias that they encountered with their mentors. Some stakeholders  said lower numbers among women investigators is the result of decisions  of some to start a family in the early stages of their careers, and further  noted the difficulty in re-entering the biomedical research workforce. In  addition, some stakeholders said that students from underrepresented  groups may lack exposure to a sufficiently rigorous education in  mathematics or the sciences prior to entering college, resulting in the low  numbers of biomedical researchers from these groups. Others said the  low numbers of investigators from these groups makes studying this issue  difficult due to a small sample size. Additional administrative demands  placed on individuals who pursue careers as investigators also affect the  number of applicants. For example, some stakeholders said that once  investigators from an underrepresented group attain faculty positions— particularly if there are few faculty members from such groups—they are  frequently tasked with additional administrative duties. We were told that,  often, they are selected because they may be one of a handful of  members of underrepresented groups at some institutions. Their  additional duties include participation on institutional committees as well  as mentoring, particularly undergraduate or graduate students from  underrepresented groups. In addition, representatives of one stakeholder  group said that some research faculty from underrepresented groups feel  additional pressure to participate in such activities, because their absence  would be more apparent and they worry that this may adversely affect  them. Stakeholders also told us that additional duties are time consuming  and leave less time to devote to applying for grant funding. They said that  some biomedical graduate students from underrepresented groups  decide to pursue other fields, because of the competing demands  associated with being an academic, such as grant writing and teaching  responsibilities. NIH Research Grant Applicant  Funding Rates Our analysis of NIH data from fiscal years 2013 through 2017 also shows  that the funding rate for applicants from underrepresented racial groups  applying for large and small NIH grants lags behind that of white  applicants. For example, in fiscal year 2017, the applicant funding rate for  large grants was about 17 percent for underrepresented racial groups and  about 24 percent for Hispanics and Latinos. The funding rate for white  applicants was about 27 percent. (See fig.2.) Among underrepresented racial groups, African American or black  applicants consistently had a lower funding rate for large and smaller  grants than well represented groups during this period (see table 5). The applicant funding rate for career development grants for  underrepresented racial groups increased from about 22 percent to about  32 percent from fiscal years 2013 to 2017, and, for Hispanic and Latino  applicants, from about 30 percent to about 36 percent during the same  period. The applicant funding rate was about 34 percent for white  applicants throughout this period. The large grant funding rate for female investigators was slightly lower  than male investigators. (See fig. 3.) When looking exclusively at R01 grants, as opposed to all large grants,  research has shown that women are less likely to have their initial R01  grant renewed. Our analysis of R01 grant renewal funding showed that,  in fiscal year 2017, the R01 grant renewal funding rate for female  applicants was about 31 percent compared to about 38 percent for male  applicants. (See fig 4.) According to research by NIH, some applicants  that are unsuccessful in obtaining an initial R01 grant may have greater  success if they reapply; however, some stakeholders we interviewed said  women, and some underrepresented racial groups, are less likely to  reapply for an initial R01 grant if they are unsuccessful with their first  attempt. (See app. III for information on the applicant funding rates for  smaller grants and career development grants by gender.) Many stakeholders attributed the underrepresented groups’ lower funding  rates to two factors. First, many stakeholders cited a perceived implicit  bias within the peer review process, which they said may affect the  funding rates for investigators from underrepresented racial and ethnic  groups. They stressed that, many times, peer reviewers approve grants  for investigators from top tier institutions that they are familiar with and  are reluctant to provide high scores to grant applications from other  institutions. Some stakeholders advocated for anonymizing grant  applications to some extent to address this issue. NIH’s Center for  Scientific Review—the center responsible for organizing peer reviews for  grants—is conducting a study that anonymizes certain large grant  applications, and a training module on implicit bias is currently being  offered to NIH peer reviewers. In addition, NIH’s African American/Black  R01 Funding Disparities Working Group has conducted an analysis on  the R01 funding disparities for African American or black applicants from  fiscal years 2010 through 2015, and is currently pursuing several efforts  to address its findings. Lower grant application priority scores and  application resubmission rates among African American or black  applicants were among their findings. The working group is also pursuing  a randomized control trial to assess the effect of mentoring and coaching  on R01 resubmissions and award rates. Second, some stakeholders told  us that only a very small percentage of biomedical science professors at  top tier research schools are from underrepresented racial or ethnic  groups. Some stakeholders suggested that many investigators from  underrepresented groups seeking grants are affiliated with institutions  outside of the top tier that may lack the infrastructure, grant writing  support, and mentoring opportunities, which could help ensure their  success. As a consequence, many investigators from underrepresented  groups are at a disadvantage compared to their peers at top tier  institutions, according to the stakeholders we interviewed. The Effect of NIH’s Efforts  to Strengthen Diversity Is  Unclear; Assessments of  Some Targeted Efforts Are  Incomplete, and Strategic  Goals Lack Quantitative  Metrics and Time Frames Although NIH has taken steps to address concerns about the diversity of  the biomedical research workforce, its accomplishments have not been  fully evaluated. Stakeholders Reported Mixed  Views on NIH’s Efforts to  Strengthen Diversity Positive comments from some stakeholders we interviewed included  praise for the steps NIH has taken to diversify the biomedical research  workforce, the value of the National Research Mentoring Network, and  the research supplements and other training grants offered by NIH’s  centers and institutes, which provide opportunities for students and  postdoctoral fellows from underrepresented groups to work with  established investigators. NIH’s support of conferences and programs,  such as the Annual Biomedical Research Conference for Minority  Students and the Institutional Research and Academic Career  Development Award, was also well regarded by stakeholders. They also  noted NIH’s commitment to diversity and willingness to investigate  diversity issues through advisory groups, and commended the agency on  working to address recommendations from the Working Group on  Diversity in the Biomedical Research Workforce, including hiring a Chief  Officer of Scientific Workforce Diversity. Some stakeholders were actively  engaged in working with NIH on diversity issues. For example, some  physicians from an organization we interviewed said they are working  with the National Institutes on Minority Health and Health Disparities on  issues related to research workforce diversity. Stakeholders, though, also offered less favorable views and characterized  NIH’s efforts as stagnant, ineffective, or in need of better coordination.  For example, some stakeholders  suggested that for NIH’s National Research Mentoring Network, the  matching of mentees to mentors could be improved or mentioned  uncertainty about the program; questioned how often research supplements are utilized, or noted that  better mentoring and follow-up after the postdoctoral fellow’s work is  completed is warranted;  reported that while their organizations initially collaborated with the  scientific workforce diversity office, that office is not very active or  communication eventually dissipated; expressed concern about NIH’s outreach to minority serving  institutions and organizations, such as historically black colleges and  universities, when it began creating programs like the Building  Infrastructure Leading to Diversity program and the National Research  Mentoring Network and for other efforts; and  stressed that NIH should collaborate more with organizations that  represent underrepresented groups, which have already implemented  programs shown to be effective in engaging these communities in  biomedical research. Multiple Assessments of  Targeted Diversity Efforts Are  Ongoing According to NIH officials, evaluations of various NIH efforts are ongoing  and have not been completed. Some examples include the following: Data collection and analysis by the Diversity Program Consortium’s  Coordination and Evaluation Center began in 2017, and is ongoing. In 2017, NIH’s Center for Scientific Review began conducting a study  to anonymize R01 grant applications from African American or black  and white applicants to detect potential reviewer bias during peer  review. The results of this study are expected in 2019. An evaluation of the National Cancer Institute’s Continuing Umbrella  of Research Experiences program, which provides training and career  development opportunities to enhance diversity in the cancer research  workforce, was submitted for publication in a scientific journal and is  currently pending review. Some NIH institutes and centers have conducted evaluations of their  specific diversity efforts. For example, in 2015, the National Institute of  General Medical Sciences analyzed the research supplements provided  to graduate students and postdoctoral fellows from underrepresented  racial and ethnic groups between 1989 and 2006. The study found that  about 65 percent of graduate students and postdoctoral fellows supported  by the program entered research careers in academia, industry, and  government research. About 41 percent of doctoral graduates and 45  percent of postdoctoral fellows supported by this program entered careers  in academic research or teaching compared to about 43 percent of the  U.S. doctoral degree workforce. In 2011, the National Institute on Aging  evaluated its research supplement program and found that the NIH  research grant applicant success rate of former participants from 2002 to  2010 was about 21 percent. The average research grant success rate for  National Institute on Aging grants was about 18 percent during this same  period. NIH Has Developed a  Scientific Workforce Diversity  Strategic Plan, but It Does Not  Include Quantitative Metrics or  Time Frames to Assess the  Progress of Its Strategic Goals In 2016, NIH’s Chief Officer of Scientific Workforce Diversity established a  5-year strategic plan that describes the agency’s five workforce diversity  goals and supporting objectives. The strategic plan includes goals and  objectives that apply to both extramural and intramural investigators.  During the course of our audit work, NIH updated this plan to describe  progress made on each of its diversity goals, which are to: expand scientific workforce diversity as a field of inquiry, build and implement evidence related to diversity outcomes, understand the role of sociocultural factors in biomedical recruitment  sustain nationwide workforce diversity with seamless career  transitions, and promote the value of scientific workforce diversity. NIH officials provided us with performance measures that its scientific  workforce diversity office will use to gauge the agency’s progress in  achieving each of its five strategic plan’s goals. However, these items  outline the particular areas that NIH plans to evaluate, rather than provide  quantitative metrics, evaluation details, or time frames associated with  any of the areas by which to evaluate progress in fulfilling the goals of the  strategic plan. For example, for the first scientific workforce diversity goal  “expand scientific workforce diversity as a field of inquiry” one of the  performance measures is “number of publications stored in the scientific  workforce diversity office’s online database.” Neither the strategic plan  nor the additional documentation that NIH provided specifies a  quantitative metric for the number of publications to be stored in its  database and the time frame for doing so. Similarly, for the second  scientific workforce diversity goal, to “build and implement evidence  related to diversity outcomes” one of the performance measures identified  by NIH is to compare the large grants awarded to African American or  black scientists to those received by scientists who are white or from  other racial and ethnic groups. However, there is no description in either  the strategic plan or the additional documentation provided by NIH that  indicates how and when these comparisons will be made, how the results  of these comparisons will be assessed, and what will be considered as  fulfilling this goal. All of the other areas or “performance measures”  associated with each of the five goals also do not include such details or  time frames. According to documentation provided by NIH its strategic  plan does not explicitly list “specific metrics” because they will be defined  within “the implementation phase of the plan.” However we are at the  midpoint of the implementation of NIH’s 5-year plan, which covers the  period of 2016 through 2020. As of May 2018, these specific metrics were  not yet available. Without quantitative metrics, evaluation details, or time frames for  assessing the agency’s performance against the five goals in its strategic  plan, NIH will be unable to hold itself accountable for fulfilling its goals.  This is inconsistent with best practices for strategic workforce planning,  which call for agencies to monitor and evaluate their progress toward their  human capital goals. These best practices also call for performance  metrics to be specified at the outset to avoid a biased determination of  what counts as “success” after the results are known. Further, this is  inconsistent with federal internal control standards for monitoring, which  require that an agency evaluate and document the results of ongoing  monitoring to determine whether its management strategies are  effectively supporting its objectives, or need corrective action. NIH’s  establishment of goals and associated areas of future evaluation are  positive steps, but absent specific measures by which to hold itself  accountable, the agency will not have a basis to judge its success. Conclusions NIH’s ability to fulfill its mission of advancing scientific knowledge and  innovation to enhance health, lengthen life, and reduce illness and  disability is dependent on its success in sustaining a thriving and diverse  workforce. For decades, concerns have been raised by the biomedical  research community about NIH’s ability to support investigators beginning  their research careers. Similar concerns have been expressed regarding  support for investigators from groups underrepresented in the sciences,  including those from racial and ethnic groups and women. While the  agency has taken many steps during this time, disparities in its research  grant funding persist. NIH has conducted some evaluations of individual  programs and activities, but these have been relatively narrow in focus  and the results of many efforts are not yet available. More recently, NIH  has taken positive steps such as by establishing the position of Chief Officer of Scientific Workforce Diversity, who in turn, created a strategic  workforce diversity plan and related goals and identified areas of future  evaluation. However, NIH does not have quantitative metrics, evaluation  details, and time frames to assess its progress in meeting its strategic  workforce diversity goals. Without these elements, NIH’s ability to assess  how its diversity strategic plan goals are being achieved is hindered.  Thus, NIH is missing an opportunity to better position itself to support  underrepresented groups and address longstanding disparities. Recommendation for  Executive Action The NIH Director should develop quantitative metrics, evaluation details,  and specific time frames to assess its current efforts to support  investigators from underrepresented groups against its scientific  workforce diversity strategic goals, and use the results of its assessment  to guide any further actions. (Recommendation 1) Agency Comments We provided a draft of this report to HHS for comment. In its written  comments, which are reproduced in appendix IV, HHS concurred with our  recommendation and outlined the steps NIH is taking to implement it.  Notably, for example, HHS indicated that NIH is establishing time frames  to assess its progress in meeting its workforce diversity goals. HHS also  provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional  committees and the Secretary of Health and Human Services. In addition,  the report is available at no charge on the GAO website at  http://www.gao.gov. If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or crossem@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix V. Appendix I: Trends in the Number of Grants  Awarded to Early Career Extramural  Investigators by Award Type, for Fiscal Years  2013 through 2017 Table 6 provides details on the number of grants awarded, number of  awardees and award type for early stage and intermediate stage  investigators from fiscal year 2013 through fiscal year 2017. Appendix II: Total Number of Applicants for  Smaller Grants and Career Development  Grants for Fiscal Years 2013 through 2017 Tables 7 through 10 provide details on the demographics of NIH grant  applicants during fiscal years 2013 through 2017. Appendix III: Applicant Funding Rates for  Smaller Grants and Career Development  Grants for Fiscal Years 2013 through 2017 Figures 5 and 6 provide details on the demographics of NIH grant  applicants during fiscal years 2013 through 2017. Appendix IV: Comments from the  Department of Health and Human Services Appendix V: GAO Contact and Staff  Acknowledgments GAO Contact Staff  Acknowledgments In addition to the contact above, Geri Redican-Bigott (Assistant Director),  Carolina Morgan (Analyst-in-Charge), Jackie Hamilton, Toni Harrison,  and Drew Long made key contributions to this report. Muriel Brown,  Giselle Hicks, and Hayden Huang also made contributions to this report.
NIH's success depends on its ability to attract, retain, develop, and otherwise support biomedical investigators—including those employed in its intramural research program as well as those working in its extramural program at universities, academic health centers, and other research institutions. For decades, the agency has faced challenges in supporting early career investigators and those from underrepresented groups, including ethnic and racial minorities and women. The 21st Century Cures Act included provisions that NIH coordinate policies and programs to promote early research independence and enhance the diversity of the scientific workforce. The act also contained a provision that GAO examine NIH's efforts. GAO reviewed the actions NIH has taken to support (1) investigators beginning their biomedical careers; and (2) investigators from underrepresented groups and women. GAO analyzed NIH data from fiscal years 2013 through 2017 on grant funding for investigators by career phase and demographic status. GAO also reviewed relevant laws and NIH policies, programs, and initiatives, and interviewed NIH officials and stakeholders from the scientific research community. The National Institutes of Health (NIH), within the Department of Health and Human Services (HHS), plays a prominent role in the nation's biomedical research. While it employs investigators in its intramural research program, over 80 percent of its budget supports its extramural program, primarily through grant funding to investigators at other research institutions. Given this, NIH has a vested interest in supporting a robust national biomedical workforce, but the agency has acknowledged that the environment is highly competitive and many investigators find that it takes years to obtain the type and amount of funding that typically spurs research independence. GAO's analysis found that extramural investigators who had received at least one large NIH research grant during fiscal years 2013 through 2017 were more likely to receive such grants in subsequent application cycles than investigators who had not yet received such grants. In response to the 21st Century Cures Act, enacted in December 2016, NIH introduced an initiative to prioritize these grants for (1) early stage investigators, who are beginning their careers and have never received a large research grant, and (2) intermediate stage investigators, who are within 10 years of receiving their first large grant as an early stage investigator. However, it is too early to assess this new initiative, which was introduced in August 2017. NIH is currently considering revising the program to include investigators whose careers are more advanced. NIH implemented recommendations made by internal advisory bodies to support investigators from racial and ethnic groups considered by NIH to be underrepresented in biomedical research. GAO's analysis shows disparities for underrepresented racial and ethnic groups, and for female investigators, from 2013 through 2017. For example, in 2017, about 17 percent of investigators from underrepresented racial groups—African Americans, American Indians/Alaska Natives, and Native Hawaiian/Pacific Islanders combined—who applied for large grants received them. In contrast, about 24 percent of Hispanic or Latino applicants, an underrepresented ethnic group, received such grants. Asians and whites—well represented groups—were successful in receiving large grants about 24 and 27 percent of the time, respectively. Though women represent about half of all doctorates in biological science, GAO found that women investigators employed by NIH in its intramural program comprised about one-quarter of tenured investigators. NIH has taken positive steps such as establishing the position of Chief Officer of Scientific Workforce Diversity, who in turn created a strategic workforce diversity plan, which applies to both extramural and intramural investigators. The plan includes five broad goals for expanding and supporting these investigators. However, NIH has not developed quantitative metrics, evaluation details, or specific time frames by which it could measure the agency's progress against these goals.