The report was titled *To Err Is Human* [4]*.* It "made the case" for patient safety, explaining the science of error-making and the theoretical and practical evidence for human-factors-based systems changes. In many ways it was an expansion of the ideas set forth in my 1994 paper, *Error in Medicine,* but it provided more detailed descriptions of methods and provided case studies to illustrate the points.

In addition to documenting the need for attention to the issue of patient safety, *To Err Is Human* explained the concept of using a systems approach based on human factors principles and proclaimed that application of this methodology could have a profound effect. It boldly called for a 50% reduction in medical harm in 5 years.

Specifc recommendations were made to galvanize the healthcare industry into action to improve safety. The recommendations were nicely summarized in a later report of the Presidential Quality Interagency Task Force [5], excerpts from which I quote here, with my comments in italics:

*Establish a Center for Patient Safety at the Agency for Healthcare Research and Quality (AHRQ)* with responsibility for promoting the development of knowledge about errors and to encourage the sharing of strategies for reducing errors. The IOM committee recommends substantial budget increases over the next several years. *(This was the most important recommendation. To move ahead in patient safety, we had to have central leadership and funding. Patient safety had to be a national priority.)*

*Promote voluntary and mandatory reporting of errors***.** First, the IOM recommends that *voluntary* reporting systems should focus on errors that result in little or no harm to patients and should be encouraged by AHRQ. Second, a *mandatory* reporting system should be established to allow state governments to collect standardized information on adverse events resulting in death or serious harm. *(Increased reporting was something everyone thought was important. I was not so sure. (See Chap. 17.) But the idea of having a voluntary system to spur improvement and a mandatory system for accountability did make sense.)*

*Protect reporting systems from being used in litigation.* The IOM urges Congress to pass legislation extending peer review protections to data related to patient safety and quality improvement that are collected and analyzed by healthcare organizations for purposes of improving safety and quality. *(We all agreed this was essential if we were ever to get people to talk about error in the current litigious environment.)*

*Make patient safety the focus of performance standards for healthcare organizations and professionals.* Regulators and accreditors should require healthcare organizations to have meaningful patient safety programs. Purchasers are also encouraged to provide incentives for patient safety programs. The IOM suggests that professional licensing organizations periodically reexamine and relicense professionals based, in part, on their knowledge of patient safety.

Licensing organizations also need to develop more effective means of identifying unsafe practitioners and taking actions against them. It also suggests that professional societies should promote patient safety education. *(This was our pitch that safety is everyone's business. Healthcare organizations, professional societies, and regulators had to step up.)*

*Increase FDA attention to safety in pre- and post-market reviews of drugs.* The IOM specifcally suggests developing standards for safe packaging and labeling; testing of drug names to prevent sound-alike and look-alike errors; and working with doctors, pharmacists, and patients to identify and rectify problems in the post-marketing phase. *(The failure of the FDA to do this was, we thought, unconscionable. They and the manufacturers knew these were problems and looked the other way. It was time to stop.)*

*Encourage healthcare organizations to make a commitment to improving patient safety and to implement safe medication practices.* Healthcare organizations should develop a culture of safety and implement nonpunitive systems for reporting and analyzing errors. These organizations should also follow recommendations for safe medication practices as published by professional and collaborative organizations interested in patient safety. (*Of course! The culture had to change. Making that happen would turn out to be the major challenge, still unresolved (Chap. 23)).*

Because the error report was expected to attract public interest and garner attention for the whole quality effort, chairman Richardson and staff director Corrigan decided that it should be released frst, and the sooner the better. The report was scheduled to be released on Wednesday, December 1, 1999, so staff made an embargoed copy of the report available to elite media ahead of time and planned for a press conference on the 1st. Then things went awry.

On Monday afternoon, November 29, Don Berwick and I were called by IOM staff and asked to come to Washington immediately. A major television network had decided to break the embargo and go with the story on that night's news. The other networks quickly found out, so all of them were going to be doing the story that evening. The IOM decided to release the report immediately but wanted Don and me to come down for interviews on these national networks that evening.

It was a bit of a circus. I was sent to one network and Don to another, and then we each went to a second. It was hot news, but fortunately the stories were reasonably faithful, and the interviews were on point and not over-sensationalized. What got attention was the estimate that there were up to 98,000 preventable deaths a year due to medical errors. That number also headlined the newspaper stories the next day. In all fairness, the news reports did note that these errors and deaths were due to systems failures, but most viewers and readers were so stunned by 98,000 that it was lost in the uproar.

The 98,000 number was actually a last minute extrapolation that Janet Corrigan made by updating to the present the number of preventable deaths estimated by the Medical Practice Study 9 years earlier, which was over 120,000 [6]. In the interim, thanks to managed care and other cost-cutting measures, the number of patients hospitalized had dropped considerably. One could argue that these were sicker patients, so the number of injuries might not have dropped proportionately, but no matter, 98,000 was shocking enough.

The IOM had never seen anything like it. As I later noted in a NEJM editorial, "The speed and intensity with which the IOM report captured media, public, political, and professional attention surprised everyone. Neither the shocking statistics nor its central message, that errors are caused by faulty systems, was new, but the report forcefully brought them to public awareness." [7] In talks about the report later I would parody a fnancial house ad, "When researchers speak, no one listens, when the IOM speaks, everyone listens."

In truth, they hadn't listened to the IOM much before, either, of course, but this was different. *To Err Is Human* was by far the most widely disseminated and commented on report ever issued by the IOM, a record that still stands 20 years later. A later survey showed that 51% of Americans were aware of the report, which was unprecedented. Skeptics have argued that we got so much coverage because it was a slow news day. While that may be true, it was also true that the message was powerful and touched everyone. If the timing was good luck, fne. The cause deserved it!

Within days, Congress scheduled hearings, and president Clinton formed the Quality Interagency Task Force—headed, fortunately, by John Eisenberg, administrator of the newly formed Agency for Healthcare Research and Quality (AHRQ)—to analyze the report and make recommendations. The administration—the president, secretary Shalala, and Chris Jennings, the health policy advisor—all wanted to do something, so they acted quickly in response.

The formation of the Task Force was announced by the president at a press conference in the Rose Garden. A number of us were invited to attend. I remember well the tingle that went up my spine, and I gave a silent "Hooray!" when I heard the president of the USA mouth our mantra: "It's not bad people, it's bad systems." Sixty days later, when the Task Force made its recommendations [5], the president called on all federal health agencies to implement the IOM recommendations. The IOM had accomplished an unprecedented act of agenda setting.

The report's shocking numbers and the recognition that errors are caused by faulty systems came from our earlier work, but the IOM report brought them to public and policy-makers attention in a way that those of us devoted to patient safety had been unable to do. Of course I was delighted. As the patient safety expert member of the IOM committee, it fell to me to be a lead spokesperson for the IOM and an unprecedented opportunity to infuence public policy.

The Annenberg Conference in 1996 brought the research and advocacy communities together for the frst time to focus on patient safety, but it was the IOM report that brought it to the attention of the public and the medical profession.

The IOM report started the patient safety movement.

#### **Postscript**

Fallout from the IOM report was not all positive. Predictably, many physicians took umbrage. They were insulted by what they saw as the implication they were not doing their job properly. A number began to question the numbers. I found this particularly ironic, since none of these objections had been raised 9 years earlier when we published the MPS results. In fact, we were very disappointed at the time with the paucity of reactions!

Over the next year or two, multiple papers were published "proving" our numbers were infated. We knew, of course, that they were underestimates (see Chap. 1). In fact, ultimately a number of later studies corroborated that, revealing adverse event rates that were 2–4 times what we found [8–11].

Several of the negative papers deserve comment. One from the VA "reanalyzed" our data (without consulting us) and claimed that most of the people who died would have died anyway [12]. Leaving aside the methodologic errors behind this conclusion, its moral repugnancy seemed to have escaped the authors: the implication that it's all right to make fatal mistakes on seriously ill patients!

A more interesting assault came from Clem McDonald and colleagues at the University of Indiana who also "reanalyzed" our data again without consulting us—using our screening criteria as risk factors to calculate "excess mortality." [13] The fallacy of this approach was so obvious that I was surprised that JAMA would publish it. To its credit, the editor asked me to comment and put the two papers in the "Controversies" section.

I explained why screening criteria cannot be used as risk factors: Risk factors are characteristics that increase the *likelihood* of a future outcome; e.g., the presence of diabetes increases the likelihood of developing a myocardial infarct. Screening criteria are indicators of an outcome that *has already occurred*. A myocardial infarction indicates that a patient could have diabetes, but it does not prove it or cause the diabetes. Risk factors look to the future, and screening factors examine the past [14].

Sadly, their paper also looked at "excess" mortality. I rebutted this directly with the analysis from our study that 86% of the preventable deaths occurred in patients who were not terminal and for whom the error was the major factor leading to their deaths.

An interesting sidebar to this discussion was that Katie Couric of NBC's Today decided to interview the two of us on TV. I told her that I was on vacation in Vermont and could not come to New York. No problem: they sent the TV crew to us and interviewed me in our living room in Newfane, VT!

I took some delight in catching her and McDonald unawares by opening my remarks with praise for his work as a pioneer in the application of computers to medicine. My compliment was sincere: Clem had made signifcant contributions. The look on Katie's face was priceless. I then went on to say that in this case unfortunately he was wrong and explained the difference between risk factors and screening criteria. I don't remember what Clem said, but I came away feeling we won that one.

But the critique that hurt came from my colleague and co-author of the MPS, Troy Brennan. Troy was not enamored of my work on error, particularly disclosure. He frmly believed that it would lead to more malpractice suits. In a paper in *The New England Journal of Medicine*, he took aim at the IOM report's preventability numbers, which were derived from our study! [15] He debunked our conclusions, asserting that we had not asked about error in the MPS, which was not true [16]. Whether this was intentional or an honest mistake, I don't know. What saddened me, though, was that he never discussed it with me or gave me—his co-author on the study—an opportunity ahead of time to review his paper. I frst learned about it when I read my copy of NEJM.

I spoke to the editor, Marcia Angell, about the fact that the paper included a falsehood and asked her to publish my rebuttal. She refused, suggesting instead that I submit a letter to the editor! Before doing so, I consulted with Bill Richardson, chair of the IOM committee, and we agreed that I should write the response, but it would have greater impact coming from the committee. We pointed out the error and also why our estimate of the number of preventable injuries was not an exaggeration [17].

After this furry of debunking reports in 2000, the academic chatter quieted down, and the serious effort to make healthcare safe began. The problem was, unhappily, worse than our numbers had indicated. But the IOM report got us moving.

#### **Appendix 9.1: Committee on Quality Of Health Care In America**

## **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 10 The Government Responds: The Agency for Healthcare Research and Quality**

When the IOM report started the patient safety movement by converting the safety interest of a few into the concern of the many, those who wished to enter this emerging feld had little to work with: few measures, few proven safe practices, and few standards. For the patient safety movement to blossom in the ways envisioned by the IOM, a substantial amount of foundational work would be necessary. Only the government could provide the resources that were needed to accomplish this work.

Fortunately—amazingly, actually—the federal government was ready and willing to provide those resources, thanks to the recent work by John Eisenberg, director of the Agency for Healthcare Policy and Research (AHCPR) and his team. The Agency had been through a tough patch. Commissioned in 1989 to conduct health services research and develop practice guidelines, it did just that under it frst director, Jarrett Clinton, a career PHS bureaucrat. But it lost many of its supporters in Congress in 1994 when the Republicans gained control of both the House and the Senate on a pledge to broadly reduce government: the Contract with America. The federal budget for FY 1996 became the focus of an extraordinarily contentious battle between the administration and Congress.

The Agency was caught in the crosshairs. It was linked to the failed Clinton health reform initiative, to which it had supplied data, and its effort to develop practice guidelines was unpopular with many physicians. Worse, its guidelines program had been criticized as being ineffective by the GAO, PPRC, and OTA.

The proverbial "straw" was when the Agency issued a guideline for spine surgery that concluded that there was no evidence to support spinal fusion to treat back pain. Their integrity and pocketbooks challenged, the orthopedists lobbied fercely to curtail the agency. Enough Congressmen had had back operations or philosophically agreed that the government had no business dictating practice, that they set out to eliminate the agency by reducing its funding to zero.

The House Budget Committee under Rep. John Kasich (R-OH) made AHCPR a symbol of waste and put the agency's name on its "hit list" of 140 discretionary programs to be eliminated [1]. Rep Sam Johnson of Texas mocked it as "the Agency for High Cost Publications and Research." [2] The joint House-Senate committee conference report in June 1996 called for complete elimination of the agency's funding.

By then, Donna Shalala, secretary of Health and Human Services (HHS), had replaced Jarrett Clinton with Cliff Gaus, the highly respected former head of the Association for Health Services Research (AHSR). Gaus went on a campaign to save the Agency. He listened to insurers, hospitals, doctors, and consumers and reframed its agenda to dissemination of guidelines rather than development through a new National Guidelines Clearinghouse to coordinate private sector guidelines.

He partnered with clinical and professional leaders to create the Quality Measures Clearinghouse and enlisted the Health Insurance Association of America (HIAA), the AMA, the Association of American Medical Colleges (AAMC), and the American Association of Health Plans (AAHP) to put their lobbyists to work on Congress for support [3].

The strategy worked, although at a price. In the fnal appropriations bill, the agency ended up with an appropriation of \$125 million, a 21 percent cut from FY 1995. The Agency was directed to stop developing practice guidelines, but it survived. By this time, Gaus had had enough. He believed it was time for a physician to direct the Agency. He recommended John Eisenberg to Shalala, noting that in addition to his superb professional qualifcations, he was also her personal physician [3].

Shalala agreed. She was determined to improve quality of care and realized that patient safety could be the wedge. Interest in safety was rising as the result of our research showing the high rate of preventable harm and the efforts of the AAAS and the NPSF. She thought John was just the person to launch a new effort. And, indeed he was.

Eisenberg brought new stature and an impressive set of strengths to the job. He was a nationally known health services researcher who was chairman of the Department of Medicine and physician in chief at Georgetown University, an IOM member, and a former AHSR president. Having chaired the Physician Payment Review Commission (PPRC) for several years, he had developed trusting relationships with key staff on both sides of the aisle. As a physician he brought increased legitimacy to the agency on matters pertaining to clinical care. He was by consensus both brilliant and politically skilled [3].

For John, it was a dream job. It would enable him to leverage his knowledge and skills on the national stage—if he could get Congressional support. Shalala promised to "have his back." He set to work to rebuild the political stature of the Agency on the Hill, shoring up relationships with HHS and members of Congress on both sides of the aisle. He brought in Gregg Meyer to head up the Center for Quality Measurement and Improvement and Nancy Foster as coordinator.

John Eisenberg. (Picture courtesy of the Agency for Healthcare Research and Quality. All rights reserved)

The battle for survival over, the stage was set for moving ahead. In early 1997, the president had established the Advisory Commission on Consumer Protection and Quality in the Health Care Industry and appointed Shalala and labor secretary Herman as co-chairs [4]. In response to its report in 1998, the president established an umbrella organization to coordinate administration efforts to improve quality: the Quality Interagency Coordination Task Force (QuIC), led by Eisenberg and co-chaired by Shalala and Herman. In addition to AHCPR, QuIC included HHS and the Departments of Labor, Veterans Affairs, Defense, and Commerce [5].

In 1998 the tide in Congress turned when Gingrich left the speakership in disgrace. Gaus' efforts had persuaded Congress of the importance of patient safety and the need for its support. The Agency was working with Congress to change its name and mission before the IOM report came out. As a result, a week later, on December 6, 1999, Congress passed the Healthcare Research and Quality Act of 1999 [6] that amended Title IX of the Public Health Service Act to replace AHCPR with the Agency for Healthcare Research and Quality (AHRQ).

Not only did this act get rid of the hated 10-year-old AHCPR, but it greatly expanded the Agency's role in patient safety, calling on it to "conduct and support research and build private-public partnerships to: 1) identify the causes of preventable health care errors and patient injury in health care delivery, 2) develop, demonstrate, and evaluate strategies for reducing errors and improving patient safety; and 3) disseminate such effective strategies throughout the health care industry." [7] It was an incredibly farsighted mandate and a remarkable boon for the cause. In retrospect, it was extraordinary that Congress would designate an agency to focus on patient safety—and fund it.

#### **Response to the IOM Report**

The IOM report in late 1999 changed everything. Within days of its release, president Clinton called on QuIC to analyze the report and make recommendations. Thanks to the amount of planning that John and his colleagues had done before the report came out, just 60 days later, in early 2000, the Task Force issued its report, *Doing What*  *Counts for Patient Safety: Federal Actions to Reduce Medical Errors and their Impact*, which made more than 100 recommendations for federal entities to address safety issues [8].

*Doing What Counts* was a blueprint for moving ahead in patient safety. Its recommendations were directed at all government departments. It called on AHRQ to take immediate action to establish the Center for Quality Improvement and Patient Safety (CQuIPS)9 and for AHRQ, the Centers for Disease Control (CDC), the Food and Drug Administration (FDA), and VA to cooperate on research on errors, reporting systems, and applied research on patient safety. CQuIPS was called on to build a national system of errors reporting, promote the development and dissemination of evidence-based patient safety practices, and develop patient safety questions for inclusion in the patient experience survey, Consumer Assessment of Healthcare Providers and Systems (CAHPS).

The National Quality Forum (NQF) was asked to defne within 12 months a set of egregious errors that are preventable and should never occur. All federal agencies providing healthcare were directed to develop systems to identify and report and learn from errors. The Health Care Financing Agency (HCFA), which paid for Medicare, was called on to promote the use of error-reduction initiatives by healthcare institutions and to require hospitals participating in the Medicare Program to implement medical error reduction programs.

The FDA was to develop standards for proprietary drug names and for packaging and labels to prevent dosing and drug mix-ups. The VA was asked to invest \$47.6 million to increase patient safety training for staff, and the Department of Defense (DoD) was directed to invest \$64 million in FY 2001 to implement a new computerized medical record system.

The scope was breathtaking. Healthcare had never seen anything like it.

Shalala asked Eisenberg to brief the president on the report. When he did, he found that Clinton had not only read the entire report but understood it. He asked John whether it was possible, as the IOM report challenged, for healthcare to reduce preventable deaths by 50% in 5 years. John said yes, or even in 2 years or 1 year—because hospitals would manipulate their measures to make it appear that happened. Such a requirement would divert their attention from improvement to measures. The president saw the wisdom of his answer, publicly approved the report, and called on all federal health agencies to implement the Task Force recommendations.

Congress did its part and appropriated an additional \$50 million to the Agency for patient safety research. This single act was crucial to building a cadre of researchers to perform the scientifc studies needed to advance the feld. It would help establish patient safety as an academic discipline with papers published in the leading journals. It did, in fact, have that effect. Within 2 years, 81 grants were awarded and 100 s of researchers were working on patient safety issues. Without this federal support, research in patient safety would have been slow in coming and spotty at best. It was a foundational initiative.

John Eisenberg was the driving force behind this reinvigorated Agency. His vision of a safer future and how to get there attracted to the Agency an exceptional group of leaders: Gregg Meyer, Carolyn Clancy, and Nancy Foster, among others, who were motivated to get these programs going. Clancy had directed the Outcomes Center at AHCPR prior to the change. They and others who worked at or with AHRQ, or were funded by it, became the frst generation of people who made safety their careers. More than any other person, John Eisenberg helped establish patient safety as a science, as a practice, and as an imperative.

Tragically, John could not see the effort through. He died from a brain tumor in early 2002, just as his efforts were beginning to pay off. To honor John's memory, the NQF and The Joint Commission established the John M. Eisenberg annual awards in patient safety that recognize exceptional contributions by individuals and organizations to the advance of patient safety.

Carolyn Clancy took over as director and expanded AHRQ's activities and infuence as the major force advancing patient safety. She put emphasis on working strategically with multiple stakeholders—hospitals, health plans, federal and private systems, and patients and families—to actually make changes to make healthcare safer. Patient advocates were added to the National Advisory Council. The CAHPS Hospital Survey (HCAHPS) was launched in collaboration with the Centers for Medicare and Medicaid Services (CMS). Proposals were made to link malpractice reforms with improvement in disclosure, apology, and compensation.

The breadth and depth of AHRQ activities in patient safety are awesome. In addition to funding research, it carries out evaluations of evidence, develops and standardizes safety measures and indicators, develops and maintains surveys, trains patient safety specialists, educates the profession and the public, and provides tools for healthcare organizations to improve safety.

AHRQ has played a critical role in establishing the feld of patient safety. A full description of all of AHRQ's programs would require several volumes. What follows is a brief summary of its initial programs in patient safety. Much of this information comes from the AHRQ website, *ahrq.gov*, and AHRQ's 10-year report, *Advancing Patient Safety: A Decade of Evidence, Design and Implementation* [7], also on the AHRQ website.

#### **AHRQ Programs**

**The Center for Quality Improvement and Patient Safety (CQuIPS)** In 1998, prior to the post-IOM reorganization, AHCPR created a center for quality improvement to bring together agencies within HHS to collaborate on improving quality of care and coordinate support for research. Eisenberg expanded this to include patient safety, renamed it, and brought in Gregg Meyer to organize it. Meyer hired Jim Battles to run the research grants program.

CQuIPS supports investigator-initiated research on patient safety, measurement, and reporting. It "develops and disseminates reports and information on health care quality measurement, reporting, and improvement, collaborates with stakeholders … to implement evidence-based practices, accelerating and amplifying improvements in quality and safety for patients." [9] CQuIPS is also responsible for CAHPS and WebM&M (see below). The Center has been the clearly identifable part of government devoted to patient safety.

**Consumer Assessment of Healthcare Providers and Systems (CAHPS)** This survey initiative began in 1995, before the Agency became involved in patient safety, in response to the recognition that quality of care issues that are important to consumers, such as communication skills of providers and ease of access to healthcare, were often overlooked. The obvious way to fnd out about them was to ask patients. The Agency began to fund, oversee, and work closely with a consortium of research organizations to conduct research on patient experience and develop the survey.

150

The survey has since been expanded to ask patients to evaluate their experiences with health plans, providers, and healthcare facilities regarding care coordination, shared decision-making, and patient engagement. The survey is now widely used by healthcare organizations, health plans, purchasers, consumer groups, and accreditation organizations to evaluate providers and improve quality and safety of care. It has been a major factor in teaching clinicians and hospitals to be more aware of patient's concerns and to engage them more meaningfully in their care. It has magnifed their voice.

**Patient Safety Indicators (PSI)** What everyone wants—CMS, policy-makers, hospitals, and the public—is an *overall* measure of safety. How bad is it? Are we getting better? But there was no such measure, nor even a set of standardized measures that the nation or a healthcare organization could use to identify its safety problems. However, CMS and other payers required hospitals to use ICD-9 billing codes for specifc harmful events, such as infections, pressure ulcers, surgical complications, falls, CLABSI rates, etc. Why couldn't hospitals use this "administrative" data to assess and improve their performance?

Under Gregg Meyer's direction, AHRQ created a list of 20 patient safety indicators using billing codes. To develop an overall score, they calculated a weight for each based on national data on risk, reliability of the measure, and extent of harm. The hospital could monitor the rate of each indicator, multiply it by its weight, and sum the values to get an overall measure of safety in the institution. While incomplete all safety risks were not measured—it was better than nothing and would help hospitals know where to focus their safety efforts.

When the PSIs were released in 2001, AHRQ made it clear that the indicators were designed only for hospitals' internal use for improvement: to "provide information on potentially avoidable safety events …that can be used to help hospitals assess the incidence of adverse events and identify issues that might need further study." [7]

It was not to be. Despite these caveats, CMS and other payers began to use them not only to assess hospitals performance, but to reduce payments if they were defcient. Suddenly PSIs were viewed not as tools for improvement, but as instruments of punishment—a complete inversion of what we were trying to do in patient safety.

The patient safety community was appalled. Hospitals quickly directed their efforts away from improvement to coding of claims, "gaming" the data to minimize penalties. This was exactly what Eisenberg had told the president would happen if hospitals were required to reduce preventable mortality by 50% in 5 years. In the end PSIs may have done more harm than good over the years. Meyer agrees. He considers them "the worst thing I ever did" at AHRQ.

**Evidence-Based Practices** Even in these early days, there were a number of established safe practices available. Which should healthcare organizations use? Which safe practices were effective? The newly established standard setter, the National Quality Forum, needed to know, so it turned to AHRQ. The Agency commissioned an Evidence-Based Practices group at the University of California at San Francisco (UCSF), led by Bob Wachter and Kaveh Shojania, to review the evidence and report in 6 months.

Many of us were looking forward to the report, so when it appeared, it was a shocker to fnd that only a small number of practices were found to have evidence of effectiveness, and most of those with the highest ratings were rarely used or fairly esoteric. These practices received high marks because someone had done a randomized controlled trial (RCT) of their effectiveness. Most of the safe practices in widespread use were not on the list because they had not been subjected to RCTs. The UCSF group had done what they were asked to do: follow the evidence. However, the evidence came from studies that individuals did because of their interest in a specifc practice, not because it prevented many errors or was in widespread use [10].

David Bates, Don Berwick, and I were concerned that this would send the wrong message that practices without evidence should be abandoned, so we crafted a critique that was published in JAMA: *What practices will most improve safety? Evidence-based medicine*  *meets patient safety* [11]*.* JAMA published it together with the authors rebuttal: *Safe but sound: patient safety meets evidence-based medicine* [12] as part of a point-counterpoint analysis to help readers understand the types of evidence needed to support use of a new practice.

Our major point was that the many accepted safe practices in current use should not be abandoned just because they had never been subjected to a controlled trial, as required in the review. (The standing joke about RCTs is that no one had ever done a randomized trial of the effectiveness of parachutes!) Handwashing, read-back, site marking, and unit dosing, for example, were practices without evidence that were clearly of value and should not be abandoned.

The paper was written both to reassure those working on the "front lines" of safety in hospitals and to serve as a resource for the NQF Safe Practices Steering Committee. It seems to have been of some value to both. The Safe Practices criteria were expanded to include experiential evidence of effectiveness. The fnal NQF list included 34 approved safe practices [13].

**WebM&M** To engage and inform physicians, AHRQ initiated WebM&M, using the familiar format of mortality and morbidity rounds to make available analysis of real-world medical error cases by experts, monthly. Edited by Bob Wachter, founder of the hospitalist specialty and later chair of the American Board of Internal Medicine, about a third of the cases are also developed as Spotlight Cases that are interactive learning modules for CME. This has proven to be one of the agencies most popular offerings.

**Survey on Patient Safety Culture (SOPS)** To support the development of a culture of patient safety, AHRQ sponsored the development of patient safety culture assessment tools for hospitals, nursing homes, ambulatory outpatient medical offces, community pharmacies, and ambulatory surgery centers. These surveys enable healthcare organizations to assess staff perceptions of various aspects of patient safety culture. They have played an important role in creating a culture of safety.

**National Healthcare Quality and Disparities Report** To justify funding, Congress wants to see results. Part of its 1999 mandate was

#### AHRQ Programs

(a) Carolyn Clancy, (b) Gregg Meyer, and (c) Bob Wachter. (All rights reserved)

that AHRQ produce annual reports on healthcare quality and disparities. The quality reports present trends for access to care, affordable care, care coordination, effective treatment, healthy living, patient safety, and person-centered care. The disparities report provides comparative information according to race and ethnicity, income, and social determinants of health. These reports have helped keep patient safety on the agenda and motivate other agencies to work on safety.

**Education and Training** AHRQ has developed educational programs for practitioners in several areas. *TeamSTEPPS®* is a training program based on an evidence-based set of ready-to-use materials and curriculum to improve teamwork in healthcare organizations by teaching communication and teamwork skills. The *Patient Safety Improvement Corps (PSIC)* is a partnership with the Department of Veterans Affairs to train midlevel professionals in investigation of medical errors and initiating improvements.

*Advancing Pharmacy Health Literacy Practices Through Quality Improvement* is a set of modules to help pharmacy faculty integrate health literacy and health literacy quality improvement into the education of pharmacy students and residents.

Central line-associated bloodstream infections (CLABSI) were a serious cause of preventable injury and death. *CLABSI Tools* help care units implement evidence-based practices to eliminate central lineassociated bloodstream infections. The *Comprehensive Unit-based*  *Safety Program (CUSP)* toolkit developed by Peter Pronovost's team includes training tools to make care safer by improving the foundation of how physicians, nurses, and other clinical team members work together [14].

Several of these education and training programs have had substantial impact. CLABSI Tools were used with the CUSP toolkit in a highly successful nationwide initiative led by Pronovost that dramatically reduced CLABSI rates in more than 1000 hospitals across the country [15]. The Patient Safety Improvement Corps (PSIC) has trained teams in every state and has been a major force in disseminating patient safety knowledge throughout the country. TeamSTEPPS has trained 1000 master trainers who in turn train colleagues at their organizations. Almost every hospital now has a patient safety offcer; many were trained in this program.

**Patient Education** AHRQ has published a number of guides for consumers, such as guides for what to do after leaving the hospital, use of blood thinners, diagnosis and treatment, and questions to ask your doctor.

**Health Information Technology** Suddenly, after only 3 years of supporting the full range of patient safety research, in 2003 Congress directed that AHRQ's \$50 million annual research funding be devoted to research in information technology. This was a shock because the unrestricted funding had been a powerful incentive for developing new knowledge and attracting new investigators to the feld. Research is the coin of the realm in academia, and research gave our new feld academic respectability. Our researchers' papers were being published and they were being promoted.

But the need for support of health IT was clear. Hospitals and doctors were being required to implement electronic health records (EHRs) and were having serious problems. The funding resulted in hundreds of projects related to all aspects of implementation of health IT, and the staff made sure other projects were funded as well. In 2004, AHRQ provided \$139 million for more than 100 multi-year demonstration grants and contracts to promote the use of health information technology.

**Patient Safety Network** To enable healthcare providers and researchers, as well as administrators and patients, to keep up with and easily access the increasing "frehose" of data and patient safety information, AHRQ created a website, psnet.ahrq.gov, with the latest news, research, legislation, and tools for patient safety.

**Healthcare-Associated Infections Network** In 2008 Congress directed AHRQ to work with CDC and CMS to develop an action plan to reduce hospital-acquired infections (HAI). Several nonproft organizations joined what became a major national effort and one of the most successful patient safety campaigns. The reductions in HAI as a result of this program account for a signifcant share of safety improvement over the past 10 years.

**Patient Safety Organizations** From the beginning of the patient safety movement, one of the goals has been to develop national or regional reporting systems so hospitals could share medical error information and learn from each other's mistakes. The IOM called for development of these voluntary systems, but it didn't happen, in part because of hospitals' fear that the information would be legally discoverable and used to sue hospitals and doctors for malpractice.

To eliminate this liability and facilitate sharing of patient error data among hospitals, in 2005 Congress established Patient Safety Organizations (PSOs). Medical error information reported to PSO confdentially is protected from legal discovery. AHRQ also coordinated the development of common defnitions and reporting formats to standardize data collection. Since the law was passed, a number of PSOs have developed across the country. Their effectiveness varies, but some have been useful vehicles for sharing lessons learned.

## **Impact of AHRQ Programs**

As is obvious from the above, AHRQ has played an immense role in the development of all aspects of patient safety. It has given substance to "it's not bad people, it's bad systems"—in research, in practice, and in policy. It has been the main funder of patient safety research. It was the prime mover developing a cadre of patient safety researchers and training hundreds of patient safety offcers.

It furthered the development of measures and set standards for their use. It developed key surveys and large databases that provide the information upon which public policy and private improvement depend. It motivated other federal agencies, such as CMS and CDC, to become major players supporting patient safety.

To evaluate its national patient safety initiative in September 2002, AHRQ entered into a 4-year contract with the RAND Corporation. In 2005 RAND published the frst report [16]. It complimented the agency on "an impressive job in starting the patient safety initiative" that balanced research and translational and practice improvements. It specifcally commended AHRQ for its work in support of epidemiology research, development of effective practices and tools, building infrastructure, and achieving broader adoption of effective practices.

Given its crucial role—and incredible success—it is disturbing and puzzling that funding for AHRQ has always been somewhat precarious. The support for research has never been adequate: large numbers of excellent proposals go unfunded each year. The sudden shift of research funding to IT mandated by Congress in the early years has been followed periodically by other requirements to target its efforts to areas of Congressional interest at the time.

As Gray observed, AHRQ's political problems are three-dimensional. Congress is willing to support basic research, as it does with NIH, only if it believes the long-term result will be new ways of preventing or treating disease. AHRQ's results so far have apparently not been suffciently convincing. Second, when the Agency produces work that affects healthcare practice or policy, it attracts enemies who are vested in the status quo (recall the orthopedists and spine surgery). Third, the agency's work is signifcant to many parties: policy-makers, decision-makers (providers, purchasers, patients), and researchers. They understandably have competing ideas about how the agency's limited resources should be spent [3].

Thus, despite appeals from many health policy experts over the years to "billionize" the Agency, annual funding has remained in the \$300–500 million range. Funding was gradually increased during the Obama years but then cut back by the current administration, which also discontinued several programs.

To health policy experts, it seems obvious that AHRQ should become an institute as part of NIH, with annual funding at \$1–2 billion level, which is less than that currently provided for several institutes for conditions that affect far fewer people and cause far fewer deaths. Surely, it is as important to fund research on *how* we deliver care as it is to fund research on *what* care we deliver. NIH has always enjoyed broad bipartisan and public support. If there were a National Institute for Quality and Safety, funding for patient safety would be adequate and secure. A reasonable hope.

#### **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 11 Setting Standards: The National Quality Forum**

When AHRQ assumed the responsibility from the Quality Interagency Coordination Task Force (QuIC) report, *Doing What Counts for Patient Safety,* to develop practice changes to reduce harm from medical errors, it faced two problems: there were few proven safe practices, and there was a dearth of standards by which to evaluate them. A standard setter was needed.

Fortuitously, a year earlier, the Advisory Commission on Consumer Protection and Quality in the Health Care Industry had recommended an independent organization be created to standardize performance measures in healthcare by means of a public-private partnership. Under vice president Gore's direction, QuIC advanced the idea of a national standard setter and took steps to establish the National Forum for Health Care Quality Measurement and Reporting, later renamed the "National Quality Forum" (NQF), "a broad-based, widely representative private body that establishes standard quality measurement tools to help all purchasers, providers, and consumers of healthcare better evaluate and ensure the delivery of quality services." [1]

The fedgling National Quality Forum fourished under the leadership of Kenneth W. Kizer, once described by Don Berwick as "…probably the most effective leader in all of American healthcare." Kizer was superbly well-equipped for the task. He was board certifed in six medical specialties and had demonstrated his executive skills in several important prior positions. An emergency physician who engineered the statewide EMS system in California and a former US Navy diver and diving medical offcer, Kizer well understood systems and systems thinking. As an active outdoor sports enthusiast and founding member of the international Wilderness Medical Society, he also had a deep appreciation for safety and planning for the unexpected.

Earlier in his career, as director of the California Department of Health Services and the state's top health offcial, Kizer orchestrated California's response to the new HIV/AIDS epidemic, led a cigarette tax increase and smoking cessation program that reduced the rate of smoking in California three times faster than the rest of the nation, and pioneered Medicaid-managed care. In 1994, president Clinton appointed him undersecretary for health in the Department of Veterans Affairs (VA) and chief executive offcer of the VA healthcare system.

During his 5-year tenure at the helm of the VA healthcare system, Kizer radically transformed it, changing it from a hospital system to a truly integrated healthcare system that was rooted in primary care. He closed hospitals, reduced the total number of acute care hospital beds by some 55% (more than 29,000 beds), opened 300 new communitybased outpatient clinics, and hired the frst healthcare system chief telehealth offcer in the country. All well before the "medical home" concept had taken hold in the rest of healthcare.

He reorganized the whole VA healthcare system into 22 new regional "Veterans Integrated Service Networks" (VISNs) that typically consisted of 8–9 hospitals, 25–30 community-based outpatient clinics, 5–7 long-term care facilities, 10–15 counseling centers, and 1 or 2 residential care facilities. Leaders of hospitals and clinics were proximately responsible to the network chiefs for providing quality care [2–4].

Kizer implemented multiple quality improvement changes that led to decreased death rates, a medication bar code system to check dose timings and reduce prescription errors, and a national formulary that resulted in savings of some \$600 million annually. Customer service standards were implemented, and patient satisfaction surveys showed a growing percentage of veterans rated their quality of care as very good to excellent.

As a result of quality assurance measures, illness and death rates from high-volume surgical procedures declined. An observational study published in *The New England Journal of Medicine* found that the VA outscored Medicare's fee for service program for the quality of

Ken Kizer. (All rights reserved)

preventive, acute, and chronic care [5]. All while the number of veterans served increased by 28% in 4 years.

Despite these truly astonishing improvements in the quality and access to care for veterans that Kizer accomplished in an amazingly short time, political opposition developed, largely as the result of his hospital closings and downsizing. Congressional hearings for his reappointment were repeatedly delayed, although the Congress passed specifc legislation extending his tenure at the end of his frst term. Finally, after continuing political drama, Kizer had had enough, and motivated in large part by his wife's serious and deteriorating health problems, he decided to leave VA.

After he resigned, *BusinessWeek* reported that the Veterans Affairs system provided "the best medical care in the US." [6] It was a remarkable transformation of a healthcare system that previously had often been regarded with distain by doctors and laymen alike. The *Harvard Business Review* characterized Kizer's work at VA as the largest and most successful healthcare turnaround in US history.

But the Clinton administration was not about to let Kizer go. Vice president Gore's offce reached out to Kizer about leading the creation of a new organization that would become the NQF. It would be an independent, consensus-based, fnancially sustainable organization having equal representation from healthcare's many and diverse stakeholders that would establish a national healthcare quality improvement strategy that included performance measures to track progress toward achieving the strategy.

This was a momentous step. Thanks to the work of Berwick and others, people in healthcare were beginning to talk about quality improvement and patient safety, but standards of care and valid methods for measuring quality and safety were few. Moreover, as later noted by Kizer, "The concept of the National Quality Forum arose in response to the strong American sentiment against government regulation and control of health care quality…. The (Advisory) commission envisioned that…the NQF would devise a national strategy for measuring and reporting health care quality that would advance the identifed national aims." [1]

Not everyone was enthusiastic, however. NCQA perceived it as a possible direct threat to what it had been doing for years and its business model. The Joint Commission also had reservations, although they later came around. Kizer recalls that Gail Warden, the inaugural chairman of the NQF board, wondered if the organization would last even 3 years.

The National Forum for Health Care Quality Measurement and Reporting was offcially launched on September 1, 1999, with startup funding from the Robert Wood Johnson Foundation, the California Health Care Foundation, the Horace W. Goldsmith Foundation, and the Commonwealth Fund.

Kizer saw the mission of the Forum as "to improve health care quality; that is, to promote delivery of care known to be effective; to achieve better health outcomes, greater patient functionality, and a higher level of patient safety; and to make health care easier to access and a more satisfying experience. The primary strategy…to accomplish this mission is to standardize the means by which health care quality is measured and reported and to make health care quality data widely available." [1]

Kizer set the context for this enterprise by noting "This strategy is premised on the philosophy that health care quality data are a public good and, therefore, health care quality measurements should be publicly disclosed. It is further based on the belief that making reliable, comparative data on health care quality publicly available will motivate providers to improve the quality of care by providing benchmarks; will facilitate competition on the basis of quality; will promote consumer choice on the basis of quality; and will inform public policy." [1]

Five key strategic goals were initially identifed: (1) developing and implementing a national agenda for measuring and reporting healthcare quality, (2) standardizing the measures used to report healthcare quality so that data collection is less arduous for healthcare providers and so that the reported data are of greater value, (3) building consumer competence for making choices based on quality of care data, (4) enhancing the capability of healthcare providers to use qualityrelated data, and (5) increasing the overall demand for healthcare quality data [1].

From the outset, it has been NQF policy that the organization itself does not develop or test performance measures, but instead uses a multistep consensus process to vet measures created by public and private entities, including, among others, NCQA, CMS, Physician Consortium for Performance Improvement, and medical specialty associations.

NQF endorses only those measures that meet the following criteria: [1] importance to measure and report; [2] scientifc acceptability of measure properties, i.e., produces reliable and valid results; [3] feasibility, i.e., require data that are readily available and create as little burden as possible; [4] usability and use, the extent to which they can be used for both accountability and performance improvement; and [5] comparison to related and competing measures to harmonize them or select the best measure [7].

NQF was to be different from any other organization, public or private, in several ways. Indeed, it was broadly viewed as a truly novel experiment in democracy. Membership was open to anyone, organization or individual. Board membership represented a broad and diverse group of stakeholders, including federal agencies (e.g., CMS and AHRQ), state agencies, professional associations (e.g., AHA and AMA), private healthcare purchasers (e.g., GM), labor unions (e.g., AFL-CIO), and consumer groups (e.g., AARP).

Kizer saw the mission of the NQF as blending and balancing consumer, purchaser, payer, and provider perspectives. All Board members had an equal vote. Multiple professional associations initially strongly objected to their vote having the same weight as consumer or purchaser organizations, but Kizer would not budge on this position. The Board's decisions resulted from a consensus process derived from the National Technology Transfer and Advancement Act (NTTAA) and principles formally espoused by the Offce of Management and Budget in OMB Circular A-119. 23 A specially convened Strategic Framework Board of experts supported the NQF's nascent efforts by providing an intellectual architecture and principles to help guide measurement and reporting [8].

Kizer believed that ensuring patient safety should be the foundation of healthcare quality. He decided to take advantage of the contemporary surge of interest in patient safety by having the inaugural NQF effort focus on a safety issue: the reporting of serious harmful events. Bolstered by a formal charge from CMS and AHRQ, he asked me and John Colmer, the program offcer for the Milbank Fund (which was funding the project), to co-chair a Serious Reportable Events Steering Committee to develop a core set of serious preventable adverse events to enable standardized data collection and reporting nationwide. The primary reason for identifying these measures was to facilitate public accountability through national mandatory reporting of these adverse events—an idea that president Clinton's administration was open to, but which was summarily rejected by the subsequent Bush administration.

#### **Serious Reportable Events**

The frst charge to the Steering Committee was to develop a defnition of "serious, avoidable adverse events." We were then to apply a consensus process to develop a set of these events. The fnal set would be voted on by the then 110 NQF member organizations and the Board of Directors. If approved, it would then be issued as a nationwide recommendation. In addition, we were to identify potential candidates for additional measures that needed more research, discuss issues relating to implementation, and develop a plan for dissemination of the measures.

The Steering Committee was composed of representatives from a cross section of healthcare providers, experts in quality and safety, public interest groups, regulators, and others. Broad representation of stakeholders was to be a cardinal principle of operation for the NQF, so a serious attempt was made to make sure all stakeholder sectors were well represented for this frst effort. (Appendix 11.1)

I could see several pitfalls ahead. Following the release of the IOM report, *To Err Is Human*, the most common reaction from the public and the press was to call for required reporting of adverse events. Many people seemed to think that if people just knew about them, they would be taken care of. Doctors and others would be shamed into doing something. Those of us working in safety knew there was much more to it than that. Reporting does not automatically or necessarily lead to change.

Safety experts and policy-makers identify two kinds of reporting: reporting for improvement and reporting for accountability.

Reporting systems for improvement are voluntary and based on frontline caregivers' desire to prevent harm. As Charles Billings, architect of the aviation safety reporting system, has noted, voluntary reporting only works when it is *safe* (does not result in punishment), *simple* (the act of reporting only takes a few minutes), and *productive* (reporting results in positive changes) [9].

Creating a safe environment for reporting within hospitals has long been challenging. Despite national campaigns, 20 years after the IOM report, nearly half of nurses surveyed say they do not feel safe talking about errors. On the other hand, outside the hospital, national voluntary systems, such as those run by specialty societies, ISMP, and the National Nosocomial Infection Survey, rely on reports from caregivers and have been quite successful.

Reporting systems for accountability rely on reports from *institutions*—in healthcare, primarily hospitals—and are mandatory. They are based on the concept that hospitals have a duty to prevent serious harm that we know how to prevent, such as amputation of the wrong leg or giving a blood transfusion to the wrong person. These events result from major system breakdowns, and it is the institution, not individual caregivers, that is in charge of the systems.

The public expects healthcare providers to ensure that care is safe, and it looks to the government to make sure that providers take the actions necessary to make care safe. The occurrence of a serious preventable adverse event suggests that a faw exists in the healthcare organization's efforts to safeguard patients. It is reasonable for the public to expect an oversight body to investigate such occurrences.

These serious reportable events are healthcare's equivalent of airplane or other public-transportation crashes. And most people think the public has a right to know about them when they occur [10]. If so, then not only reporting, but making the reports public should be mandatory.

Reporting is of little value if it doesn't lead to improvement. The healthcare organization must also be required to investigate the event to determine the underlying system problems and/or failures (i.e., root cause analysis) and then correct the failures to prevent recurrence of the event. This information should be disseminated to other healthcare organizations so all can beneft from the lessons learned.

In the USA, the only mandatory systems for reporting of serious events are those run by the states. However, in 1999 only 15 states had such programs, and these varied considerably in what hospitals were required to report and what happened when they did. In most cases, nothing happened: no analysis and no feedback to the hospital—and no reporting of results to the public. The programs were typically understaffed, underfunded, and ineffective [10]. Perhaps providing a nationally accepted, industry-endorsed list of serious preventable adverse events would be an incentive for improvement.

My personal feeling was that it was important to focus on clearly defned adverse events, not on errors or vague things like "loss of function," which appeared in some systems. I argued for events that were simple to defne and "unfudgeable," i.e., not susceptible to interpretation or debate about whether it is or is not reportable. At the time, hospitals routinely gamed the system, going to great lengths to "prove" that an event was not preventable and therefore didn't need to be reported. However, if certain events were *by defnition* preventable, perhaps this charade could be curtailed.

The Steering Committee met for the frst time on December 20, 2000. We defned "serious, avoidable adverse events" as patient harms that hospitals can reasonably be expected to prevent 100% of the time. We had interesting, thoughtful, and sometimes spirited discussions about the purpose of the list, preventive strategies, priorities, verifability of reporting, and specifcity of events. I suggested we eliminate "unanticipated" as being too diffcult to defne and too easy to weasel. More easily than I had expected, we agreed on the defnitions of an adverse event and a serious event, as well as the criteria for inclusion on the list. We did a frst pass, discussing a list of 25 candidates for inclusion that the NQF staff had prepared.

Mandatory reporting is a contentious issue. Many have strong feelings about the public's right to know when these events occur, while hospitals are afraid of liability and loss of reputation from going public with a mistake. Although some hospitals had gone public and found their honesty led people to trust them more, most still did not believe in this degree of transparency.

At the Steering Committee's second meeting in February 2001, Kizer announced a special advisory panel of state health professionals to help us ensure our list would be relevant. We agreed on four criteria for selection of events for the list: events must be (1) serious, (2) clearly defnable, (3) usually preventable, and (4) quantifable (i.e., capable of being easily audited). In other words, they should be events that are serious and obvious to all observers when they occur (the "unfudgeable" part).

In April we approved the fnal report. However, the group did not agree with Ken's interest in calling them "never events," undoubtedly rooted in the frmly held doctrine in medicine that you cannot say that anything "never" happens. So it was decided that the list should be offcially titled "Serious Reportable Events." Nevertheless, they quickly began to be referred to as "never events."

In the fnal version, 27 items were grouped into 6 categories: (1) surgical events (e.g., wrong site, retained foreign body), (2) device or product events (contamination, malfunction), (3) patient protection events (suicide, infant discharged to wrong person), (4) care management events (death or disability from medication error, blood mismatch, kernicterus), (5) environmental events (death or severe disability from electric shock, burn, falls, restraints), and (6) criminal events (sexual assault, impersonation) [11]. The full list is shown in Appendix 11.2. In 3 subsequent updates, the list has been expanded to 29 events.

The Committee's report was readily approved by the NQF Board, with only the American Hospital Association and one state hospital association voting against the report's adoption, and it was published a few months later in early 2002. It was generally well received by the press and the public. It did, in fact, become the model for some state reporting systems. Later, CMS used the list to deny payments for Medicare patients. This was not our intended use, and the matter was vigorously debated and discouraged during the Committee's deliberations. However, I personally felt it had merit. The best way to get hospitals' attention is to hit them in the pocketbook.

All in all, this was an interesting and important initiative. We established important defnitions and expectations. Within 2 years, the number of states with mandatory reporting of serious events increased to 20, and by 2010, 27 states and the District of Columbia had enacted mandatory reporting systems, incorporating all or part of NQF's list.

#### **Safe Practices for Better Healthcare**

The reporting initiative got national attention and started the NQF on the way to Kizer's goal of it becoming "the" trusted and respected national standard-setting organization. He then took on the second QuIC challenge, to "identify a set of patient safety practices critical to prevention of medical errors." This initiative was more ambitious than the reporting project and destined to have far greater impact. It was to be a list and description of evidence-based and standardized care processes that promote safety and reduce patient harm. The objective was to stimulate healthcare organizations to adopt a systems approach by providing effective processes that could be used "off the shelf," saving the care team the effort of developing their own new practices and systems de novo.

To begin the initiative, NQF asked AHRQ to commission an independent review of the evidence behind safe practices. As described in the previous chapter on AHRQ, this effort was led by Kaveh Shojania and Bob Wachter of UCSF [12]. But they found only 11 practices that met its criteria! As noted in the previous chapter. David Bates, Don Berwick, and I were concerned that the many accepted safe practices in current use would be suspect just because they had never been subjected to a controlled trial. Our paper and a rebuttal by the authors were published in JAMA as part of a point-counterpoint analysis [13, 14]. The Safe Practices Committee expanded its criteria to include experiential evidence of effectiveness.

Other sources of candidate practices were the Leapfrog Group, NQF member organizations, Steering Committee members themselves, and an open call for candidate practices to more than 100 medical, nursing, and pharmacy specialty societies.

The fnal criteria for inclusion as a safe practice were:

The practices were organized into fve broad categories for improving patient safety: (1) creates a culture of safety, (2) matches healthcare needs with service-delivery capabilities, (3) facilitates information transfer and clear communication, (4) adopts safe practices in specifc clinical settings or for specifc processes of care, and (5) increases safe medication use.

The fnal list of 30 practices included both those with research evidence of effectiveness and those that were already in wide use and well accepted. Some examples of the latter practices included the following: staffng of ICUs with critical care specialists (intensivists); "read-back" of orders; prohibited abbreviations; medication reconciliation; hand hygiene; unit dosing; adoption of computerized patient order entry; the universal protocol for preventing wrong site, procedure, and patient for all invasive procedures; and protocols for prevention of central line-associated bloodstream infections (CLABSI), surgical site infections, MRSA, and catheter-associated urinary tract infections [15]. (For the full list, see Appendix 11.3.) The list was formally approved by the NQF Board in late 2002.

In April 2004, the Leapfrog Group adopted the full list of safe practices as their fourth safety "leap." (Three of the safe practices were based on its frst three safety leaps.) Also in 2004 several purchaser coalitions (e.g., Pacifc Business Group on Health, The Employer Alliance Health Care Cooperative, Midwest Business Group on Health, among others) endorsed the safe practices.

Individual hospital adoption of the safe practices has varied greatly. Although now widely accepted as the standard toward which to strive, they are not easy to implement. (See Chapters 6 and 8.) Success requires strong support at the executive level, education and training of personnel, a "champion," and teamwork. Physician buy-in is critical. Outside pressure, as from The Joint Commission, has helped.

NQF has periodically updated the safe practices in response to the development of new practices as patient safety matured. In the frst update, in 2006, safe practices were added that addressed leadership and staffng, and the practices were harmonized with safety initiatives from other national groups such as the CMS, AHRQ, and The Joint Commission. In 2009, practices were added to address pediatric imaging, organ donation, caring for caregivers, glycemic control, and prevention of falls.

#### **Performance Measures**

While the serious reportable events and safe practices were highly visible projects, multiple other projects were concomitantly undertaken by NQF during its formative years. For example, performance measures were endorsed for, among other things, adult diabetes care, home healthcare, cardiac surgery, child healthcare, medication safety, hospital care, substance use disorders, and nursing care.

Likewise, a consensus framework for hospital care performance evaluation was developed, approved by the Board, and published, as were position papers or guidance documents on the role of hospital governing boards in promoting quality care, health information technology and electronic health records, health literacy, pay for performance, and improving healthcare quality for minority populations.

The NQF also closely worked with the eHealth Initiative, CMS, AHRQ, and other groups to facilitate the adoption of health information technology and new payment models that supported quality improvement. Kizer strongly lobbied HHS secretary Tommy Thompson to promote the adoption of electronic health records, and he worked closely with CMS administrator Scully to promote adoption of public reporting of performance measurement data.

Throughout this time, a problem that plagued the NQF's efforts was the lack of stable fnancing and especially not having funds to undertake projects that were "for the public good"—i.e., projects that were not linked to a specifc healthcare constituency and its interests. Kizer spent a large amount of time fnding and cobbling together funds to fnance the many projects that NQF undertook in these early years.

#### *New Leadership*

At the end of 2005, Kizer stepped down, and Janet Corrigan took over as president of the National Quality Forum. Corrigan was ideally suited to the role. An expert in health policy and management, she was highly respected for leading the staff at the Institute of Medicine that produced the legendary *To Err Is Human*. But most notably, Corrigan was the executive director of president Clinton's Advisory Commission on Consumer Protection and Quality, which recommended the creation of NQF.

Corrigan faced several challenges. Despite generous support from several foundations, the fnancial situation was precarious. Other reliable sources were needed. In addition, the endorsement process had become unruly. It needed to be put on more rigorous scientifc foundation. It needed to continue expanding the membership base, but more importantly it needs expanded public support. Corrigan brought in Helen Burstin from AHRQ to straighten it out.

Corrigan envisioned new opportunities for NQF. She believed that the NQF would be more effective if it focused more on measures needed to achieve national safety and quality goals. But what were the national priorities; what were the goals? And who set them? Well, it wasn't clear. AHRQ had its priorities, as did CMS, IHI, and others, but there was no uniformity, no consistency, and no single voice.

With support from the Robert Wood Johnson Foundation, in 2007 Corrigan persuaded HHS to ask NQF to establish the National Priorities Partnership (NPP) to provide input to the secretary for consideration as it developed priorities. Under the leadership of Helen Burstin, NPP was developed as a public-private partnership of 51 partner organizations that represent the diverse perspectives of consumers, purchasers, healthcare providers and professionals, community alliances, health plans, accreditation and certifcation bodies, and government agencies. NPP identifed six national priorities that were embraced by many national organizations and health systems.

As the debate around health reform heated up in 2009, NQF helped organize a coalition of quality leaders known as Stand for Quality to encourage legislators to provide stable and adequate support for the core measure activities recognizing that they are fundamental building blocks for virtually all approaches to payment and delivery system reform and for recognition of the important role of having a publicprivate partnership to carry out this work.

The passage of the Affordable Care Act (ACA) in 2010 permitted the realization of these goals. Federal support of NQF's work increased. ACA directed HHS to obtain multi-stakeholder input on setting priorities and selecting measures for use in various federal

Helen Burstin. (All rights reserved**)**

programs. Signifcant support was provided for HHS to contract with a "voluntary consensus standard setting body" (aka NQF) to conduct much of this work.

In response and to complement its priority-setting NPP, NQF developed the Measure Applications Partnership (MAP) to advise the federal government and private sector payers on the optimal measures for use in payment and accountability programs. This closed the loop linking the endorsement process to measures needed to advance the goals established by the NPP. Under the leadership of Helen Burstin, the MAP built on the earlier efforts of the various "quality alliances" but provided for a more patient-centered, coordinated approach to measure selection across various providers, settings, and programs.

MAP has two overarching objectives: to *focus* accountability programs on achieving the NQS priorities and goals and to *align* measurement across the public and private sectors and across settings and populations served. The MAP Coordinating Committee and workgroups are composed of representatives from more than 60 private sector stakeholder organizations, 9 federal agencies, and 40 individual technical experts.

CMS found the recommendations of the MAP essential in 2012 when it adopted value-based purchasing for Medicare and Medicaid services that linked hospitals' payments to their success in achieving reductions in specifc measured bad outcomes, such as catheter-associated urinary tract infections and central line infections. Suddenly, the incentive for hospitals to implement safe practices increased dramatically. CMS contracted with the MAP for further measures to use in this program.

Through the MAP the NQF has advised the government on the selection of measures for use in more than 20 federal public reporting and pay-for-performance programs. About 300 NQF-endorsed measures are currently in use in federal, state, and private sector programs. Over 90 percent of all Medicare payments are now performance-based.

The ACA placed responsibility for setting national priorities within CMS. NQF's National Priorities Partnership provides input to CMS for this function and also plays a role in convening stakeholders to develop action plans to achieve the national priorities and goals.

ACA also charged HHS with developing a National Quality Strategy (NQS) to improve the delivery of healthcare services, patient outcomes, and population health and required that the NQS be shaped by input from a broad range of stakeholders. HHS requested NQF to convene the NPP to provide input to the secretary for consideration as it developed this national body of work. In 2011 it established six priorities: healthy living, prevention of leading causes of mortality, patient safety, person and family engagement, communication and coordination, and affordable care.

NQF thus manages the "supply chain" for quality and safety priorities, setting standards and applying them: the NPP, which sets priorities and goals; measure stewards, who develop and test measures; the evaluation and endorsement consensus process; the MAP that advises on selection of measures for use in accountability applications; and public and private accountability efforts. It is the neutral convener of multi-stakeholder groups that provide the "bridge" between public and private sectors.

In addition to NPP and MAP, the NQF has a broad array of quality and safety programs. Its health IT initiatives support the complex move toward electronic measurement to facilitate data sharing between healthcare providers and their patients. NQF provides information and tools to help healthcare decision-makers and has programs in person- and family-centered care, effective communication, palliative and end-of-life care, and disparities [16].

Many of these programs have been institutionalized by NQF into multi-stakeholder Standing Committees in topical areas. Standing Committees are charged to review and recommend submitted measures for endorsement to NQF's Consensus Standards Approval Committee that considers all measures recommended for NQF endorsement.

#### **Confict of Interest Scandal**

In 2014, the patient safety movement was rocked by its frst major scandal when Charles Denham reached an out-of-court settlement with the US Department of Justice for receiving over \$11 million from a medical products company, CareFusion, to promote their products [17, 18]. Denham had served as co-chair of various NQF committees that produced safe practices reports dating back to 2003; he was chair of the Leapfrog Group's Safe Practices Committee and editor of the *Journal of Patient Safety*. Denham's exposure was a blow to the entire patient safety community, but NQF by far suffered the greatest fallout.

The leadership of NQF was taken completely by surprise. Despite NQF's strict confict of interest policies, Denham had not disclosed his commercial ties. NQF immediately severed its ties to Denham and his foundation TMIT.

Denham had come under suspicion earlier at NQF in 2009 when concerns were raised by both staff and committee members when he lobbied the committee to insert a specifc recommendation in Safe Practice 22 (surgical site infection prevention) to use chlorhexidine gluconate 2% and isopropyl alcohol solution as skin antiseptic preparation, i.e., CareFusion's ChloraPrep. After investigation, the recommendation was replaced with a more generic one, and Denham was removed from his co-chairmanship of the Safe Practices Committee, but no one knew he was being paid by CareFusion.

There was another problem. For years, Denham had been providing substantial fnancial support for NQF. Much of the staff work for the Safe Practices Committee was supplied gratis by Denham's "nonproft" company, Health Care Concepts. Between 2006 and 2009, this organization donated grants totaling \$725,000 to NQF.

When the scandal broke, NQF offcials said that Denham never reported his conficts, despite a specifc requirement for all members to do so. After his fring, NQF took immediate steps to strengthen its processes to ensure the integrity of quality measures and safe practices, and it reviewed all of the standards set by the committee Denham co-chaired. It also established a policy of not accepting money from funding organizations whose leaders are on its committees. Denham was also relieved of his editorship of the *Journal of Patient Safety* and his leadership of a committee of the Leapfrog Group.

## **Conclusion**

NQF is one of the few healthcare organizations defned as consensus-based by the National Institute of Standards and Technology, part of the Department of Commerce. This status allows the federal government to rely on NQF-defned measures or healthcare practices as the best, evidence-based approaches to improving care. Because they must meet rigorous criteria, NQF's endorsed measures are trusted and used by the federal government, states, and private sector organizations to evaluate performance and share information with patients and their families.

NQF's prompt and transparent response to the Denham affair confrmed its legitimacy as standard setter. It has continued to expand its role as envisioned by Kizer to promote effective care, achieve better outcomes, improve patient safety, and improve access to care through rigorous measures and collection and analysis of data.

Working together, NQF and AHRQ became the institutional foundation that permitted patient safety to advance both as a science and in practice. They represent the ideal of a public-private partnership where collaboration, commitment, leadership, and good will produce powerful and important change.

#### **Appendix 11.1: Serious Reportable Events Steering Committee** [11]

#### **Appendix 11.2: NQF Serious Reportable Events** [11]

# **Appendix 11.3: NQF Safe Practices** [15]

# **References**

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 12 Enforcing Standards: The Joint Commission**

On March 30, 1981, Ronald Reagan, president of the USA, was shot in an assassination attempt. During his lifesaving surgery at the George Washington Hospital, the nation was riveted by the clear and calm account of its progress by the hospital's physician spokesman, Dennis O'Leary. Five years later, O'Leary became the head of the Joint Commission on Accreditation of Hospitals.

# **History of the Joint Commission1** [1]

The Joint Commission has been for many years the principal driver of healthcare quality in hospitals and, in more recent decades, other types of healthcare organizations. Its roots go back to the early years of the twentieth century when medicine was undergoing rapid changes as the result of scientifc advances in understanding the causes of diseases, the use of antisepsis, and the development of x-rays. Over a short period, hospitals move from "pest houses" to essential resources and proliferated. But quality of care varied widely. A few people began to be concerned.

<sup>1</sup>Most of what follows of the early history of The Joint Commission comes from its excellent 50-year anniversary report, *Champions of Quality in Health Care.*

One was Ernest Amory Codman, a surgeon at the MGH in Boston who developed the End Result Idea, a system for following up patients after surgery to determine their outcomes. The Idea was suffciently unpopular with his colleagues at the MGH that he found it necessary to leave and found his own hospital in 1911. He kept meticulous records and later published his results in *A Study in Hospital Effciency* [2]*.* Along the way, however, he went out of his way to publicly accuse surgeons and hospitals of being more interested in making money, for which he was widely ostracized.

At about the same time, in 1912, a prominent Chicago surgeon, Franklin H. Martin, and several others founded the American College of Surgeons (ACS). The purpose was to distinguish those who were trained in surgery from others, to establish the specialty. Their other concern was about the lamentable state of hospitals, which led them to call for "a system of standardization of hospital equipment and hospital work." Ernest Codman was tapped to chair a Hospitalization Standardization Committee.

In 1918 the ACS initiated the standardization program. In its initial foray surveying hospitals of 100 beds or more, only 89 of 692 hospitals surveyed met the minimum standards! Although that number was reported out to the college, the list of the names of the hospitals was burned to keep it from being obtained by the press! The hospital standards approval process was here to stay, however, and by 1950, half of all hospitals, 3290, were on the ACS approval list.

But the ACS was in fnancial trouble and unable to sustain the program. It approached the American Hospital Association (AHA) about taking it over. The AHA was interested and willing to provide fnancial support. The American Medical Association (AMA) got wind of this and made a counteroffer. The AMA had an accreditation program for internships and residencies and disliked the idea of a program run by administrators, not doctors. After months of wrangling, the three parties, along with the American College of Physicians (ACP), worked out their differences and in 1951 founded the Joint Commission on Accreditation of Hospitals (JCAH).

In the early years, the JCAH carried on the ACS standards. Surveys focused only on the hospital environment, what Donabedian would later call "structure." They examined physical aspects, such as proper use of autoclaves, function of clinical laboratories, medical staff organization, and patient records. An early effort to include evaluation of clinical care was shot down by the AMA, who consistently claimed this to be the prerogative of physicians. They even threatened to absorb JCAH into the AMA if they did not get their way.

The enactment of Medicare in 1965 was a watershed moment for JCAH. The legislation gave it "deemed" status, meaning that hospitals that were accredited by JCAH were deemed to have met the Medicare *Conditions of Participation*. This may have been of necessity—the federal government had no capacity to inspect hospitals and the states were notoriously poor at it—but it had immense impact and dramatically enhanced the prestige and power of JCAH as a "quasi-public" licensing body. The fip side was that it exposed it to much greater public accountability.

The JCAH board saw the new status as an opportunity to elevate the standards and use the surveys to improve quality of care. By using lessons learned from the surveys, they could move standards from "minimum essential" to "optimum achievable." They undertook a major project to elevate the standards over the next few years.

These were also the years of rising concerns about civil rights, however, and the JCAH soon found itself in the crosshairs of public interest groups that were concerned—justifably—about poor conditions in municipal hospitals in major cities. The Commission was pushed to deny accreditation to these hospitals in order to stimulate increased funding.

Not only would this be counterproductive, but closing the hospitals would deny care to indigent people, and most of these hospitals would be able to get certifcation from their state or the Department of Health, Education and Welfare (HEW). The JCAH stuck to provisional accreditation and consulting with hospitals on how to improve.

However, in 1971 the preamble in the new *Accreditation Manual for Hospitals* declared that patients were entitled to "equitable and humane treatment" and that "no person should be denied impartial access to treatment ... on the basis of such considerations as race, color, creed, national origin or the nature of the source of payment for his care." The frst patients' bill of rights. It was not just a statement. Observance of patient rights would be a factor in the determination of accreditation.

The tension between the JCAH's concept of its accreditation role as informing and stimulating hospitals to improve quality of care and the public's desire for accountability came to a head a few years later when Congress passed legislation that gave HEW (now Health and Human Services (HHS)) authority to do "validation" studies of accredited hospitals in which they would conduct surveys in response to complaints alleging noncompliance with Medicare standards and to establish standards that exceeded those of the JCAH. The Commission was being pushed to be not just an accreditor, but a certifer, i.e., a regulator. It continued to resist but in some states it did begin to survey hospitals with state licensing teams.

It seemed to work. In 1979, a General Accounting Offce (GAO) report endorsed The Joint Commission process as superior to that of HHS and later commended its cooperative relationships with states for licensure and accreditation. The Commission focused on improving its surveys.

#### **The Agenda for Change**

In 1986, the course of The Joint Commission changed dramatically when Dennis O'Leary was appointed president. O'Leary was a boardcertifed internist and hematologist. He grew up in Kansas City, graduated from Harvard and Cornell Medical School, and trained in internal medicine at the University of Minnesota and Strong Memorial Hospital in Rochester, NY. He had been at George Washington University Medical Center since 1971 where he had taken on progressively more administrative duties, including chairing the medical staff executive committee at the hospital and being the dean for clinical affairs at the medical center.

O'Leary had a clear idea of what he wanted to accomplish. He wanted The Joint Commission to change the focus of accreditation to performance improvement.

The time was ripe. In the 1960s, Avedis Donabedian at the University of Michigan had developed his classical defnition of quality of care as encompassing three components, structure, process, and outcome, but his teachings had little impact except in academe. The Joint Commission focused on structure, and practicing physicians were too busy in the 1960s and 1970s keeping up with the fast pace of scientifc advances that were transforming medicine to give much thought to how their systems worked or to analyzing their results. (Codman redux!)

But costs were increasing almost exponentially. When Medicare was introduced in 1965, it was estimated that it would cost \$275–325 million a year [3]. The actual expenditure in 1966 was \$1.8 billion, and by 1985 the expenditure was \$71 billion and seemingly out of control [4, 5]. Questions began to arise for the frst time about the effectiveness of care. What were we getting for our money?

In 1973, John Wennberg, a researcher from the Harvard Center for Community Health and Medical Care, added fuel to the fre when he published the results of his studies of geographic variation that showed two- to fourfold variations in the provision of common surgical procedures, such as tonsillectomy, hemorrhoidectomy, and prostatectomy [6]. Charles Lewis had previously shown three- to fourfold variation in the rates of performance of six surgical procedures in Kansas, including tonsillectomy, appendectomy, and herniorrhaphy [7]. These differences were not trivial, and the conclusion was inescapable: either too many patients were getting the service in one area or too few in the other. Did the doctors know what they were doing? Over the next few years, Wennberg expanded his studies to other regions, but the results were similar.

It took a while for people to take notice, but by the early 1990s, a movement was afoot in medicine to take a different approach to quality of care. Paul Batalden and Don Berwick had studied under Deming and began work on applying industrial continuous quality improvement (CQI) concepts to healthcare [8]. Berwick had founded the Institute for Healthcare Improvement (IHI) (see Chap. 6).

O'Leary and his senior colleagues Jim Roberts and Paul Schyve could see that its applications in healthcare were the future. It was time to get The Joint Commission on board. Schyve was the director of standards at The Joint Commission from 1986 to 1989 and then vice president for research and standards. He would function as The Joint Commission's quality and safety guru for the next two decades, representing it to the NQF and participating in NPSF and LLI initiatives.

There were other pressures as well: by this time The Joint Commission had 2600 standards! Hospitals and doctors wanted relief,

(a) Dennis O'Leary and (b) Paul Schyve. (All rights reserved)

and they wanted accreditation to be more relevant to their work. Why not focus on quality of care?

## **Changing Accreditation**

O'Leary and his staff proposed that the Commission change the accreditation survey focus from standards for organizational structure to standards important to the provision of quality care. A steering committee of six board members plus Paul Griner, John Wennberg, Steven Shortell, and Lincoln Moses was formed to guide them.

Steven Shortell ended up leading the reorganization of the accreditation manual to a series of chapters on clinical care functions and a series on management support functions. The hospital standards manual shrunk from 2600 standards to less than 500. The Joint Commission was now talking about performance, not structure. The new standards had clearly moved the bar to a higher level.

Meanwhile, work was begun to develop clinical indicators—discrete measures of outcomes and related processes—in selected areas such as the management of heart attack, heart failure, and pneumonia. Indicator data were then gathered from accredited hospitals and analyzed through an Indicator Measurement System. Integrating these data into the accreditation process was a challenge, however, since they were not standards against which to measure compliance, but more like a rife shot that defned a narrow signifcant goal.

The Commission also changed its name to the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) to refect its increasing scope. It was already accrediting long-term care organizations, mental health institutions, and ambulatory surgery centers, among others; it now added home care. Five public members were added to the board. In 1990, JCAHO changed its mission statement to "improve the quality of healthcare provided to the public."

Hospitals resisted the changes. External factors, such as the failed Clinton health plan and continuing increases in healthcare and survey costs, compounded the problem. The combination of anxiety over the new standards and potential public disclosure of performance led the AHA at one point to seek O'Leary's ouster as The Joint Commission president and replace him with a senior AHA staff member who had previously been The Joint Commission surveyor. However, Board leaders, headed by incoming Board chair William Kridelbaugh of the American College of Surgeons, successfully defected the assault.

In 1994 and 1995, the Commission piloted a new approach to accreditation: the Orion Project. The Orion Project tested several innovations in the survey process: changing surveys from announced to unannounced so hospitals had to be continuously ready, use of an integrated team of surveyors who were all on-site at the same time, use of laptop technology by surveyors, focus on the effectiveness of staffng, and including performance measures in the accreditation process.

After testing in several states and refning them, some of the innovations were rolled out nationwide. These were major changes in the accreditation process, but the new approach was well-received. A survey of hospitals showed that 80% found new process "more interactive, consultative, and valuable" than previous accreditation surveys. By 1995 accreditation included six new functional areas: patient rights and organizational ethics, care of patients, continuum of care, management of the environment of care, management of human resources, and surveillance and prevention of infection.

#### **Focus on Patient Safety: Sentinel Events**

Something else was happening in 1995. Starting with the death of Betsy Lehman in Boston from an overdose of chemotherapy, a number of nationally reported cases of major medical mishaps were reported in the press. These led the public and advocacy groups to raise questions about the effectiveness of The Joint Commission accreditation. If it was doing its job, how could these things happen?

O'Leary recalls that some years earlier, in 1990, I had visited The Joint Commission to suggest that, based on our experience with the medical practice study in New York, The Joint Commission should pay more attention to patient safety issues in accrediting hospitals. The discussion was cordial, but The Joint Commission had a substantial development agenda on its plate and was not anxious to add more to its agenda. However, the seed had been planted.

The patient safety shocker that spring for The Joint Commission was when it learned that a surgeon in an accredited hospital had amputated the wrong leg of a diabetic patient with severe peripheral vascular disease. Wrong-site surgery? Who had ever heard of that? Nor were there any identifable case reports in the medical literature. (It was not so rare, it turned out. In the ensuing years, The Joint Commission would typically learn of 50–75 new cases of wrong-site surgery each year.)

Reacting to this and the other major mishaps, The Joint Commission moved into action. Rick Croteau, a senior surveyor, former aerospace engineer and surgeon, was tasked with creating an entirely new policy framework for addressing what came to be known as sentinel events. His knowledge of systems thinking, analysis, and applications would stand him in good stead.

A sentinel event was defned as "a serious undesirable occurrence that results in the loss of patient life, limb, or function." A new performance improvement standard was added that required "intensive assessment of undesirable variation in performance," i.e., root cause analysis (RCA) for each sentinel event, as well as the creation of a corrective action plan. A monograph describing a thorough RCA and how to perform one was also prepared and released by The Joint Commission. Hospitals experiencing a sentinel event were placed in Conditional Accreditation until they had submitted a thorough RCA and had it approved.

In early 1996, vice president for performance measurement Jerod Loeb conceived of holding a national conference on medical error. He persuaded the American Association for the Advancement of Science (AAAS), the Annenberg Center, and the AMA to be co-conveners (Chap. 4). At the conference in the fall, the Commission announced that in an effort to be less punitive, it was changing its policy regarding sentinel events from "Conditional" Accreditation to "Accreditation Watch."

It didn't work. Although The Joint Commission leaders thought this was making their response to sentinel events less punitive, hospitals saw it as a stigma—less threatening, perhaps, but punitive, nonetheless.

By 1997, antagonism between The Joint Commission and hospitals was again in full fower. Some of this related to the formal initiation of ORYX, a next iteration of efforts to integrate clinical indicators into the accreditation process. ORYX set requirements for the number and types of standardized measures each organization had to collect and report. To ease the burden for hospitals, The Joint Commission approved the use of commercial measurement systems to collect the data, audit its reliability, and transmit it to the Commission. But the process was expensive and labor-intensive.

Adding to the friction was the Commission's decision to improve public transparency by introducing quality check on its website. Quality check was a directory of accredited organizations and their performance reports. The combination of these initiatives with the perceived threat of exposing their patient safety performance was almost too much for the hospital feld to tolerate.

Hospitals would agree with doing an RCA in principle (how could they not?), but in practice they regarded this expectation as intrusive. A more collaborative approach was needed. Reporting of sentinel events was made voluntary. Hospitals were "encouraged" to report sentinel events and to do RCAs. "Accreditation Watch" would only be assigned if a hospital failed to do an RCA for review by The Joint Commission or if it did not report a sentinel event that The Joint Commission learned of by other means, such as from a patient complaint or a story in the press.

Although their brief was with The Joint Commission, hospitals had many reasons for not wanting to report sentinel events in addition to resistance to performing RCAs. Their lawyers were concerned that despite all assurances of confdentiality, information might be revealed that would lead to a lawsuit. CEOs worried about damage to the reputation of their hospitals if events became known. In addition, if the internal environment was punitive, as most still were, the CEO might not even know an event had occurred. And, despite all the efforts of patient safety experts, many people clung to the idea that some injuries are not preventable and therefore didn't need to be reported.

#### **Sentinel Event Alerts**

Since 1995, The Joint Commission had been steadily amassing details regarding sentinel events and their related RCAs and correction plans into a "Sentinel Event Database." The Database now contained a great deal of useful information about the prevalence of serious hazards. Why not disseminate these "lessons learned" widely? To every hospital. The idea of a Sentinel Event Alert was born.

In 1998, the frst "Sentinel Event Alert" was issued, describing patient deaths resulting from accidental infusion of concentrated potassium chloride (KCl). Nurses confused its vial with that of dilute sodium chloride, which was used to "fush" intravenous lines. This rarely occurred, but when it did, it was deadly.

The Alert recommended that concentrated KCl be removed from nursing units and that its use be restricted to the pharmacy, where it could be better controlled. This was a classic example of a "forcing function," a powerful human factors technique for preventing error. It worked. Hospitals complied, and within a few years, deaths from accidental infusion of KCl virtually disappeared. Sentinel Event Alert was one of The Joint Commission's most successful initiatives.

In 1999 O'Leary invited me to speak to the Board at their annual planning retreat. I urged them to take the lead in patient safety and to do three specifc things: (1) defne and require implementation (i.e., inspect for) of known safe practices, (2) replace scheduled triennial inspections with unannounced inspections, and (3) move their reporting system to a separate entity with confdentiality protection. I have no idea whether my entreaties helped, but unannounced surveys were later implemented, and in 2001 The Joint Commission defned 11 new safe practices that they began inspecting for in January 2003.

The Commission continued to push hospitals to focus on quality improvement, but in 1999, the IOM report, *To Err Is Human*, put the spotlight on safety. In that year, The Joint Commission rolled out a major new chapter on patient safety for its accreditation standards manual and changed its mission statement to ". . . continuously improve the *safety and* quality of care provided to the public." However, it needed to do more. It needed to motivate hospitals to implement safe practices.

#### **Patient Safety Goals**

Sentinel Event Alerts gave hospitals an incentive to deal with patient safety issues, but many did not know how. Removing concentrated KCl from nursing units was simple: issue a decree (although it is interesting that some hospitals didn't.) But few hazards could be dealt with that easily; most required changing a process. Thanks to IHI and others, some hospitals were beginning to learn how to do this, but it was diffcult work, and most were still not engaged. By this time, however, the National Quality Forum (NQF) was identifying evidencebased safe practices that hospitals could adopt. Hospitals wouldn't have to reinvent the wheel; they would just have to put it on.

The Joint Commission Sentinel Event Database was the treasure trove of information that made the Alerts possible. Why not take the next step and recommend safe practices to address the high-priority problems? In addition to The Joint Commission, the NQF, IHI, and others were developing safe practices.

In 2002, the Commission established a Sentinel Event Alert Advisory Group of nurses, physicians, pharmacists, risk managers, and engineers to formalize the work already being done to identify patient safety issues based on lessons learned from sentinel events the Alerts—as well as from accreditation surveys and recommendations from the NQF and other safety organizations. The Advisory Group analyzed potential remedies for practicality, cost-effectiveness, and evidence.

From this analysis, Patient Safety Goals were developed. Each goal had one or more specifc recommendations—the practice changes to be implemented. Box 12.1 shows one of the early goals, reporting critical test results, that was informed by our work at the Massachusetts Coalition for the Prevention of Medical Error (Chap. 8) (Box 12.1).

#### **Box 12.1 Patient Safety Goal 2**

#### **Goal 2**

Improve the effectiveness of communication among caregivers.

#### **NPSG.02.03.01**

Report critical results of tests and diagnostic procedures on a timely basis.

#### **--Rationale for NPSG.02.03.01--**

Critical results of tests and diagnostic procedures fall signifcantly outside the normal range and may indicate a life-threatening situation. The objective is to provide the responsible licensed caregiver these results within an established time frame so that the patient can be promptly treated.

#### **Elements of Performance for NPSG.02.03.01**

Reprinted by permission of the Joint Commission Resources. All rights reserved

Note that these were goals, not required practices. In line with the commission's commitment to encourage voluntary improvement, the goals were to be aspirational statements of intent for organizations to pursue. Recommendations, not requirements. A hospital's progress in implementing them would, however, be evaluated during accreditation surveys, and performance expectations that followed each goal were surveyed and scored. Compliance data were aggregated and periodically published in *Joint Commission Perspectives*.

The frst set of six Patient Safety Goals was published in January 2003:

Each goal had two or more specifc recommendations. More detailed processes for achieving the Goals were available for each safe practice from other sources such as IHI and the Massachusetts Coalition. At the time they were released, Rick Croteau noted: "These six Joint Commission National Patient Safety Goals and recommendations provide a clearly defned, practical, and achievable approach to addressing…the most critical threats to patient safety." [9]

The Advisory Group continually reviews the goals. It adds new goals annually and retires old ones as high compliance rates are achieved. In 2004, Goals 1b (time-outs) and 4 were consolidated into a new "Universal Protocol" for the prevention of wrong-site, wrong-person, wrongprocedure surgery. A seventh goal was added to address healthcareacquired infections, which included complying with CDC hand hygiene guidelines. By 2008, 16 goals had been issued, including the frst call to involve patients in their care: Goal 13, Encourage Patients' Active Involvement in Their Own Care as a Patient Safety Strategy.

The Goals were well accepted by hospitals. Follow-up data from surveys in 2005 showed high rates of implementation for a number of specifc recommendations: 95% use of two identifers for patient identifcation, 82% use of time-outs, 90% implementation of critical test result procedures, 99% removal of KCl, and 96% use of a wrongsite checklist. They have been an effective mechanism for motivating hospitals to improve patient safety.

#### **Core Measures**

By 2000, The Joint Commission, working with expert panels, had expanded its original core measure sets for acute myocardial infarction (AMI), heart failure (HF), and pneumonia to a total of 14 individual measures. Box 12.2 shows the core measures for AMI. Hospitals began collecting measures July 1, 2002. (Box 12.2)

Responding to the pleas from hospitals to reduce duplication of effort, the Commission worked with the Centers for Medicare and Medicaid Services (CMS) to create one common set known as the Specifcations Manual for National Hospital Inpatient Quality Measures to be used by both organizations.

The measures worked. From 2002 to 2005, hospitals adherence improved: AMI 87 to 90%, pneumonia 72 to 81%, and CHF 60 to 76%. Some improvements were dramatic. For example, hospitals provided smoking cessation advice to 92.1 percent of patients in 2005 compared with 66.6 percent in 2002. More importantly, outcomes improved: the inpatient mortality rate for heart attack patients declined from 9.2% in 2002 to 8.5% in 2005, representing thousands of lives saved [10].

Currently, The Joint Commission's ORYX initiative integrates performance measurement data into the accreditation process. The measures are aligned as closely as possible with the Centers for Medicare and Medicaid Services (CMS), and chart-abstracted data are publicly reported on The Joint Commission's *Quality Check*® website.

#### **Public Policy Initiative**

In 2001, The Joint Commission launched a set of public policy initiatives to amplify its patient safety and quality improvement messaging. It convened a series of topic-oriented roundtables to frame relevant discussions and develop recommendations. Each roundtable had 30–45 participants and met on 2–3 occasions. The Joint Commission staff used the input to draft a white paper. Each white paper was eventually 35–50 pages long and contained fndings, recommendations, and accountabilities for seeing each recommendation through to fruition.

The frst paper on the nurse staffng crisis struck a chord in the hospital, nursing, and patient safety communities and was an immediate success, eventually being downloaded from The Joint Commission website almost two million times. The report of a roundtable on patient safety and tort reform was also a hit and was downloaded over 300,000 times. A number of other roundtables were convened in the next few years.

#### **Accreditation Process Improvement**

In 2000, The Joint Commission decided it had to do more to improve its accreditation process. The triennial inspections had been a tremendous burden for hospitals. For several weeks beforehand, all work other than patient care stopped as hospital departments got their records in shape for the survey. Everyone dreaded them. Worse, they often failed to identify some serious performance problems.

O'Leary asked Russ Massaro, a seasoned surveyor who knew how the minds of healthcare organization leaders worked, to rethink the process. The result was the Shared Visions-New Pathways initiative that was launched in 2004. It completely changed the accreditation process in three fundamental ways.

First, organizations were asked to periodically do their own indepth self-assessments and share the results and their plans for improvement with The Joint Commission. From now on, surveys would concentrate on the fndings from the self-assessments and hospital-reported data, such as from ORYX and sentinel events.

Second, and this was a biggie, on-site surveys from here on would be unannounced. Gone was the dreaded triannual ritual of stopping work to get ready for The Joint Commission. You would have to be at the top of your game at all times. Unannounced meant fnding out that the survey would be tomorrow.

Finally, instead of focusing on records, the on-site reviews would focus on the care actually provided to patients. They would use a tracer methodology in which the care to individual patients in the hospital at the time was evaluated by observing and interviewing the patients and hospital staff in real time. Patient care was reviewed for relevant standards compliance, such as medication management and nurse staffng.

It was a huge change. And it was very successful. The reviews engaged doctors, nurses, and other frontline staff. Suddenly, surveys made sense to them, while also giving them an opportunity to demonstrate their skills. Hospitals welcomed the new accreditation process. This was now continuous engagement with The Joint Commission toward the mutual goal of continuous improvement.

#### **Conclusion**

What do we make of all this? First and foremost, The Joint Commission has clearly been a leader in reducing harm in healthcare. Without its infuence—and persistence—we would not have made the progress we have made in many areas and likely not made any at all in others. Deciding what to do for patient safety and how to do it has been an incredibly complicated business, and the Commission has navigated that thicket well.

From the beginning, the patient safety movement has had to confront the tension between those who call for the greater accountability and regulation that has worked so well in other hazardous industries such as transportation and nuclear power and those who believe that making real change is voluntary and that our job is to motivate people and provide them with the tools to do it.

Your author has found himself squarely in the middle of this debate, embracing the need to change the culture and helping to teach professionals to change systems, but also of the mind that we need to do much more to hold the leaders of healthcare organizations publicly accountable for failure to prevent harm, particularly serious harm where the methods are known, such as serious reportable events.

The Commission has also found itself in the middle, and over the years it has experimented with one approach and another regarding reporting of sentinel events, collection and reporting of data, and how to respond to failures revealed by accreditation visits. All compounded by the ambiguities and vicissitudes that result from the fact that participation by hospitals is voluntary and from changes in views about whether accreditation suffces for deemed status or state licensing.

Compounded also by pushback on all sides: consumer groups who want tougher oversight, hospitals and doctors who want less, and Congress who doesn't know what it wants or changes what it wants according to external pressures. To say, "You can't please everyone" is an understatement at best.

Despite all this, The Joint Commission has been a well-spring of innovations, a great many of which have measurably reduced harm and improved quality of care. More than any other organization, public or private, it has consistently pursued a data-driven analytic approach to helping hospitals improve care. We can all sleep better knowing that it will continue to be a major force for improving patient safety in the future.

#### **References**

for medical care for the aged: part 2. Washington: U.S. Government Printing Offce; 1965.

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 13 Partners in Progress: Patient Safety in the UK**

In 1997, Britons were shocked by a report from the General Medical Council (GMC) of a series of deaths from bungled surgery at the Bristol Royal Infrmary. In response to parents' complaints, the GMC had launched an investigation into the high mortality of cardiac surgery of children at the Infrmary. It found that of 53 children who were operated on, 29 had died and 4 suffered severe brain damage. Three surgeons were found guilty of serious professional misconduct, and two were stricken from the medical register [1].

The public and the profession were shocked that this could happen in the National Health Service (NHS). Richard Smith, editor of the BMJ, wrote, "All changed, changed utterly. British medicine will be transformed by the Bristol case." [2]

Transforming the NHS was already on the mind of Tony Blair and the Labour Party when they took over the government that same year. Britons were unhappy with the quality of care in the NHS, especially long wait times. A recent OECD report had shown that the UK was underperforming its competitors. Blair made improving quality of care and increased funding of the NHS a keystone of his campaign. The Bristol case added fuel to his fre.

#### **A National Commitment**

One of Blair's frst acts was to appoint Liam Donaldson chief medical offcer (CMO) in 1998. Donaldson was a surgeon who had retrained in public health. He had been CEO of the country's northern regional authority, where he received occasional notifcations from hospitals of "accidents" in which patients had been harmed or died from a complication of medical care. As he read them, he recognized that the purpose of the reports was not learning from the incident but to cover the hospitals' leaders backs if the case became public.

These hospital reports were eye-opening for Donaldson, who was previously only dimly aware of the problem. Like your author, another surgeon converted to public health a few years earlier, he began to read about accidents in other industries. He discovered the work of James Reason and human factors experts and became excited about applying lessons from industry to medicine. He asked local managers to report all incidents and set up a database to learn more. He began to develop a list of safety measures that needed to be implemented.

When Blair was elected, Donaldson shared this information and his ideas with him. Shocked, Blair and his new health ministers also

Liam Donaldson. (All rights reserved)

recognized that this was just what he needed politically. He appointed Donaldson as CMO and gave him his full support.

Although the government commitment was new, interest in patient safety in the UK went back at least to 1985 when Charles Vincent began work on avoidable mishaps in medicine, which he published in 1986. Vincent later wrote an editorial in BMJ in 1989 about systems changes [3], followed by a book that he edited with Maeve Ennis and Bob Audley, *Medical Accidents*, in 1993, in which he introduced the concept of applying human factors principles and systems analysis to healthcare [4].

Vincent referenced the fndings of the Harvard Medical Practice Study and called for more research into the causes of medical accidents and the development of a comprehensive safety program. Two years later, in 1995, he further expanded these ideas in *Clinical Risk Management* [5]*.* Risk managers began to think about patient safety, but, as in the USA, the medical profession in general had little interest in these developments. They were fxated on the problem of malpractice litigation and worried that investigation of errors would expose them to more risk. Not much happened.

#### **The Patient Safety Movement**

Donaldson would change that. In early 2000, within months of the publication of the IOM report in the USA, he launched the patient safety movement in the UK by releasing his own report, *An Organisation with a Memory* [6]*.*

The report was the product of a panel with a wide-ranging set of disciplines and expertise, including people from other industries. It coupled a comprehensive analysis of the quality and safety problems in the NHS with strong recommendations about what needed to be done. Based on another study done by Charles Vincent [7], it estimated that 11% of hospitalized Britons suffered an adverse event each year, at a cost of one billion pounds for additional hospital stays alone.

*An Organisation with a Memory* echoed the IOM report in condemning the typical approach when things go wrong of blaming the individual and made the case for a systems approach. The report cited the organizational culture and the lack of an effective reporting system as major barriers. It also acknowledged that some of the existing systems in the UK did work well, especially the confdential inquiries and reporting systems for medical devices.

*An Organisation with a Memory* called for a fundamental rethinking of the way the NHS approached the challenge of learning from adverse events (AEs). It called for four specifc changes: a unifed mechanism for reporting and analysis of AE, a more open culture where people can safely report and discuss errors, mechanisms for putting recommended changes in place, and wider appreciation of the systems approach.

About this time, at the urging of Donaldson and BMJ editor Richard Smith, the British Medical Association and the NHS hosted the frst UK national symposium on medical error. It was timed to coincide with the publication of a special BMJ issue on patient safety that Smith had conceived of a year earlier and Don Berwick and I edited (Chap. 17). The conference drew a wide audience from Britain and European countries. For many, this was their introduction to the problem of medical errors. Beth Lilja, later the driving force behind the Danish patient safety movement, told me that it was a defning moment for her.

Donaldson came to the CMO job with a passion and an agenda. He recognized that what he had in mind for patient safety would not succeed if it were absorbed into the NHS bureaucracy. The effort needed a full-time commitment and independence. He persuaded the Department of Health to establish an independent Special Health Authority, the National Patient Safety Agency (NPSA). Its mission was to improve patient safety by "encouraging voluntary reporting of medical errors, conducting analysis and initiating preventative measures." [8]

It is worth noting to American readers that the British response to the error threat refected the ability of the NHS to make changes rapidly and to implement them nationwide. It stands in stark contrast to the USA, where Congress has long been resistant to all forms of regulation and grudging in its support of safety.

#### **The National Patient Safety Agency (NPSA)**

The NPSA was designed to identify problems and recommended solutions, not to implement them. There were two major divisions: reporting and solutions. Two experienced administrators, Sue Osborn and Sue Williams, the "two Sues," were given joint responsibility for leading the new agency.

The frst priority was to establish the National Reporting and Learning System (NRLS). Like many others, Donaldson believed that a national reporting system was essential to making progress in patient safety. Reporting and analysis could not only lead to increased awareness but also produce actionable recommendations. Getting the NRLS going was a massive information technology challenge that dominated everything else at NPSA.

The NPSA declared its objective was "to promote an open and fair culture in hospitals and across the health service, encouraging doctors and other staff to report incidents and 'near misses'." It was made clear that the purpose of reporting was to enable healthcare providers to learn lessons from each other in order to improve safety—not to identify individuals to punish [9].

The system was set up to receive and analyze reports from all sources (including, later, the public) and to recommend changes. Reporting of adverse events was already well-established in the UK. Hospitals were required to report adverse events to their regional authorities. The NPSA required the reports to also be sent to a central authority, the NRLS, so that the whole NHS could learn from them. The system soon received hundreds of thousands reports, later over a million, annually.

Its success proved to be its undoing. The huge volume and the logistics of categorizing reports impaired its ability to do meaningful analysis as Donaldson had hoped. What the system could do, however, was identify problems requiring action. This information was passed on to the solutions division that then issued alerts with recommendations for implementation of safe practices. NPSA later established a national register to which hospitals were required to report what they did in response to the alerts and why.