Developing and Evaluating an Automated All-Cause Harm Trigger System

Open AccessPublished:February 16, 2017DOI:https://doi.org/10.1016/j.jcjq.2017.01.004

      Background

      From 2009 through 2012, the Adventist Health System Patient Safety Organization (AHS PSO) used the Global Trigger Tool method for harm identification and demonstrated harm reduction. Although the awareness of harm demonstrated opportunities for improvement across the system, leaders determined that the human and fiscal resources required to continue with a retrospective manual harm identification process were unsustainable. In addition, there was growing concern that the identification of harm after the patient's discharge did not allow for intervention during the hospital stay. Therefore, the AHS PSO decided to seek an alternative method for patient harm identification.

      Methods

      The AHS PSO and another PSO jointly developed a novel automated all-cause harm trigger identification system that allowed for real-time bedside intervention, real-time trend analysis affecting patient safety, and continued learning about harm measurement. A sociotechnical approach of people, process, and technology was used at two pilot hospitals sharing the same electronic health record platform. Automated positive harm triggers and work-flow models were developed and evaluated.

      Results

      Combined data from the two hospitals in a period of 11 consecutive months indicated (1) a total of 2,696 harms (combined hospital-acquired and outside-acquired); (2) that hypoglycemia (blood glucose ≤ 40 mg/dL) was the most frequently identified harm; (3) 256 harms related to the Patient Safety Indicator 90 (PSI 90) Composite descriptions versus 77 harms reported to regulatory harm reduction programs; and (4) that almost one third (32%) of total harms were classified as outside-acquired.

      Conclusion

      The automated harm trigger system revealed not only more harm but a broader scope of harm and led to a deeper understanding of patient safety vulnerabilities.
      Health care leaders hold the Institute of Medicine (IOM) reports, To Err Is Human: Building A Safer Health System
      • Institute of Medicine
      To Err Is Human: Building a Safer Health System.
      and Crossing the Quality Chasm: A New Health System for the 21st Century,
      • Institute of Medicine
      Crossing the Quality Chasm: A New Health System for the 21st Century.
      as seminal work in patient safety. These reports addressed the high incidence of patient harms occurring in hospitals and were a catalyst for the passage of the Patient Safety and Quality Improvement Act of 2005. Subsequently, the US Department of Health and Human Services Office of Inspector General (OIG) released a series of reports that established the critical issue of the underreporting of harms occurring in hospitals. This prompted another report by the OIG in 2010, Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries,
      • US Department of Health and Human Services
      Office of Inspector General. Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries.
      which recommended that the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS) encourage internal hospital reporting of all harms—whether considered a complication, a preventable harm, or a harm caused by system failures or errors (all-cause harm). The OIG proposed that hospitals should strive to report all-cause harm—“any event during the care process that results in harm to a patient, regardless of the cause”(p. 2)—rather than limiting efforts toward the narrow subset of harms found in regulatory harm reduction programs such as Value-Based Purchasing
      • QualityNet
      Hospital Value-Based Purchasing (VBP) Mortality and AHRQ PSI 90 Composite Measures Overview.
      and Hospital-Acquired Conditions from CMS, and Patient Safety Indicator 90 (PSI 90)
      • QualityNet
      Hospital Value-Based Purchasing (VBP) Mortality and AHRQ PSI 90 Composite Measures Overview.
      from AHRQ. A previous effort for harm measurement focused on broader definitions of harm reported in the Harvard Medical Practice Study,
      • Brennan T.A.
      • et al.
      Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I.
      which was later refined by the Institute for Healthcare Improvement (IHI) Global Trigger Tool (GTT) for Measuring Adverse Events. The GTT method focused on all-cause harm.
      • Griffin F.A.
      • Resar R.K.
      IHI Global Trigger Tool for Measuring Adverse Events, 2nd ed. IHI Innovation Series white paper.
      The GTT harm measurement method involved a retrospective manual review of inpatient records using triggers (discrete data elements) suggesting the possible presence of harm or potential harm. The use of triggers, along with a precisely defined health record review process, allowed for a more comprehensive and reliable approach to harm identification than did the traditional methods of voluntary incident reporting and observational tracking. Hospitals and health care systems that utilized trigger methodologies found up to 40% of adult,
      • Classen D.C.
      • et al.
      Computerized surveillance of adverse drug events in hospital patients.
      • Landrigan C.P.
      • et al.
      Temporal trends in rates of patient harm resulting from medical care.
      • US Department of Health and Human Services
      Office of Inspector General. Adverse Events in Hospitals: Methods for Identifying Events.
      • Classen D.C.
      • et al.
      “Global trigger tool” shows that adverse events in hospitals may be ten times greater than previously measured.
      • Good V.S.
      • et al.
      Large-scale deployment of the global trigger tool across a large hospital system: refinements for the characterisation of adverse events to support patient safety learning opportunities.
      • Garrett Jr, P.R.
      • et al.
      Developing and implementing a standardized process for global trigger tool application across a large health system.
      • Kennerly D.A.
      • et al.
      Characterization of adverse events detected in a large health care delivery system using an enhanced global trigger tool over a five-year interval.
      • Classen D.C.
      • et al.
      Description of a computerized adverse drug event monitor using a hospital information system.
      pediatric,
      • Sharek P.J.
      • et al.
      Adverse events in the neonatal intensive care unit: development, testing, and findings of an NICU-focused trigger tool to identify harm in North American NICUs.
      • Takata G.S.
      • et al.
      Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals.
      • Kirkendall E.S.
      • et al.
      Measuring adverse events and levels of harm in pediatric inpatients with the Global Trigger Tool.
      • Stockwell D.C.
      • et al.
      Automated adverse event detection collaborative: electronic adverse event identification, classification, and corrective actions across academic pediatric institutions.
      • Stockwell D.C.
      • et al.
      A trigger tool to detect harm in pediatric inpatient settings.
      and skilled nursing facility populations
      • US Department of Health and Human Services
      Office of Inspector General. Adverse Events in Skilled Nursing Facilities: National Incidence Among Medicare Beneficiaries.
      had experienced at least one harm while under hospital or skilled nursing care, illustrating the vast scope of inpatient harm and the need for continued harm reduction initiatives. Furthermore, in 2011, the IOM issued the Health IT and Patient Safety: Building Safer Systems for Better Care report that emphasized the importance of automated real-time harm detection systems to supplement or possibly replace manual record review, which could allow clinicians to intervene immediately to prevent or mitigate imminent harm.
      • Institute of Medicine
      Health IT and Patient Safety: Building Safer Systems for Better Care.
      Interestingly, 20 years prior to the IOM report, researchers had identified a method for using automated triggers to identify patient harm, but this research went relatively unnoticed.
      • Classen D.C.
      • et al.
      Computerized surveillance of adverse drug events in hospital patients.
      • Classen D.C.
      • et al.
      Description of a computerized adverse drug event monitor using a hospital information system.
      Recently, a study suggested that preventable harm may lead to as many as 440,000 deaths and more than 6 million injuries per year,
      • James J.T.
      A new, evidence-based estimate of patient harms associated with hospital care.
      which highlights the potential usefulness of all-cause harm detection systems.
      From 2009 through 2012, the Adventist Health System Patient Safety Organization (AHS PSO) successfully used the GTT method for harm identification and demonstrated harm reduction. The major harm findings drove systemwide collaborative improvement projects.
      • Garrett Jr, P.R.
      • et al.
      Developing and implementing a standardized process for global trigger tool application across a large health system.
      Although the awareness of harm, provided by the retrospective manual review method, demonstrated opportunities for improvement across the system, leaders determined that the human and fiscal resources required to continue with a retrospective manual harm identification process were unsustainable. In addition, there was growing concern that the identification of harm after the patient's discharge did not allow for intervention during the hospital stay. Therefore, the AHS PSO decided to seek an alternative method for patient harm identification. The objective of this paper is to demonstrate how two PSOs developed a novel automated all-cause harm trigger identification system that allowed for real-time bedside intervention, real-time trend analysis affecting patient safety, and continued learning about harm measurement.

      Methods

       Development of the Automated Harm Trigger System

      The AHS PSO and Pascal Metrics, Inc. (a PSO dedicated to patient safety and high reliability of care), partnered to develop a proof of concept using Risk Trigger® Monitoring (RTM), an automated Web-based harm trigger decision support solution. The RTM platform allowed for an interactive environment where all users shared potentially sensitive safety data that were protected by the PSOs. The PSOs ensured freedom from discovery, allowing for the ability to learn from and bolster a well-informed safety performance improvement program. We piloted this novel decision support solution at two large urban hospitals, using the sociotechnical approach that recognizes the interaction between people, process, and technology in the workplace
      and is reflected in this article.

       The Technology

       Trigger Development

      Trigger development began in 2012 by building a set of automated triggers. Trigger criteria and algorithms were developed and customized for the pilot hospital's health information technology system. The trigger definitions we used were very similar to the IHI GTT definitions, and we automated those triggers that used discrete data readily available in the electronic health record (EHR). We found that some GTT triggers could not be automated because the documentation was on paper or entered as free text within the EHR. In addition, we did not automate the GTT trigger diphenhydramine administration because it was often used therapeutically and not always associated with patient harm. Between 2012 and 2014, we continued to refine the automated triggers based on expert clinical opinion from both PSOs because there was not a gold standard for this novel approach. The refinement process was done by observing the relationship of firing frequency to harm identification and the severity of harm identified. For example, the fall trigger logic was modified to improve the firing frequency and increase the positive predictive value (PPV). The trigger initially was written to look for the date of fall. We discovered that the date of fall element included both the current fall date and the historical fall date, which occurred prior to hospital admission. The logic was updated to fire only if the fall date occurred after the patient was admitted to the hospital. This change resulted in a more specific trigger with a PPV improvement from 1% to 14%.
      Following the pilot phase, we continued the trigger and refinement process and found that harm triggers associated with Clostridium difficile–positive stool, hypoglycemia, and 3rd and 4th degree vaginal lacerations had the greatest PPV (range, 44%–91%). Triggers associated with coagulation disturbances, Carbapenem-resistant Enterobacteriaceae culture, digoxin overdose, and pressure ulcers also had good PPV (range, 25%–40%).

       Trigger Transmission

      In 2013 we created an electronic flat file extract of clinical data from the EHR (demographic data, bed movement, laboratory results, radiology orders, surgery times, and nursing assessments) that was securely transmitted to the RTM platform daily and piloted at one hospital. We found that the automated trigger system, populated with the EHR flat file extract, demonstrated that it was possible to automate harm triggers. In early 2014, with the addition of a second hospital, it was determined that utilizing Health Level Seven® (HL7®) messages, a real-time stream of data from the EHR, would provide a more reliable and timely data feed. In addition, HL7 data streams allowed us to broaden the types of triggers and events that the system could monitor because the hospitals had existing outbound HL7 messages for most major systems such as admission, discharge, transfer; laboratory; medication orders; and radiology. Because we could not develop HL7 data streams from nursing flow sheets, electronic medicationadministration records, and surgical procedure documentation systems, we developed hourly flat file extracts to mimic real-time transmission. Utilizing HL7 data feeds augmented by the hourly flat file extracts allowed for a more robust and real-time data feed. Both hospitals went live in mid-2014, in a production environment, with a basic set of 41 automated triggers.

       The People and the Process: Harm Identification

      Concurrent with trigger development, the AHS PSO team of registered nurses and physician authenticators, experienced in harm identification using the GTT manual process,
      • Garrett Jr, P.R.
      • et al.
      Developing and implementing a standardized process for global trigger tool application across a large health system.
      established standard operating procedures for using the RTM solution in tandem with the EHR. The team referenced the IHI definition of harm—“unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalization, or that results in death,”
      • Griffin F.A.
      • Resar R.K.
      IHI Global Trigger Tool for Measuring Adverse Events, 2nd ed. IHI Innovation Series white paper.
      (p. 5) and identified only harms of commission. Definitions and descriptions of harm were developed for each positive trigger in the AHS PSO Automated Risk Trigger Tool Guideline for Record Review. Identified harms were categorized according to the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index for Categorizing Medication Errors.
      • National Coordinating Council for Medication Error Reporting and Prevention
      NCC MERP Index for Categorizing Medication Errors.
      The patient population consisted of inpatients 18 years of age or older and excluded psychiatric, rehabilitation, and hospice admissions.
      A centralized nurse review team model for work flow was developed (Figure 1). During the review process, the nurse reviewers used dual computer monitors—one monitor to access the list of patients with positive triggers from the Web-based RTM solution, and a second monitor to view thepatient's open EHR. The nurse reviewer followed the automated positive trigger in the EHR and determined if a patient harm had occurred. This process averaged approximately five minutes per trigger. If a harm was identified, it was classified as hospital-acquired or outside-acquired and was grouped into one of five harm categories (medication, patient care, surgery or procedure, perinatal, and health care–associated infection) and assigned a severity level (E–I) according to the NCC MERP
      • National Coordinating Council for Medication Error Reporting and Prevention
      NCC MERP Index for Categorizing Medication Errors.
      (Sidebar 1). We captured severity level E harm because from the patient's perspective the harm was an unexpected event that required intervention. In addition, Adler et al. found that level E harms were positively correlated with total cost, variable cost, length of stay, and 30-day readmissions.

      Adler L., et al. Impact of inpatient harms on hospital finances and patient clinical outcomes. J Patient Saf. Epub 2015 Mar 23.

      While following the trigger, the experienced nurse reviewer sometimes found harm that was not associated with an automated trigger, and it was documented as a harm. All harms were confirmed and approved after discharge by a physician or a registered nurse. If no harm was found, the review ended. When potential harms were identified, the nurse reviewer added a communication note into the RTM platform which was viewable at the hospital level. These notifications provided the hospital care team with information regarding potential harms that could require bedside intervention, as described in a case study (Sidebar 2).
      Figure 1
      Figure 1The daily work flow was used by the centralized nurse reviewers, hospital intervention coordinators, and hospital care team. The centralized nurse reviewer accessed the list of patients with positive triggers from the Risk Trigger® Monitoring (RTM) solution and followed the automated positive trigger in the patient's EHR to determine if a harm occurred. If no harm was found, the review ended. Harms or potential harms were identified, confirmed, and categorized upon patient discharge. Positive triggers, harms, and potential harms were viewable at the hospital level and communicated via the RTM platform to the hospital intervention coordinator. The hospital intervention coordinator communicated with the hospital care team, which provided intervention or action as appropriate.
      Harm Severity Ratings*
      The National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index for Categorizing Errors to define adverse events as follows:
      • Category E = temporary harm and required intervention;
      • Category F = temporary harm and required initial or prolonged hospitalization;
      • Category G = permanent harm;
      • Category H = intervention required to sustain life;
      • Category I = death
      *Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events, 2nd ed. IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement, 2009. (Available at http://www.ihi.org)
      Case Study
      Identifying inpatient harm or potential harm in real time provides the hospital care team with multiple triggers to analyze patient data quickly and supports informed care decisions at the bedside.
      The centralized review team received an automated positive harm trigger patient list. For one patient, multiple harm triggers fired consisting of (1) consecutive blood glucose levels of 180 mg/dL within 12 hours, (2) glucose greater than 250 mg/dL, and (3) lactate level > 4.0 mL. A review of the electronic health record (EHR) revealed that a 70-year-old woman with a history of obstructive pulmonary disease had been admitted to a medical/surgical unit via the emergency department with increased dyspnea, wheezing, and possible pneumonia. Due to the group of multiple harm triggers fired and review of the patient's condition as recorded in the EHR, the centralized review team recognized that this patient, despite a normal temperature and blood pressure, was at high risk for sepsis. A potential for sepsis harm was recorded in the RTM platform. At the hospital, the intervention coordinator viewed the multiple harm triggers and the potential for sepsis harm notation. A review of the emergency department record showed that sepsis surveillance had been initiated. The combination of automated triggers and the potential for sepsis notation led the care team to promptly conduct a physical assessment revealing that the patient was in moderate distress with increased lethargy. The patient was subsequently transferred to intensive care for treatment of pneumonia, sepsis, uncontrolled hyperglycemia, and volume depletion. The patient recovered and was discharged home in stable condition four days later.
      We developed a work flow for the hospital intervention coordinator who had the capability to view the list of inpatients with positive triggers, as well as harms, potential harms, and communication notes recorded by the centralized team. The intervention coordinator communicated with the hospital care team, which provided intervention or action as appropriate. Regular collaborative calls between both PSOs and the hospital care teams were established to share ideas, issues, and ongoing improvements.

      Results

      First, the automated review process captured more harms and a wider array of harms. We found a total of 2,696 harms, hospital-acquired and outside-acquired, using the automated trigger review process, as compared to 132 harms found using a manual review process for similar time periods (Table 1, Figure 2, Figure 3).
      Table 1Number (Percentage) of Hospital- and Outside-Acquired Harm Types Identified by Automated Risk Trigger and Global Trigger Tool (GTT) Manual Methods
      Harm Types Identified (Hospital-Acquired and Outside-Acquired)Automated Risk Trigger Method # of Harms (Period of 11 Consecutive Months
      Data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      ; 40,592 Records Reviewed)
      GTT Manual Method # of Harms (Period of Same 11 Consecutive Months, Two Years Prior
      Data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      ; 440 Records Reviewed)
      Hospital-Acquired% of Total # of HarmsOutside-Acquired% of Total # of HarmsHospital-Acquired% of Total # of HarmsOutside-Acquired% of Total # of Harms
      Health Care–Associated Infection Harms
      Pneumonia (non-ventilator associated)5814955
      Clostridium difficile medication-associated infection1459852
      Other health care–associated infection7300
      Total21011%25029%1010%723%
      Medication-Related Harms
      Medication-related bleeding129245133
      Medication-related glycemic events1774700
      Medication-related delirium, confusion, or oversedation731131
      Chemotherapy-related113501
      Medication-related acute renal insufficiency/renal injury23800
      Medication-related allergic reaction201000
      Medication-related nausea and vomiting25100
      Contrast dye–related acute renal injury18000
      Other medication-related harms5221172
      Total52829%37844%3332%723%
      Patient Care–Related Harms
      Venous thromboembolism1303200
      Stage I or II pressure ulcer983596
      Fall with injury875822
      Stage III or IV pressure ulcer73100
      Catheter-related urinary retention20000
      Other patient care–related harms572502
      Total39922%18121%1111%1034%
      Perinatal Harms
      3rd- or 4th-degree lacerations93000
      Maternal tissue trauma due to instrument delivery40000
      Retained placenta or tissue5100
      Postpartum hematoma, cervical laceration, bleeding harms2000
      Total1408%10%00%00%
      Surgery-Related Harms
      Respiratory complications related to surgery or procedure994161
      Postoperative/Postprocedure wound infection4012110
      Abnormal bleeding/Blood loss/Hematoma following surgery70970
      Blood clots and other occlusions related to surgery51320
      Postoperative/Postprocedure ileus46000
      Postoperative/Postprocedure acute renal failure26100
      Postoperative/Postprocedure urinary retention22020
      Removal, injury, or repair of organ20420
      Cardiac complications related to surgery or procedure26100
      Surgical site infection22400
      Other surgery-related harms1341585
      Subtotal55630%536%4847%620%
      Total183368%86332%10277%3023%
      Automated Risk Trigger Method # of Harms (Hospital-Acquired+Outside-Acquired)GTT Manual Method # of Harms (Hospital-Acquired+Outside-Acquired)
      Grand Total2,696132
      IV, intravenous.
      * Data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      Figure 2
      Figure 2The automated risk trigger method captured significantly more hospital-acquired harms in all five harm categories than the Global Trigger Tool (GTT) manual method for similar time periods. For both methods, the data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      Figure 3
      Figure 3The automated risk trigger method captured significantly more outside-acquired harms in all five harm categories than the Global Trigger Tool (GTT) manual method for similar time periods. For both methods, the data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      Second, we found the 10 most frequently identified hospital-acquired harms and corresponding severity levels (Table 2). Hypoglycemia (blood glucose ≤ 40 mg/dL) related to the use of insulin or oral hypoglycemic agents was the most frequently identified harm. In addition, bleeding related to medication, and oversedation related to medication were consistently found in the top 10 harms; none of which were included in a regulatory harm reduction program.
      Table 2Ten Most Frequently Indentified Hospital-Acquired Harms by Severity Level (Period of 11 Consecutive Months)
      Data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      EFGHITotal
      1Hypoglycemia related to medication15110250177
      2Clostridium difficile infection10341001145
      3Bleeding related to medication4881203134
      4Venous thromboembolism6961000130
      5Respiratory complications related to surgery or procedure865021599
      6Stage 1 or II pressure ulcer98000098
      7Fall with injury82230087
      83rd or 4th degree laceration during vaginal delivery78000078
      9Oversedation related to medication537013073
      10Other surgery- or procedure-related event55425470
      Total695312764131,091
      * Data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      Third, we compared the number of harms found using the automated trigger review process to the number of harms reported to the PSI 90 regulatory program, which, at the time of this study, included eight PSIs. We found 256 harms related to the PSI 90 Composite descriptions, whereas only 77 harms were reported to CMS for the same time frame (Table 3).
      Table 3Comparison of Hospital-Acquired Harms Using an Automated Trigger Methodolgy vs. Reported Patient Safety Indicator 90 (PSI 90) Harms (Period of 11 Consecutive Months)
      Data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      PSIMeasureAutomated Method # of HarmsPSI 90 Reported # of Harms
      PSI 03Pressure Ulcer1120
      PSI 06Iatrogenic Pneumothorax117
      PSI 07Central Venous Catheter-Related Blood Stream Infection43
      PSI 08Postoperative Hip Fracture11
      PSI 12Postoperative Pulmonary Embolism or Deep Vein Thrombosis6926
      PSI 13Postoperative Sepsis1819
      PSI 14Postoperative Wound Dehiscence160
      PSI 15Accidental Puncture or Laceration2521
      Total25677
      * Data format complies with non-identification standard of the Patient Safety and Quality Improvement Act of 2005.
      Finally, we found that almost one third (32%) of harm events occurred outside the hospital. These outside-acquired harms mirrored the top 10 hospital-acquired harms. Awareness of all-cause harm, including outside-acquired harms, emphasizes the need for proactive partnerships and alignment of incentives between hospitals and community care providers and services.

      Discussion

      While hospitals focus on regulatory harm reduction programs, the descriptions of harm included in these programs are narrow and capture only a small set of harm. For example, obstetric cases are excluded in seven of the eight PSI 90 harm measures. One study suggested that including obstetric cases “would likely provide useful guidance regarding potential areas for improvement.”
      • Gregory K.D.
      • et al.
      AHRQ Patient Safety Indicators: time to include hemorrhage and infection during childbirth.
      (p. 114)
      One reason we captured more harm was because our definitions of harm in the AHS PSO Descriptions of Harm Guidelines were less restrictive than the descriptions of harm in the PSI 90 Composite. We observed three PSI 90 measures that showed a compelling difference in the number of reported harms compared to the number of harms found using the automated trigger review process and more inclusive descriptions (Sidebar 3). For instance, PSI 03 captures only Stages III and IV pressure ulcers and excludes cases with a principal diagnosis of pressure ulcer and a secondary diagnosis that is present on admission. With the RTM solution, an automated trigger fired for any pressure ulcer. Because the AHS PSO pressure ulcer trigger description was not as restrictive as the PSI 03 description, the nurse reviewer was able to find all pressure ulcers, including Stage I. Because we followed the IHI GTT definitions for pressure ulcers, we considered that Stages I–IV were harms.
      • Griffin F.A.
      • Resar R.K.
      IHI Global Trigger Tool for Measuring Adverse Events, 2nd ed. IHI Innovation Series white paper.
      Likewise, while the automated harm trigger system identified PSI 12 (Perioperative Pulmonary Embolism or Deep Vein Thrombosis), as one of the top 10 harms acquired during hospitalization, the system also triggered when a postsurgical patient was readmitted with venous thromboembolism (VTE). We found that up to 70% of patients with VTE were readmitted to the hospital within 15 days with postsurgical VTE. Although this type of case is excluded from the PSI 12 measure, it is a readmission and is included in the AHS PSO Description of Harm Guidelines. The harm definition in PSI 14 (Postoperative Wound Dehiscence) excludes “cases in which the abdominal wall reclosure occurs on or before the day of the first abdominopelvic surgery, cases with an immunocompromised state, cases with stays less than two (2) days, and obstetric cases” (Sidebar 3). The AHS PSO Description of Harm Guidelines include all operative wound dehiscence occurring within 30 days of any surgery related to reasons such as bleeding, infection, and/or surgical misadventures requiring intervention. We acknowledge that one possible difference in the number of harms we found may be due to local clinical and/or coding practices. Undoubtedly however, we believe that our robust harm descriptions give a more complete picture of all-cause harm, leading to more effective improvement efforts.
      Adventist Health System (AHS) Patient Safety Organization (PSO) Harm Descriptions vs. PSI 90 Harm Descriptions
      Tabled 1
      AHS PSO Harm Description
      Adventist Health System Patient Safety Organization. Description of Harm Guidelines. 2015.
      # Harms from RTM SolutionPatient Safety Indicator 90 (PSI 90) Description
      Agency for Healthcare Research and Quality. Draft Patient Safety Indicators Technical Specifications Updates—Version 5.0 (ICD 10), October 2015. Oct 2015. Accessed Jan 23, 2017. http://qualityindicators.ahrq.gov/modules/psi_techspec_icd10.aspx.
      # Harms from Coded Data
      PSI 03Pressure Ulcer “All pressure ulcers (Stages I to IV) occurring at any time during the course of hospitalization or that are outside-acquired from an extended care facility or another acute care facility are harms. Pressure ulcers that progress in stage during hospitalization are harms.”112Pressure Ulcer “Stage III or IV pressure ulcers or unstageable (secondary diagnosis) per 1,000 discharges among surgical or medical patients ages 18 years and older. Excludes stays less than 5 days; cases with a principal diagnosis of pressure ulcer; cases with a secondary diagnosis of Stage III or IV pressure ulcer or unstageable that is present on admission; cases with diseases of the skin, subcutaneous tissue and breast; obstetric cases; cases with hemiplegia, paraplegia, quadriplegia, spina bifida, or anoxic brain damage; cases in which debridement or pedicle graft is the only operating room procedure; discharges with debridement or pedicle graft before or on the same day as the major operating room procedure; and transfers from another facility.”0
      PSI 12Perioperative Venous Thromboembolism (VTE) “A VTE is either a deep vein thrombosis (DVT) and/or pulmonary embolism (PE) that occurs at any time during the course of hospitalization and within 90 days of a previous hospitalization is a harm. A VTE may not manifest itself until after the patient has been discharged from the hospital and may result in an unplanned readmission and is a harm.”69Perioperative Pulmonary Embolism or Deep Vein Thrombosis “Perioperative pulmonary embolism or deep vein thrombosis (secondary diagnosis) per 1,000 surgical discharges for patients ages 18 years and older. Excludes cases with principal diagnosis for pulmonary embolism or deep vein thrombosis; cases with secondary diagnosis for pulmonary embolism or deep vein thrombosis present on admission; cases in which interruption of vena cava occurs before or on the same day as the first operating room procedure; and obstetric discharges.”26
      PSI 14Postoperative Wound Dehiscence “All operative wound dehiscence occurring within 30 days of any surgery related to reasons such as bleeding, infection and/or surgical misadventures requiring intervention are investigated for harms.”16Postoperative Wound Dehiscence “Postoperative reclosures of the abdominal wall per 1,000 abdominopelvic surgery discharges for patients ages 18 years and older. Excludes cases in which the abdominal wall reclosure occurs on or before the day of the first abdominopelvic surgery, cases with an immunocompromised state, cases with stays less than two (2) days, and obstetric cases.”0
      RTM, Risk Trigger® Monitoring.
      * Adventist Health System Patient Safety Organization. Description of Harm Guidelines. 2015.
      Agency for Healthcare Research and Quality. Draft Patient Safety Indicators Technical Specifications Updates—Version 5.0 (ICD 10), October 2015. Oct 2015. Accessed Jan 23, 2017. http://qualityindicators.ahrq.gov/modules/psi_techspec_icd10.aspx.
      The automated trigger solution allows for a dynamic view of all-cause harm; harm not limited to a regulatory harm reduction program. For example, harm not included as a regulatory requirement was apparent in surgical harms, specifically high-risk bariatric surgery. In six months, we found 13 patients with bariatric surgery–associated harms. Four (31%) of those patients developed harm during surgical admission; 9 (69%) were readmitted with postoperative complications such as a pneumoperitoneum, abscesses, or a perforated viscus; and 7 (54%) required a return to surgery. Indeed, an automated harm trigger system helps spotlight harms that occur outside of the national regulatory harm reduction programs.
      The automated trigger review process gave us the ability to identify patterns of harm as they evolved, allowing hospital quality departments the opportunity to respond proactively by providing awareness, education, and intervention trainings as needed. For example, soon after the implementation of the RTM solution, the centralized nurse reviewers identified a pattern of doubling of serum creatinine in patients who received intravenous contrast. On analysis, it was noted that in one month 7% of the patients who underwent computed tomography (CT) with contrast had a doubling of serum creatinine (measured 3 days preprocedure and 3 days postprocedure and sustained for 24 hours). We also found that 13% of the patients who underwent percutaneous transluminal coronary angioplasty (PTCA) had a doubling of serum creatinine sustained for greater than 24 hours. This trend analysis was the impetus for the hospitals to review and revise as appropriate policies and procedures related to intravenous contrast imaging. Two years later, in a follow-up analysis of patients who underwent CT scans with contrast, using the same criteria and time frame, the doubling of serum creatinine was reduced to 1%. Likewise, for patients who underwent cardiac catheterization or PTCA procedures, using the same criteria and time frame, the doubling of serum creatinine was reduced to 1%. Learnings from these types of experiences were readily spread across the health system.
      In addition to the clinical value, we learned there are many benefits of moving from a retrospective to a real-time review process that improved efficiency and accuracy of harm identification. In the manual review process, the triggers were not automated, necessitating reviewers to spend time searching for the harm within the record; therefore, decreasing efficiency. For example, following the GTT manual review process of 20 minutes per record,
      • Griffin F.A.
      • Resar R.K.
      IHI Global Trigger Tool for Measuring Adverse Events, 2nd ed. IHI Innovation Series white paper.
      a nurse reviewer could review 20 records in approximately 6½ hours, whereas with the use of automated triggers, a nurse could review the same number of records in 1½ hours. The time saved allowed for all records with positive triggers to be reviewed, and harm identified, without adding additional nurse reviewer full-time equivalents. In this study, we reviewed 40,592 records using RTM versus 440 records using GTT for similar time frames. We found it interesting that the manual random sampling approach missed VTE altogether (Table 1), and we appreciated that the automated trigger process allowed for a larger number of records to be reviewed.
      In an era of value-based care, health care leaders often look to state or federal regulators to define patient harm that is affected by penalty programs, but these measures of harm are limited in scope and definition. Because the automated harm review process identifies all-cause harm, we recognize that hospital leaders may be cautious because of the human and fiscal resources necessary for an all-cause harm identification and management program. However, recent studies have demonstrated that harm reduction is associated with lower costs and higher contribution margins for hospitals.

      Adler L., et al. Impact of inpatient harms on hospital finances and patient clinical outcomes. J Patient Saf. Epub 2015 Mar 23.

      • Ramly E.P.
      • et al.
      The financial impact of intraoperative adverse events in abdominal surgery.
      Certainly, more work is needed to understand the cost of harm.
      Currently, innovative payment and service delivery models are being tested that call for alignment across providers, including hospitals, post-acute care providers, physicians, and other practitioners, which would incentivize them to work together across all specialties and settings. As hospital leaders prepare for these innovative new payment structures, it will be important for them to be aware of the potential payment risk of all-cause harm occurring outside of the hospital. The automated trigger process for harm identification is one method hospital leaders can use to better understand outside-acquired harm and to proactively partner with community providers to mitigate or prevent costly patient harm.
      As we neared completion of the pilot phase, we developed processes for moving from a centralized review process to a hospital-level review and performance improvement process. During the transition phase a challenge we faced was cultural change at the hospital level. Training of physicians and staff began with “What is a harm?” and progressed to an understanding of all-cause harm identification and principles of mitigation and performance improvement to reduce harm. The role of the centralized team evolved from a total review process to sample auditing of the hospital-level review process in order to maintain reliability.

       Limitations

      First, the two community hospitals were disparate in geographic location, and the results may not be generalizable to all geographic locations and academic, specialty, or tertiary care hospitals. Second, because only records with positive triggers were reviewed, this was not a 100% chart review. Third, because this was a novel approach there was no gold standard for capturing specificity or sensitivity of automated triggers. Fourth, during the course of the pilot phase, microbiology triggers for health care–associated infections were not available; they are currently in use.

      Conclusion

      The sociotechnical approach was critical for the successful development of this automated all-cause harm trigger identification system. By using a novel technology, coupled with skilled nurse reviewers, we demonstrated that an automated harm trigger identification system captures a wider array of harms in a larger sample of patients accurately and efficiently. The process allows for real-time analysis of harm at the hospital level that can be shared throughout the system. In addition, it allows for identification of harm that occurs outside of the hospital, offering opportunities for alignment of safety goals with community partners.
      From this innovative pilot, we have learned the value of real-time all-cause harm detection. The AHS PSO will continue to share patient safety findings, best practices, and patient stories within a transparent collaborative environment throughout the health system. We deem that an automated all-cause harm trigger system is most valuable at the hospital level close to the bedside care team; progressing from solely a harm measurement process to a real-time bedside intervention, which allows for a hospital care team to identify, mitigate, ameliorate, and perhaps prevent harm.

      Conflicts of Interest

      All authors report no conflicts of interest.

      Acknowledgments

      The authors gratefully acknowledge the Adventist Health System Patient Safety Organization Clinical Patient Safety nurse reviewers: Patricia Dalton, BS, RN, and Holly Manley BSN, RN, JD, LLM. The authors recognize Stanley L. Pestotnik, MS, RPh, Chief Strategy Officer, Pascal Metrics, who pioneered the computerized harm identification work and who was influential in the development of the novel risk trigger solution described in this manuscript. They also thank Kathleen Butler, MPH, RN, Director of Clinical Operations, Analytics, and Data Governance, Adventist Health System, for her contributions to data analysis.

      References

        • Institute of Medicine
        To Err Is Human: Building a Safer Health System.
        National Academy Press, Washington DC2000
        • Institute of Medicine
        Crossing the Quality Chasm: A New Health System for the 21st Century.
        National Academy Press, Washington, DC2001
        • US Department of Health and Human Services
        Office of Inspector General. Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries.
        (OEI-06-09-00090; Nov; Accessed Jan 23, 2017)
        • US Centers for Medicare & Medicaid Services
        Ref: S&C: 13-19-Hospitals.
        (Mar 15; Accessed Jan 23, 2017)
        • QualityNet
        Hospital Value-Based Purchasing (VBP) Mortality and AHRQ PSI 90 Composite Measures Overview.
        (Accessed Jan 23, 2017)
        • US Centers for Medicare & Medicaid Services
        Hospital-Acquired Conditions.
        (Updated: Aug 19, 2015); Accessed Jan 23, 2017
        • Brennan T.A.
        • et al.
        Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I.
        Qual Saf Health Care. 2004; 13: 145-152
        • Griffin F.A.
        • Resar R.K.
        IHI Global Trigger Tool for Measuring Adverse Events, 2nd ed. IHI Innovation Series white paper.
        (Cambridge, MA: Institute for Healthcare Improvement; Accessed Jan 23, 2017)
        • Classen D.C.
        • et al.
        Computerized surveillance of adverse drug events in hospital patients.
        JAMA. 1991; 266 (Nov 27; Erratum in: JAMA. 1992 Apr 8;267:1922): 2847-2851
        • Landrigan C.P.
        • et al.
        Temporal trends in rates of patient harm resulting from medical care.
        N Engl J Med. 2010; 363 (Nov 25; Erratum in: N Engl J Med. 2010 Dec 23;363:2573): 2124-2134
        • US Department of Health and Human Services
        Office of Inspector General. Adverse Events in Hospitals: Methods for Identifying Events.
        (OEI-06-08-00221; Mar; Accessed Jan 23, 2017)
        • Classen D.C.
        • et al.
        “Global trigger tool” shows that adverse events in hospitals may be ten times greater than previously measured.
        Health Aff (Millwood). 2011; 30 (Erratum in: Health Aff (Millwood). 2011;30:1217): 581-589
        • Good V.S.
        • et al.
        Large-scale deployment of the global trigger tool across a large hospital system: refinements for the characterisation of adverse events to support patient safety learning opportunities.
        BMJ Qual Saf. 2011; 20: 25-30
        • Garrett Jr, P.R.
        • et al.
        Developing and implementing a standardized process for global trigger tool application across a large health system.
        Jt Comm J Qual Patient Saf. 2013; 39: 292-297
        • Kennerly D.A.
        • et al.
        Characterization of adverse events detected in a large health care delivery system using an enhanced global trigger tool over a five-year interval.
        Health Serv Res. 2014; 49: 1407-1425
        • Classen D.C.
        • et al.
        Description of a computerized adverse drug event monitor using a hospital information system.
        Hosp Pharm. 1992; 27
        • Sharek P.J.
        • et al.
        Adverse events in the neonatal intensive care unit: development, testing, and findings of an NICU-focused trigger tool to identify harm in North American NICUs.
        Pediatrics. 2006; 118: 1332-1340
        • Takata G.S.
        • et al.
        Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals.
        Pediatrics. 2008; 121: e927-e935
        • Kirkendall E.S.
        • et al.
        Measuring adverse events and levels of harm in pediatric inpatients with the Global Trigger Tool.
        Pediatrics. 2012; 130: e1206-e1214
        • Stockwell D.C.
        • et al.
        Automated adverse event detection collaborative: electronic adverse event identification, classification, and corrective actions across academic pediatric institutions.
        J Patient Saf. 2013; 9: 203-210
        • Stockwell D.C.
        • et al.
        A trigger tool to detect harm in pediatric inpatient settings.
        Pediatrics. 2015; 135: 1036-1042
        • US Department of Health and Human Services
        Office of Inspector General. Adverse Events in Skilled Nursing Facilities: National Incidence Among Medicare Beneficiaries.
        (OEI-06-11-00370; Feb; Accessed Jan 23, 2017)
        • Institute of Medicine
        Health IT and Patient Safety: Building Safer Systems for Better Care.
        National Academies Press, Washington, DC2012
        • James J.T.
        A new, evidence-based estimate of patient harms associated with hospital care.
        J Patient Saf. 2013; 9: 122-128
      1. Trist E. Murray H. Trist B. The Social Engagement of Social Science: A Tavistock Anthology. The Socio-Technical Perspective. Vol. 2. University of Pennsylvania Press, Philadelphia1993
        • National Coordinating Council for Medication Error Reporting and Prevention
        NCC MERP Index for Categorizing Medication Errors.
        (Accessed Jan 23, 2017)
      2. Adler L., et al. Impact of inpatient harms on hospital finances and patient clinical outcomes. J Patient Saf. Epub 2015 Mar 23.

        • Gregory K.D.
        • et al.
        AHRQ Patient Safety Indicators: time to include hemorrhage and infection during childbirth.
        Jt Comm J Qual Patient Saf. 2013; 39: 114-122
        • Ramly E.P.
        • et al.
        The financial impact of intraoperative adverse events in abdominal surgery.
        Surgery. 2015; 158: 1382-1388

      Biography

      Christine Sammer, DrPH, RN, is Director, Corporate Patient Safety, Office of Clinical Effectiveness, Adventist Health System, Altamonte Springs, Florida.
      Susanne Miller, RN, MS, is Vice President, Risk Trigger, Pascal Metrics, Washington, DC.
      Cason Jones, MLS, MHA, is Senior Research Analyst, Office of Clinical Effectiveness, Adventist Health System, Altamonte Springs, Florida; Antoinette Nelson, RN, BSN, MSHSA, is Manager, Clinical Patient Safety, Office of Clinical Effectiveness, Adventist Health System, Altamonte Springs, Florida; and Paul Garrett, MD, is Senior Medical Director, Evidence Based Practice, Office of Clinical Effectiveness, Adventist Health System.
      David Classen, MD, MS, is Associate Professor of Medicine, University of Utah, and Chief Medical Information Officer, Pascal Metrics.
      David Stockwell, MD, is Vice President, Clinical Services, Pascal Metrics, and Associate Professor of Pediatrics, Children's National Health System, George Washington University School of Medicine & Health Sciences, Washington, DC.