If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
From 2009 through 2012, the Adventist Health System Patient Safety Organization (AHS PSO) used the Global Trigger Tool method for harm identification and demonstrated harm reduction. Although the awareness of harm demonstrated opportunities for improvement across the system, leaders determined that the human and fiscal resources required to continue with a retrospective manual harm identification process were unsustainable. In addition, there was growing concern that the identification of harm after the patient's discharge did not allow for intervention during the hospital stay. Therefore, the AHS PSO decided to seek an alternative method for patient harm identification.
The AHS PSO and another PSO jointly developed a novel automated all-cause harm trigger identification system that allowed for real-time bedside intervention, real-time trend analysis affecting patient safety, and continued learning about harm measurement. A sociotechnical approach of people, process, and technology was used at two pilot hospitals sharing the same electronic health record platform. Automated positive harm triggers and work-flow models were developed and evaluated.
Combined data from the two hospitals in a period of 11 consecutive months indicated (1) a total of 2,696 harms (combined hospital-acquired and outside-acquired); (2) that hypoglycemia (blood glucose ≤ 40 mg/dL) was the most frequently identified harm; (3) 256 harms related to the Patient Safety Indicator 90 (PSI 90) Composite descriptions versus 77 harms reported to regulatory harm reduction programs; and (4) that almost one third (32%) of total harms were classified as outside-acquired.
The automated harm trigger system revealed not only more harm but a broader scope of harm and led to a deeper understanding of patient safety vulnerabilities.
Health care leaders hold the Institute of Medicine (IOM) reports, To Err Is Human: Building A Safer Health System
as seminal work in patient safety. These reports addressed the high incidence of patient harms occurring in hospitals and were a catalyst for the passage of the Patient Safety and Quality Improvement Act of 2005. Subsequently, the US Department of Health and Human Services Office of Inspector General (OIG) released a series of reports that established the critical issue of the underreporting of harms occurring in hospitals. This prompted another report by the OIG in 2010, Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries,
which recommended that the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS) encourage internal hospital reporting of all harms—whether considered a complication, a preventable harm, or a harm caused by system failures or errors (all-cause harm).
The GTT harm measurement method involved a retrospective manual review of inpatient records using triggers (discrete data elements) suggesting the possible presence of harm or potential harm. The use of triggers, along with a precisely defined health record review process, allowed for a more comprehensive and reliable approach to harm identification than did the traditional methods of voluntary incident reporting and observational tracking. Hospitals and health care systems that utilized trigger methodologies found up to 40% of adult,
had experienced at least one harm while under hospital or skilled nursing care, illustrating the vast scope of inpatient harm and the need for continued harm reduction initiatives. Furthermore, in 2011, the IOM issued the Health IT and Patient Safety: Building Safer Systems for Better Care report that emphasized the importance of automated real-time harm detection systems to supplement or possibly replace manual record review, which could allow clinicians to intervene immediately to prevent or mitigate imminent harm.
which highlights the potential usefulness of all-cause harm detection systems.
From 2009 through 2012, the Adventist Health System Patient Safety Organization (AHS PSO) successfully used the GTT method for harm identification and demonstrated harm reduction. The major harm findings drove systemwide collaborative improvement projects.
Although the awareness of harm, provided by the retrospective manual review method, demonstrated opportunities for improvement across the system, leaders determined that the human and fiscal resources required to continue with a retrospective manual harm identification process were unsustainable. In addition, there was growing concern that the identification of harm after the patient's discharge did not allow for intervention during the hospital stay. Therefore, the AHS PSO decided to seek an alternative method for patient harm identification. The objective of this paper is to demonstrate how two PSOs developed a novel automated all-cause harm trigger identification system that allowed for real-time bedside intervention, real-time trend analysis affecting patient safety, and continued learning about harm measurement.
Development of the Automated Harm Trigger System
The AHS PSO and Pascal Metrics, Inc. (a PSO dedicated to patient safety and high reliability of care), partnered to develop a proof of concept using Risk Trigger® Monitoring (RTM), an automated Web-based harm trigger decision support solution. The RTM platform allowed for an interactive environment where all users shared potentially sensitive safety data that were protected by the PSOs. The PSOs ensured freedom from discovery, allowing for the ability to learn from and bolster a well-informed safety performance improvement program. We piloted this novel decision support solution at two large urban hospitals, using the sociotechnical approach that recognizes the interaction between people, process, and technology in the workplace
Trigger development began in 2012 by building a set of automated triggers. Trigger criteria and algorithms were developed and customized for the pilot hospital's health information technology system. The trigger definitions we used were very similar to the IHI GTT definitions, and we automated those triggers that used discrete data readily available in the electronic health record (EHR). We found that some GTT triggers could not be automated because the documentation was on paper or entered as free text within the EHR. In addition, we did not automate the GTT trigger diphenhydramine administration because it was often used therapeutically and not always associated with patient harm. Between 2012 and 2014, we continued to refine the automated triggers based on expert clinical opinion from both PSOs because there was not a gold standard for this novel approach. The refinement process was done by observing the relationship of firing frequency to harm identification and the severity of harm identified. For example, the fall trigger logic was modified to improve the firing frequency and increase the positive predictive value (PPV). The trigger initially was written to look for the date of fall. We discovered that the date of fall element included both the current fall date and the historical fall date, which occurred prior to hospital admission. The logic was updated to fire only if the fall date occurred after the patient was admitted to the hospital. This change resulted in a more specific trigger with a PPV improvement from 1% to 14%.
Following the pilot phase, we continued the trigger and refinement process and found that harm triggers associated with Clostridium difficile–positive stool, hypoglycemia, and 3rd and 4th degree vaginal lacerations had the greatest PPV (range, 44%–91%). Triggers associated with coagulation disturbances, Carbapenem-resistant Enterobacteriaceae culture, digoxin overdose, and pressure ulcers also had good PPV (range, 25%–40%).
In 2013 we created an electronic flat file extract of clinical data from the EHR (demographic data, bed movement, laboratory results, radiology orders, surgery times, and nursing assessments) that was securely transmitted to the RTM platform daily and piloted at one hospital. We found that the automated trigger system, populated with the EHR flat file extract, demonstrated that it was possible to automate harm triggers. In early 2014, with the addition of a second hospital, it was determined that utilizing Health Level Seven® (HL7®) messages, a real-time stream of data from the EHR, would provide a more reliable and timely data feed. In addition, HL7 data streams allowed us to broaden the types of triggers and events that the system could monitor because the hospitals had existing outbound HL7 messages for most major systems such as admission, discharge, transfer; laboratory; medication orders; and radiology. Because we could not develop HL7 data streams from nursing flow sheets, electronic medicationadministration records, and surgical procedure documentation systems, we developed hourly flat file extracts to mimic real-time transmission. Utilizing HL7 data feeds augmented by the hourly flat file extracts allowed for a more robust and real-time data feed. Both hospitals went live in mid-2014, in a production environment, with a basic set of 41 automated triggers.
The People and the Process: Harm Identification
Concurrent with trigger development, the AHS PSO team of registered nurses and physician authenticators, experienced in harm identification using the GTT manual process,
established standard operating procedures for using the RTM solution in tandem with the EHR. The team referenced the IHI definition of harm—“unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalization, or that results in death,”
(p. 5) and identified only harms of commission. Definitions and descriptions of harm were developed for each positive trigger in the AHS PSO Automated Risk Trigger Tool Guideline for Record Review. Identified harms were categorized according to the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index for Categorizing Medication Errors.
The patient population consisted of inpatients 18 years of age or older and excluded psychiatric, rehabilitation, and hospice admissions.
A centralized nurse review team model for work flow was developed (Figure 1). During the review process, the nurse reviewers used dual computer monitors—one monitor to access the list of patients with positive triggers from the Web-based RTM solution, and a second monitor to view thepatient's open EHR. The nurse reviewer followed the automated positive trigger in the EHR and determined if a patient harm had occurred. This process averaged approximately five minutes per trigger. If a harm was identified, it was classified as hospital-acquired or outside-acquired and was grouped into one of five harm categories (medication, patient care, surgery or procedure, perinatal, and health care–associated infection) and assigned a severity level (E–I) according to the NCC MERP
(Sidebar 1). We captured severity level E harm because from the patient's perspective the harm was an unexpected event that required intervention. In addition, Adler et al. found that level E harms were positively correlated with total cost, variable cost, length of stay, and 30-day readmissions.
While following the trigger, the experienced nurse reviewer sometimes found harm that was not associated with an automated trigger, and it was documented as a harm. All harms were confirmed and approved after discharge by a physician or a registered nurse. If no harm was found, the review ended. When potential harms were identified, the nurse reviewer added a communication note into the RTM platform which was viewable at the hospital level. These notifications provided the hospital care team with information regarding potential harms that could require bedside intervention, as described in a case study (Sidebar 2).
The National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index for Categorizing Errors to define adverse events as follows:
Category E = temporary harm and required intervention;
Category F = temporary harm and required initial or prolonged hospitalization;
Category G = permanent harm;
Category H = intervention required to sustain life;
Category I = death
*Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events, 2nd ed. IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement, 2009. (Available at http://www.ihi.org)
Identifying inpatient harm or potential harm in real time provides the hospital care team with multiple triggers to analyze patient data quickly and supports informed care decisions at the bedside.
The centralized review team received an automated positive harm trigger patient list. For one patient, multiple harm triggers fired consisting of (1) consecutive blood glucose levels of 180 mg/dL within 12 hours, (2) glucose greater than 250 mg/dL, and (3) lactate level > 4.0 mL. A review of the electronic health record (EHR) revealed that a 70-year-old woman with a history of obstructive pulmonary disease had been admitted to a medical/surgical unit via the emergency department with increased dyspnea, wheezing, and possible pneumonia. Due to the group of multiple harm triggers fired and review of the patient's condition as recorded in the EHR, the centralized review team recognized that this patient, despite a normal temperature and blood pressure, was at high risk for sepsis. A potential for sepsis harm was recorded in the RTM platform. At the hospital, the intervention coordinator viewed the multiple harm triggers and the potential for sepsis harm notation. A review of the emergency department record showed that sepsis surveillance had been initiated. The combination of automated triggers and the potential for sepsis notation led the care team to promptly conduct a physical assessment revealing that the patient was in moderate distress with increased lethargy. The patient was subsequently transferred to intensive care for treatment of pneumonia, sepsis, uncontrolled hyperglycemia, and volume depletion. The patient recovered and was discharged home in stable condition four days later.
We developed a work flow for the hospital intervention coordinator who had the capability to view the list of inpatients with positive triggers, as well as harms, potential harms, and communication notes recorded by the centralized team. The intervention coordinator communicated with the hospital care team, which provided intervention or action as appropriate. Regular collaborative calls between both PSOs and the hospital care teams were established to share ideas, issues, and ongoing improvements.
First, the automated review process captured more harms and a wider array of harms. We found a total of 2,696 harms, hospital-acquired and outside-acquired, using the automated trigger review process, as compared to 132 harms found using a manual review process for similar time periods (Table 1, Figure 2, Figure 3).
Table 1Number (Percentage) of Hospital- and Outside-Acquired Harm Types Identified by Automated Risk Trigger and Global Trigger Tool (GTT) Manual Methods
Harm Types Identified (Hospital-Acquired and Outside-Acquired)
Automated Risk Trigger Method # of Harms (Period of 11 Consecutive Months
Second, we found the 10 most frequently identified hospital-acquired harms and corresponding severity levels (Table 2). Hypoglycemia (blood glucose ≤ 40 mg/dL) related to the use of insulin or oral hypoglycemic agents was the most frequently identified harm. In addition, bleeding related to medication, and oversedation related to medication were consistently found in the top 10 harms; none of which were included in a regulatory harm reduction program.
Table 2Ten Most Frequently Indentified Hospital-Acquired Harms by Severity Level (Period of 11 Consecutive Months)
Third, we compared the number of harms found using the automated trigger review process to the number of harms reported to the PSI 90 regulatory program, which, at the time of this study, included eight PSIs. We found 256 harms related to the PSI 90 Composite descriptions, whereas only 77 harms were reported to CMS for the same time frame (Table 3).
Table 3Comparison of Hospital-Acquired Harms Using an Automated Trigger Methodolgy vs. Reported Patient Safety Indicator 90 (PSI 90) Harms (Period of 11 Consecutive Months)
Finally, we found that almost one third (32%) of harm events occurred outside the hospital. These outside-acquired harms mirrored the top 10 hospital-acquired harms. Awareness of all-cause harm, including outside-acquired harms, emphasizes the need for proactive partnerships and alignment of incentives between hospitals and community care providers and services.
While hospitals focus on regulatory harm reduction programs, the descriptions of harm included in these programs are narrow and capture only a small set of harm. For example, obstetric cases are excluded in seven of the eight PSI 90 harm measures. One study suggested that including obstetric cases “would likely provide useful guidance regarding potential areas for improvement.”
One reason we captured more harm was because our definitions of harm in the AHS PSO Descriptions of Harm Guidelines were less restrictive than the descriptions of harm in the PSI 90 Composite. We observed three PSI 90 measures that showed a compelling difference in the number of reported harms compared to the number of harms found using the automated trigger review process and more inclusive descriptions (Sidebar 3). For instance, PSI 03 captures only Stages III and IV pressure ulcers and excludes cases with a principal diagnosis of pressure ulcer and a secondary diagnosis that is present on admission. With the RTM solution, an automated trigger fired for any pressure ulcer. Because the AHS PSO pressure ulcer trigger description was not as restrictive as the PSI 03 description, the nurse reviewer was able to find all pressure ulcers, including Stage I. Because we followed the IHI GTT definitions for pressure ulcers, we considered that Stages I–IV were harms.
Likewise, while the automated harm trigger system identified PSI 12 (Perioperative Pulmonary Embolism or Deep Vein Thrombosis), as one of the top 10 harms acquired during hospitalization, the system also triggered when a postsurgical patient was readmitted with venous thromboembolism (VTE). We found that up to 70% of patients with VTE were readmitted to the hospital within 15 days with postsurgical VTE. Although this type of case is excluded from the PSI 12 measure, it is a readmission and is included in the AHS PSO Description of Harm Guidelines. The harm definition in PSI 14 (Postoperative Wound Dehiscence) excludes “cases in which the abdominal wall reclosure occurs on or before the day of the first abdominopelvic surgery, cases with an immunocompromised state, cases with stays less than two (2) days, and obstetric cases” (Sidebar 3). The AHS PSO Description of Harm Guidelines include all operative wound dehiscence occurring within 30 days of any surgery related to reasons such as bleeding, infection, and/or surgical misadventures requiring intervention. We acknowledge that one possible difference in the number of harms we found may be due to local clinical and/or coding practices. Undoubtedly however, we believe that our robust harm descriptions give a more complete picture of all-cause harm, leading to more effective improvement efforts.
Agency for Healthcare Research and Quality. Draft Patient Safety Indicators Technical Specifications Updates—Version 5.0 (ICD 10), October 2015. Oct 2015. Accessed Jan 23, 2017. http://qualityindicators.ahrq.gov/modules/psi_techspec_icd10.aspx.
# Harms from Coded Data
Pressure Ulcer “All pressure ulcers (Stages I to IV) occurring at any time during the course of hospitalization or that are outside-acquired from an extended care facility or another acute care facility are harms. Pressure ulcers that progress in stage during hospitalization are harms.”
Pressure Ulcer “Stage III or IV pressure ulcers or unstageable (secondary diagnosis) per 1,000 discharges among surgical or medical patients ages 18 years and older. Excludes stays less than 5 days; cases with a principal diagnosis of pressure ulcer; cases with a secondary diagnosis of Stage III or IV pressure ulcer or unstageable that is present on admission; cases with diseases of the skin, subcutaneous tissue and breast; obstetric cases; cases with hemiplegia, paraplegia, quadriplegia, spina bifida, or anoxic brain damage; cases in which debridement or pedicle graft is the only operating room procedure; discharges with debridement or pedicle graft before or on the same day as the major operating room procedure; and transfers from another facility.”
Perioperative Venous Thromboembolism (VTE) “A VTE is either a deep vein thrombosis (DVT) and/or pulmonary embolism (PE) that occurs at any time during the course of hospitalization and within 90 days of a previous hospitalization is a harm. A VTE may not manifest itself until after the patient has been discharged from the hospital and may result in an unplanned readmission and is a harm.”
Perioperative Pulmonary Embolism or Deep Vein Thrombosis “Perioperative pulmonary embolism or deep vein thrombosis (secondary diagnosis) per 1,000 surgical discharges for patients ages 18 years and older. Excludes cases with principal diagnosis for pulmonary embolism or deep vein thrombosis; cases with secondary diagnosis for pulmonary embolism or deep vein thrombosis present on admission; cases in which interruption of vena cava occurs before or on the same day as the first operating room procedure; and obstetric discharges.”
Postoperative Wound Dehiscence “All operative wound dehiscence occurring within 30 days of any surgery related to reasons such as bleeding, infection and/or surgical misadventures requiring intervention are investigated for harms.”
Postoperative Wound Dehiscence “Postoperative reclosures of the abdominal wall per 1,000 abdominopelvic surgery discharges for patients ages 18 years and older. Excludes cases in which the abdominal wall reclosure occurs on or before the day of the first abdominopelvic surgery, cases with an immunocompromised state, cases with stays less than two (2) days, and obstetric cases.”
RTM, Risk Trigger® Monitoring.
* Adventist Health System Patient Safety Organization. Description of Harm Guidelines. 2015.
The automated trigger solution allows for a dynamic view of all-cause harm; harm not limited to a regulatory harm reduction program. For example, harm not included as a regulatory requirement was apparent in surgical harms, specifically high-risk bariatric surgery. In six months, we found 13 patients with bariatric surgery–associated harms. Four (31%) of those patients developed harm during surgical admission; 9 (69%) were readmitted with postoperative complications such as a pneumoperitoneum, abscesses, or a perforated viscus; and 7 (54%) required a return to surgery. Indeed, an automated harm trigger system helps spotlight harms that occur outside of the national regulatory harm reduction programs.
The automated trigger review process gave us the ability to identify patterns of harm as they evolved, allowing hospital quality departments the opportunity to respond proactively by providing awareness, education, and intervention trainings as needed. For example, soon after the implementation of the RTM solution, the centralized nurse reviewers identified a pattern of doubling of serum creatinine in patients who received intravenous contrast. On analysis, it was noted that in one month 7% of the patients who underwent computed tomography (CT) with contrast had a doubling of serum creatinine (measured 3 days preprocedure and 3 days postprocedure and sustained for 24 hours). We also found that 13% of the patients who underwent percutaneous transluminal coronary angioplasty (PTCA) had a doubling of serum creatinine sustained for greater than 24 hours. This trend analysis was the impetus for the hospitals to review and revise as appropriate policies and procedures related to intravenous contrast imaging. Two years later, in a follow-up analysis of patients who underwent CT scans with contrast, using the same criteria and time frame, the doubling of serum creatinine was reduced to 1%. Likewise, for patients who underwent cardiac catheterization or PTCA procedures, using the same criteria and time frame, the doubling of serum creatinine was reduced to 1%. Learnings from these types of experiences were readily spread across the health system.
In addition to the clinical value, we learned there are many benefits of moving from a retrospective to a real-time review process that improved efficiency and accuracy of harm identification. In the manual review process, the triggers were not automated, necessitating reviewers to spend time searching for the harm within the record; therefore, decreasing efficiency. For example, following the GTT manual review process of 20 minutes per record,
a nurse reviewer could review 20 records in approximately 6½ hours, whereas with the use of automated triggers, a nurse could review the same number of records in 1½ hours. The time saved allowed for all records with positive triggers to be reviewed, and harm identified, without adding additional nurse reviewer full-time equivalents. In this study, we reviewed 40,592 records using RTM versus 440 records using GTT for similar time frames. We found it interesting that the manual random sampling approach missed VTE altogether (Table 1), and we appreciated that the automated trigger process allowed for a larger number of records to be reviewed.
In an era of value-based care, health care leaders often look to state or federal regulators to define patient harm that is affected by penalty programs, but these measures of harm are limited in scope and definition. Because the automated harm review process identifies all-cause harm, we recognize that hospital leaders may be cautious because of the human and fiscal resources necessary for an all-cause harm identification and management program. However, recent studies have demonstrated that harm reduction is associated with lower costs and higher contribution margins for hospitals.
Certainly, more work is needed to understand the cost of harm.
Currently, innovative payment and service delivery models are being tested that call for alignment across providers, including hospitals, post-acute care providers, physicians, and other practitioners, which would incentivize them to work together across all specialties and settings. As hospital leaders prepare for these innovative new payment structures, it will be important for them to be aware of the potential payment risk of all-cause harm occurring outside of the hospital. The automated trigger process for harm identification is one method hospital leaders can use to better understand outside-acquired harm and to proactively partner with community providers to mitigate or prevent costly patient harm.
As we neared completion of the pilot phase, we developed processes for moving from a centralized review process to a hospital-level review and performance improvement process. During the transition phase a challenge we faced was cultural change at the hospital level. Training of physicians and staff began with “What is a harm?” and progressed to an understanding of all-cause harm identification and principles of mitigation and performance improvement to reduce harm. The role of the centralized team evolved from a total review process to sample auditing of the hospital-level review process in order to maintain reliability.
First, the two community hospitals were disparate in geographic location, and the results may not be generalizable to all geographic locations and academic, specialty, or tertiary care hospitals. Second, because only records with positive triggers were reviewed, this was not a 100% chart review. Third, because this was a novel approach there was no gold standard for capturing specificity or sensitivity of automated triggers. Fourth, during the course of the pilot phase, microbiology triggers for health care–associated infections were not available; they are currently in use.
The sociotechnical approach was critical for the successful development of this automated all-cause harm trigger identification system. By using a novel technology, coupled with skilled nurse reviewers, we demonstrated that an automated harm trigger identification system captures a wider array of harms in a larger sample of patients accurately and efficiently. The process allows for real-time analysis of harm at the hospital level that can be shared throughout the system. In addition, it allows for identification of harm that occurs outside of the hospital, offering opportunities for alignment of safety goals with community partners.
From this innovative pilot, we have learned the value of real-time all-cause harm detection. The AHS PSO will continue to share patient safety findings, best practices, and patient stories within a transparent collaborative environment throughout the health system. We deem that an automated all-cause harm trigger system is most valuable at the hospital level close to the bedside care team; progressing from solely a harm measurement process to a real-time bedside intervention, which allows for a hospital care team to identify, mitigate, ameliorate, and perhaps prevent harm.
Conflicts of Interest
All authors report no conflicts of interest.
The authors gratefully acknowledge the Adventist Health System Patient Safety Organization Clinical Patient Safety nurse reviewers: Patricia Dalton, BS, RN, and Holly Manley BSN, RN, JD, LLM. The authors recognize Stanley L. Pestotnik, MS, RPh, Chief Strategy Officer, Pascal Metrics, who pioneered the computerized harm identification work and who was influential in the development of the novel risk trigger solution described in this manuscript. They also thank Kathleen Butler, MPH, RN, Director of Clinical Operations, Analytics, and Data Governance, Adventist Health System, for her contributions to data analysis.
Cason Jones, MLS, MHA, is Senior Research Analyst, Office of Clinical Effectiveness, Adventist Health System, Altamonte Springs, Florida; Antoinette Nelson, RN, BSN, MSHSA, is Manager, Clinical Patient Safety, Office of Clinical Effectiveness, Adventist Health System, Altamonte Springs, Florida; and Paul Garrett, MD, is Senior Medical Director, Evidence Based Practice, Office of Clinical Effectiveness, Adventist Health System.
David Classen, MD, MS, is Associate Professor of Medicine, University of Utah, and Chief Medical Information Officer, Pascal Metrics.
David Stockwell, MD, is Vice President, Clinical Services, Pascal Metrics, and Associate Professor of Pediatrics, Children's National Health System, George Washington University School of Medicine & Health Sciences, Washington, DC.