Advertisement

Increasing Naloxone Prescribing in the Emergency Department Through Education and Electronic Medical Record Work-Aids

Open AccessPublished:March 06, 2021DOI:https://doi.org/10.1016/j.jcjq.2021.03.002

      Background

      Emergency department (ED) visits for opioid overdose continue to rise. Evidence-based harm reduction strategies for opioid use disorder (OUD), such as providing home naloxone, can save lives, but ED implementation remains challenging.

      Methods

      The researchers aimed to increase prescribing of naloxone to ED patients with OUD and opioid overdose by employing a model for improvement methodology, a multidisciplinary team, and high-reliability interventions. Monthly naloxone prescribing rates among discharged ED patients with opioid overdose and OUD–related diagnoses were tracked over time. Interventions included focused ED staff education on OUD and naloxone, and creation of electronic medical record (EMR)–based work-aids, including a naloxone Best Practice Advisory (BPA) and order set. Autoregressive interrupted time series was used to model the impact of these interventions on naloxone prescribing rates. The impact of education on ED staff confidence and perceived barriers to prescribing naloxone was measured using a published survey instrument.

      Results

      After adjusting for education events and temporal trends, ED naloxone BPA and order set implementation was associated with a significant immediate 21.1% increase in naloxone prescribing rates, which was sustained for one year. This corresponded to increased average monthly prescribing rates from 1.5% before any intervention to 28.7% afterward. ED staff education had no measurable impact on prescribing rates but was associated with increased nursing perceived importance and increased provider confidence in prescribing naloxone.

      Conclusions

      A significant increase in naloxone prescribing rates was achieved after implementation of high-reliability EMR work-aids and staff education. Similar interventions may be key to wider ED staff engagement in harm reduction for patients with OUD.
      Opioid overdoses in the United States have increased at an alarming rate over the last 20 years, with more than 300,000 emergency department (ED) visits and nearly 48,000 deaths in 2017 related to opioid overdose.
      • Vivolo-Kantor AM
      • et al.
      Nonfatal drug overdoses treated in emergency departments—United States, 2016–2017.
      ,
      • Wilson N
      • et al.
      Drug and opioid-involved overdose deaths—United States, 2017–2018.
      Among patients who present to the ED with nonfatal overdose, there remains a high 5.5% mortality rate within one year.
      • Weiner SG
      • et al.
      One-year mortality of patients after emergency department treatment for nonfatal opioid overdose.
      Since the start of the COVID-19 pandemic, ED visits for opioid overdose have climbed even higher,

      North Carolina Department of Health and Human Services, Injury and Violence Prevention Branch. North Carolina Emergency Department (ED) Visits for Opioid Overdoses: December 2020. Jan 13, 2021. Accessed Mar 8, 2021. https://www.injuryfreenc.ncdhhs.gov/DataSurveillance/StatewideOverdoseSurveillanceReports/OpioidOverdoseEDVisitsMonthlyReports/StatewideOpioidOverdoseSurveillanceEDData-Dec2020.pdf.

      reinforcing the need for ED practitioners to intervene in this crisis by initiating harm reduction strategies. In particular, improved access of patients with opioid use disorder (OUD) to the opioid reversal agent, naloxone, through both increased prescribing and community distribution programs has been associated with layperson administration of naloxone and reductions in overdose mortality.
      • Walley AY
      • et al.
      Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis.
      • Clark AK
      • Wilder CM
      • Winstanley EL.
      A systematic review of community opioid overdose prevention and naloxone distribution programs.
      • Maxwell S
      • et al.
      Prescribing naloxone to actively injecting heroin users: a program to reduce heroin overdose deaths.
      • Dwyer K
      • et al.
      Opioid education and nasal naloxone rescue kits in the emergency department.
      Despite being recommended by numerous national organizations,

      American College of Emergency Physicians, Trauma & Injury Prevention Section. Emergency Department Naloxone Distribution: Key Considerations and Implementation Strategies. Samuels EA, et al. 2015. Accessed Mar 8, 2021. https://prescribetoprevent.org/wp2015/wp-content/uploads/TIPSWhitePaper.pdf.

      National Institute on Drug Abuse. Naloxone for Opioid Overdose: Life-Saving Science. Mar 2017. Accessed Mar 8, 2021. https://www.drugabuse.gov/publications/naloxone-opioid-overdose-life-saving-science.

      Substance Abuse and Mental Health Services Administration. Naloxone. (Updated: Aug 19, 2020.) Accessed Mar 8, 2021. https://www.samhsa.gov/medication-assisted-treatment/medications-counseling-related-conditions/naloxone.

      US Food and Drug Administration. New Recommendations for Naloxone. Jul 23, 2020. Accessed Mar 8, 2021. https://www.fda.gov/drugs/drug-safety-and-availability/new-recommendations-naloxone.

      naloxone dispensing and prescribing from the ED are often limited due to multiple barriers, including negative perceptions and lack of knowledge by providers and hospital administrators, legal and logistical constraints on medication dispensation procedures, cost of the medication for patients and/or health systems, and ability or willingness of patients to fill prescriptions or receive medication.
      • Eswaran V
      • et al.
      Take-home naloxone program implementation: lessons learned from seven Chicago-area hospitals.
      ,
      • Gunn AH
      • et al.
      The emergency department as an opportunity for naloxone distribution.
      Reported ED naloxone dispensing and prescribing rates for patients with OUD or opioid overdose prior to 2018 range from 0% to 8%.
      • Drainoni M-L
      • et al.
      Why is it so hard to implement change? A qualitative examination of barriers and facilitators to distribution of naloxone for overdose prevention in a safety net environment.
      • Marino R
      • et al.
      Do electronic health record prompts increase take-home naloxone administration for emergency department patients after an opioid overdose?.
      • Holland WC
      • et al.
      Interrupted time series of user-centered clinical decision support implementation for emergency department–initiated buprenorphine for opioid use disorder.
      These low rates have been ascribed to ED staff and provider perceptual, training, and work-flow barriers.
      • Drainoni M-L
      • et al.
      Why is it so hard to implement change? A qualitative examination of barriers and facilitators to distribution of naloxone for overdose prevention in a safety net environment.
      More recent studies have reported higher naloxone dispensing rates of 25%–35% of ED patients with opioid overdose or OUD–related visits, but it is unclear whether these higher rates are related to increased provider awareness of the opioid crisis, incorporation of electronic medical record (EMR)–based order sets for prescribing, or restriction of the reported population to just patients seen for complications of OUD.
      • Eswaran V
      • et al.
      Development of a take-home naloxone program at an urban academic emergency department.
      ,
      • Samuels EA
      • et al.
      Adoption and utilization of an emergency department naloxone distribution and peer recovery coach consultation program.
      Few studies have analyzed the impact of processes to improve naloxone prescribing. One group recently implemented an EMR–based clinical decision support tool and order set for patients with OUD,
      • Holland WC
      • et al.
      Interrupted time series of user-centered clinical decision support implementation for emergency department–initiated buprenorphine for opioid use disorder.
      while two other groups used EMR–based prompts such as a Best Practice Advisory (BPA) to alert providers to patients with opioid overdose or OUD who may benefit from naloxone.
      • Marino R
      • et al.
      Do electronic health record prompts increase take-home naloxone administration for emergency department patients after an opioid overdose?.
      ,
      • Hussain S.
      A multi-modal quality- improvement study of strategies for enhancing adoption of naloxone- prescribing clinical decision support (master's thesis).
      Early findings from these studies have shown modest improvements in naloxone prescribing rates, but evidence of sustainability remains limited.
      • Marino R
      • et al.
      Do electronic health record prompts increase take-home naloxone administration for emergency department patients after an opioid overdose?.
      ,
      • Holland WC
      • et al.
      Interrupted time series of user-centered clinical decision support implementation for emergency department–initiated buprenorphine for opioid use disorder.
      ,
      • Hussain S.
      A multi-modal quality- improvement study of strategies for enhancing adoption of naloxone- prescribing clinical decision support (master's thesis).
      Faced with similar barriers, we sought to improve provision of naloxone to patients with OUD and opioid overdose through a low-cost and feasible strategy aimed at increasing ED naloxone prescribing rates. We hypothesized that combining focused ED staff education and a BPA reminder coupled with a naloxone order set would significantly increase the percentage of ED patients with OUD or opioid overdose receiving a naloxone prescription at ED discharge. Quality improvement (QI) methods and interrupted time series (ITS) analysis were employed to develop, implement, and assess the impact of this strategy on ED naloxone prescribing rates (Figure 1). As our baseline prescribing rate was found to be 1.5%, our QI goal was to achieve a 50% naloxone prescribing rate.
      Figure 1:
      Figure 1This diagram displays the key drivers and the interventions implemented to achieve the project aims. ED, emergency department, SMART, specific, measurable, achievable, relevant, time-bound; DUH, Duke University Hospital; EMR, electronic medical record; BPA, Best Practice Advisory.

      Methods

      Setting

      This was an ITS analysis of QI interventions using EMR data for patient visits from January 2017 to March 2020 in a large, urban, tertiary care academic ED with approximately 80,000 visits annually. ED providers at our institution include emergency medicine–trained attending physicians, resident physicians, and physician assistants (PAs), as well as rotating nonemergency medicine “off-service” residents undergoing a training month in the ED. ED staff also include ED–trained RNs and clinical support staff, including nursing assistants and technicians. The project team was comprised of ED physicians (attendings and residents) with expertise in addiction research and implementation science, pharmacists, nurses, and social workers. This study was deemed exempt by our Institutional Review Board.

      Study Population

      The study population included adult patients who were at least 18 years old, discharged from the ED, and had a provider clinical impression (encounter diagnosis) and/or hospital billing diagnosis of opioid overdose or OUD based on ICD-10 codes T40.0*, T40.1*, T40.2*, T40.3*, T40.4*, T40.6*, and F11*. Patients admitted to the hospital from the ED were excluded.

      Primary Interventions

      (1) Focused ED Staff Education

      Several in-service education interventions designed to expand knowledge of OUD and harm reduction strategies based on information from freely publicly available resources

      American College of Emergency Physicians, Trauma & Injury Prevention Section. Emergency Department Naloxone Distribution: Key Considerations and Implementation Strategies. Samuels EA, et al. 2015. Accessed Mar 8, 2021. https://prescribetoprevent.org/wp2015/wp-content/uploads/TIPSWhitePaper.pdf.

      National Institute on Drug Abuse. Naloxone for Opioid Overdose: Life-Saving Science. Mar 2017. Accessed Mar 8, 2021. https://www.drugabuse.gov/publications/naloxone-opioid-overdose-life-saving-science.

      Substance Abuse and Mental Health Services Administration. Naloxone. (Updated: Aug 19, 2020.) Accessed Mar 8, 2021. https://www.samhsa.gov/medication-assisted-treatment/medications-counseling-related-conditions/naloxone.

      ,

      US Department of Health and Human Services. How to Respond to an Opioid Overdose. Sep 25, 2020. Accessed Mar 8, 2021. https://www.hhs.gov/opioids/treatment/overdose-response/index.html.

      were delivered to ED providers, nurses, and clinical support staff in various meetings, conferences, didactic seminars, and interactive learning opportunities. Educational content reflected the results of the ED staff needs assessment survey described below—in particular, the evidence base supporting naloxone prescribing.
      Lecture 1 occurred in September 2017, and Lecture 2 in August 2018, during the standard weekly ED resident education conference that is open to all ED providers, residents, and medical students. Lecture 1 was a one-hour session on current trends in opioid-related visits and complications across the nation and state, recognition and management of opioid overdose, harm reduction strategies, management of opioid withdrawal, and treatments available for OUD. Lecture 2 consisted of a three-hour symposium that included a one-hour lecture on similar material, as well as interactive sessions with both a panel of individuals with OUD in recovery and a panel of experts and community partners providing treatment to patients with OUD.
      Last, an ED staff-wide education series (Supplemental Figures 1 and 2, available in the appendix in online article) was delivered from November 2018 to January 2019 and included brief (5–10 minute) formal and informal slideshow presentations, trivia contests, printed materials hung in the department for perusal by all ED personnel, and structured small-group discussions at change-of-shift handoffs and other standard staff gatherings. All formal presentations were delivered at required staff and provider meetings; meeting minutes, trivia, and printed materials were e-mailed to all ED providers and staff for asynchronous review.

      (2) EMR–Based High-Reliability Work-Aids

      A BPA was created to trigger within the ED Discharge Navigator for all ED providers based on one of three criteria: (1) an ED chief complaint of “overdose”; (2) a provider-recorded clinical impression of opioid overdose or OUD matching ICD-10 codes T40.0*, T40.1*, T40.2*, T40.3*, T40.4*, T40.6*, or F11*; or (3) if naloxone was ordered for administration in the ED. The BPA opens a new window on the provider's screen with brief best practice information on naloxone prescribing for opioid overdose and a reminder with a link to open an ED naloxone order set (Supplemental Figure 3). The BPA was designed with a hard stop requiring selecting a reason if declining to open the order set. If the BPA is simply dismissed, it continues to fire each time the Discharge Navigator is opened until a naloxone prescription is placed.
      The ED naloxone order set is accessible from the BPA as well as through the standard EMR order entry workflow (Supplemental Figure 3). The order set includes a default set of printed prescriptions for naloxone at discharge. Prescriptions for four different formulations of naloxone are written to enable access to the patient's preferred naloxone product based on insurance coverage, cost, route of administration, and/or ease of use. New standardized discharge instructions are automatically included with easy-to-understand pictorial instructions on administering naloxone at home (Supplemental Figure 4) as well as updated information on community resources for OUD treatment, support services, and free naloxone. The order set also includes a nursing order to show the patient a short video on how to use naloxone and an option to consult with the ED social worker to discuss outpatient resources. The order set also enables ordering of naloxone for administration while in the ED, if needed.
      ED providers and staff were educated on both the BPA and the order set when they went live in March 2019. Off-service residents did not receive formal training, but the BPA with its link to the order set was still triggered for all patients with OUD or overdose discharged by any provider in the ED.

      Data Collection

      (1) Patient Characteristics and Naloxone Prescriptions

      For all patients meeting inclusion and exclusion criteria during the study period, EMR data were extracted for patient characteristics at the time of ED visit, including age, sex, method of payment, method of arrival, chief complaint, and clinical diagnosis. In addition, whether the patient received a naloxone prescription during their ED visit was coded as yes or no.

      (2) ED Staff Perceptions and Barriers Toward OUD and Naloxone

      A previously published survey instrument
      • Samuels EA
      • et al.
      Emergency department-based opioid harm reduction: moving physicians from willing to doing.
      was used to assess ED nurse and provider perceptions toward patients with OUD as well as ED staff perceived importance, confidence, willingness, and barriers to participating in opioid harm reduction strategies, chiefly naloxone prescribing. All survey items were scored on a five-point Likert scale, ranging from 1 (“strongly disagree”) to 5 (“strongly agree”) or from 1 (“not at all”) to 5 (“completely”). Surveys were administered anonymously over a one-month time frame to all ED providers in May 2018 and to nurses in September 2018, before the main staff education events, and compiled in a secure REDCap (Research Electronic Data Capture) database.
      • Harris PA
      • et al.
      Research Electronic Data Capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support.
      Surveys were repeated for both groups over a one-month time frame during March–April 2019 after the last education event.

      Analysis

      (1) Summary Statistics

      Patient characteristics were summarized for each year of the study period and presented using mean with standard deviation, median with interquartile range, or count and percentage as appropriate. The primary outcome was the monthly percentage of ED patients with OUD or opioid overdose meeting study criteria who were discharged from the ED with a prescription for naloxone.

      (2) Naloxone Prescribing Rates

      An autoregressive ITS model was used to model the proportion of ED patients with OUD or opioid overdose who received a naloxone prescription each month. This model allowed for adjustment for any overall trend in the prescription rate, as well as the potential impact of a variety of events and programs that could have affected prescription rates while estimating both the immediate impact of the BPA and order set along with any sustained change to the prescription rate over time. Backward elimination using the Durbin-Watson statistic was used to select the number of correlated lags, if any, to be included in the model. All ITS analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, North Carolina). Annotated monthly p-charts in QI Macros (KnowWare International Inc, Denver) using traditional rules for statistical process control
      • Provost LP
      • Murray SK.
      The Health Care Data Guide: Learning from Data for Improvement.
      were used to display the proportion of ED patients with OUD or opioid overdose who received naloxone prescriptions during the study period.

      (3) Impact on ED Staff Perceptions and Barriers Toward OUD and Naloxone

      Survey responses were analyzed in separate nursing (RN) and provider (MD, DO, and PA) cohorts. Descriptive statistics, including medians and interquartile ranges or means and standard deviations, were used to characterize staff demographic data and survey responses. Change in median score on each survey item before and three months after staff education was compared using Mann-Whitney U tests. Anonymization of the data and staff turnover prevented paired before-after analyses. Statistical analyses were performed using R, version 3.6.3 (R Foundation for Statistical Computing, Vienna).

      Additional QI Processes

      To identify potential reasons for lack of naloxone prescribing to patients with OUD or opioid overdose, we performed a manual chart review on two consecutive months of ED encounters (60 patients) at six months after the EMR work-aids went into effect. Pareto analysis was used to identify the most common reasons and group them into themes for future improvements to our intervention.
      To evaluate the impact of the ED naloxone BPA and order set on a more granular level, data were obtained from the BPA cube database available within the Epic EMR (Epic Systems Corporation, Verona, Wisconsin). These data included rates of BPA firing and subsequent end-user actions, including opening the order set, prescribing naloxone, and closing the BPA without opening the order set. These data were evaluated for additional reasons for not prescribing naloxone, as well as unintended triggers for BPA firing that could contribute to BPA fatigue and decrease order set use. In addition, overall order set use was captured from an EMR query that aggregates orders placed by their order set of origin and visualized using Tableau (Tableau Software, LLC, Seattle).

      Concurrent Intervention: Randomized Controlled Trial of Provider Prescribing Feedback

      Six months after our EMR work-aids went into effect, a separate unaffiliated randomized controlled trial (RCT) of clinician feedback on their opioid-related prescribing patterns was performed during a six-month period, September 2019 to March 2020. Approximately half of ED providers were randomized to the intervention and received monthly personalized reports via e-mail on their rates relative to peers of (1) opioid + benzodiazepine coprescribing, (2) opioid + skeletal muscle relaxant coprescribing, and (3) naloxone prescribing for patients with the same ICD-10 diagnosis codes as our study inclusion criteria to define the OUD and opioid overdose population.

      Results

      In the Duke University Emergency Department between January 1, 2017, and March 31, 2020, there were 1,426 encounters with provider clinical impression and/or hospital billing code of opioid overdose or OUD. Of these, 966 encounters occurred before the EMR work-aid go-live in March 2019, and 460 encounters occurred afterward. Patient characteristics over this time period are presented by year in Table 1.
      Table 1Patient Characteristics by Year During the Study Period, January 2017–March 2020
      2017

      (N = 489)
      2018

      (N = 426)
      2019

      (N = 408)
      2020

      (N = 103)
      Total

      (N = 1,426)
      Age
       Mean (SD)39.6 (13.4)38.8 (12.6)41.1 (13.4)40.3 (13.6)39.8 (13.2)
       Range(18.0–79.0)(18.0–72.0)(18.0–94.0)(18.0–78.0)(18.0–94.0)
       Missing20103
      Sex, n (%)
       Female208 (42.5)170 (39.9)168 (41.2)36 (35.0)582 (40.8)
       Male279 (57.1)256 (60.1)239 (58.6)67 (65.0)841 (59.0)
       Missing2 (0.4)0 (0.0)1 (0.2)0 (0.0)3 (0.2)
      Method of Payment, n (%)
       Medicaid130 (26.6)119 (27.9)101 (24.8)25 (24.3)375 (26.3)
       Medicare91 (18.6)49 (11.5)64 (15.7)15 (14.6)219 (15.4)
       Private Insurance75 (15.3)58 (13.6)44 (10.8)14 (13.6)191 (13.4)
       Self-Pay189 (38.7)190 (44.6)196 (48.0)44 (42.7)619 (43.4)
       Other4 (0.8)10 (2.3)3 (0.7)5 (4.9)22 (1.5)
      Method of Arrival, n (%)
       EMS220 (45.0)195 (45.8)221 (54.2)51 (49.5)687 (48.2)
       Private Vehicle157 (32.1)124 (29.1)101 (24.8)29 (28.2)411 (28.8)
       Walk-in61 (12.5)52 (12.2)52 (12.7)11 (10.7)176 (12.3)
       Law Enforcement45 (9.2)50 (11.7)31 (7.6)12 (11.7)138 (9.7)
       Other6 (1.2)5 (1.2)3 (0.7)0 (0.0)14 (1.0)
      Chief Complaint, n (%)
       Altered Mental Status8 (1.6)4 (0.9)14 (3.4)3 (2.9)29 (2.0)
       Drug Overdose104 (21.3)92 (21.6)111 (27.2)26 (25.2)333 (23.4)
       Withdrawal18 (3.7)13 (3.1)15 (3.7)3 (2.9)49 (3.4)
       Pain81 (16.6)69 (16.2)41 (10.0)14 (13.6)205 (14.4)
       Psychiatric119 (24.3)114 (26.8)78 (19.1)23 (22.3)334 (23.4)
       Other159 (32.5)134 (31.5)149 (36.5)34 (33.0)476 (33.4)
      N, number of encounters; SD, standard deviation; EMS, emergency medical services.

      (1) Naloxone Prescribing Rates

      Naloxone prescribing rates during the period of this study were tracked monthly using the annotated statistical process control chart (p-chart) in Figure 2. The baseline average monthly rate of naloxone prescribing to ED patients with OUD or opioid overdose from January to July 2017 was 1.5%. Special cause variation (8 points above center line) was observed during the period of the education interventions (shift from 1.5% to 6.3%) and after implementation of the EMR work-aids (shift from 6.3% to 28.7%).
      Figure 2:
      Figure 2This p-chart shows the monthly percentage of emergency department patients with opioid use disorder or opioid overdose who received a naloxone prescription from January 2017 to March 2020. UCL, upper control limit; CL, center line; OUD, opioid use disorder; BPA, Best Practice Advisory; RCT, randomized controlled trial.
      The results of the ITS analysis are displayed in Table 2. The ITS model was first adjusted for the two educational lectures, staff education event, prescribing feedback RCT, and trends over time. The implementation of the BPA and order set was associated with an immediate increase in the percentage of naloxone prescriptions by 21.1% (95% confidence interval [CI] = 5.9–36.4, p = 0.008), which correlates with the 22.4% relative increase observed in Figure 2, and was sustained for one year afterward. In this model, the education interventions, prescribing feedback RCT, and temporal trends did not contribute significantly to the naloxone prescribing rate.
      Table 2Interrupted Time Series Coefficient Estimates
      Estimates for the adjusted model include the BPA and order set, education intervention components, and concurrent feedback RCT. Estimates for the unadjusted model include only the naloxone BPA and order set. CI, confidence interval; OUD, opioid use disorder; BPA, Best Practice Advisory; RCT, randomized controlled trial.
      VariableAdjusted ModelUnadjusted Model
      Estimate (95% CI)P ValueEstimate (95% CI)P Value
      Intercept3.2 (-6.2–12.6)0.4942.3 (-2.1–6.8)0.296
      Time-0.4 (-2.5–1.7)0.6990.2 (-0.1–0.5)0.152
      OUD Lecture 16.0 (-4.3–16.2)0.243
      Time After OUD Lecture 10.4 (-1.9–2.7)0.717
      OUD Lecture 24.9 (-8.3–18.1)0.455
      Time After OUD Lecture 2-1.8 (-5.5–1.8)0.310
      Staff Education Event6.1 (-20.3–32.5)0.638
      Time After Staff Education Event1.5 (-14.7–17.7)0.848
      Order Set + BPA21.1 (5.9–36.4)0.00814.6 (6.9–22.3)0.001
      Time After Order Set + BPA-0.9 (-17.0–15.1)0.9040.8 (0.0–1.7)0.064
      Start of Feedback RCT10.9 (-1.5–23.3)0.083
      Time After Start of Feedback RCT1.7 (-1.7–5.1)0.326
      low asterisk Estimates for the adjusted model include the BPA and order set, education intervention components, and concurrent feedback RCT. Estimates for the unadjusted model include only the naloxone BPA and order set.CI, confidence interval; OUD, opioid use disorder; BPA, Best Practice Advisory; RCT, randomized controlled trial.
      This analysis was repeated without adjusting for the nonsignificant interventions. In the unadjusted ITS model, implementation of the BPA and order set was associated with a 14.6% (95% CI = 6.9–22.3, p < 0.001) increase in the percentage of patients receiving naloxone prescriptions (Table 2). In this unadjusted model, there were no significant temporal or secular trends over time; however, the combined effects of the nonsignificant interventions have reduced the attributable contribution of the EMR work-aids to naloxone prescribing when compared to the results of the adjusted model.

      (2) ED Staff Perceptions and Barriers

      Surveys were completed by 74.4% of ED providers (61 of 82 eligible MDs, DOs, and PAs) and 83.1% of ED nurses (98 of 118 eligible nurses) before education, and by 52.4% (n = 43) and 58.5% (n = 69), respectively, after education. Survey respondent characteristics are presented in Supplemental Table 1 (available in online article). Baseline responses demonstrated high willingness among ED providers but relatively lower willingness among ED nurses to implement opioid harm reduction strategies (Tables 3 and 4). However, more than 70% of respondents from both groups agreed at least somewhat that a lack of knowledge of the evidence was a barrier to prescribing naloxone.
      Table 3Comparison of Survey Responses Among Provider Cohort
      Preintervention surveys administered in May 2018; postintervention surveys administered March 26, 2019, to April 25, 2019. Likert scale ranges from 1 (“not at all”) to 5 (“completely”) or from 1 (“strongly disagree”) to 5 (“strongly agree”).
      Provider Survey ItemPreintervention Mean (SD) (n = 61)Postintervention Mean (SD) (n = 43)P Value for Median Difference
      Bold indicates statistical significance. SD, standard deviation; n, number of respondents; ED, emergency department.
      1. How important is it for the ED to:

       a. Provide naloxone prescriptions to opioid users

       b. Screen for and educate about substance use

       c. Facilitate referrals to detoxification programs


      4.21 (1.04)

      4.28 (1.09)

      4.54 (0.84)


      4.35 (1.03)

      4.30 (1.21)

      4.44 (1.00)


      0.461

      0.710

      0.715
      2. How confident are you in your ability to:

       a. Provide substance use screening and referral

       b. Provide brief substance use counseling

       c. Prescribe naloxone in the ED

       d. Refer to an opioid treatment program


      2.67 (0.94)

      2.90 (0.97)

      2.80 (1.14)

      1.92 (0.89)


      3.00 (0.89)

      3.14 (0.95)

      3.58 (1.08)

      2.35 (1.14)


      0.097

      0.175

      0.001

      0.068
      3. How willing are you to:

       a. Provide substance use screening and referral

       b. Provide brief substance use counseling

       c. Prescribe naloxone in the ED

       d. Refer to an opioid treatment program


      3.77 (0.96)

      3.82 (0.98)

      4.07 (1.02)

      3.89 (0.94)


      4.30 (0.88)

      4.23 (1.10)

      4.26 (0.97)

      4.21 (0.90)


      0.003

      0.011

      0.270

      0.067
      4. To what extent do you agree or disagree that:

       a. Addiction is a chronic medical illness

       b. Addiction results from changes in brain neurocircuitry

       c. Addiction is influenced by psychological factors

       d. Addiction is influenced by environmental factors


      4.25 (0.69)

      4.34 (0.67)

      4.56 (0.53)

      4.49 (0.56)


      4.30 (0.85)

      4.49 (0.54)

      4.72 (0.50)

      4.65 (0.48)


      0.405

      0.350

      0.088

      0.167
      5. To what extent do you agree or disagree that:

       a. I feel that there is little I can do to help drug users

       b. I am able to work with drug users as effectively as others

       c. I am inclined to feel like a failure working with drug users

       d. In general, I have less respect for drug users than others

       e. I often feel uncomfortable when working with drug users

       f. In general, I feel I can understand drug users


      2.70 (0.91)

      2.84 (0.89)

      2.51 (0.84)

      2.61 (1.00)

      2.31 (0.86)

      3.23 (0.86)


      2.53 (0.97)

      2.95 (1.18)

      2.40 (1.01)

      2.37 (0.92)

      2.37 (0.92)

      3.35 (0.91)


      0.330

      0.631

      0.379

      0.235

      0.727

      0.662
      6. How prohibitive do you find the following barriers to incorporating naloxone prescribing in your ED practice:

       a. Lack of time during the clinical encounter

       b. Lack of adequate training to prescribe naloxone

       c. Limited knowledge of research for naloxone prescribing

       d. Lack of hospital/ED administrative support

       e. Concern about the legal aspects of naloxone prescribing




      2.59 (1.06)

      2.98 (1.08)

      2.34 (1.08)

      2.64 (1.17)

      1.97 (1.16)




      2.70 (1.05)

      2.19 (1.17)

      1.74 (1.01)

      1.81 (1.02)

      1.58 (0.97)




      0.567

      0.001

      0.003

      < 0.001

      0.072
      low asterisk Preintervention surveys administered in May 2018; postintervention surveys administered March 26, 2019, to April 25, 2019. Likert scale ranges from 1 (“not at all”) to 5 (“completely”) or from 1 (“strongly disagree”) to 5 (“strongly agree”).
      Bold indicates statistical significance.SD, standard deviation; n, number of respondents; ED, emergency department.
      Table 4Comparison of Survey Responses Among Nursing Cohort*
      Nursing Survey ItemPreintervention Mean (SD) (n = 98)Postintervention Mean (SD) (n = 69)P Value for Median Difference
      Bold indicates statistical significance. SD, standard deviation; n, number of respondents; ED, emergency department.
      1. How important is it for the ED to:

       a. Provide naloxone prescriptions to opioid users

       b. Screen for and educate about substance use

       c. Facilitate referrals to detoxification programs


      3.54 (1.45)

      3.95 (1.22)

      4.14 (1.14)


      4.22 (1.28)

      4.48 (1.00)

      4.48 (1.00)


      0.001

      0.003

      0.038
      2. How confident are you in your ability to:

       a. Screen patients for risks of substance use

       b. Facilitate referral to treatment for substance use

       c. Educate patients on how to use naloxone in the ED

       d. Recommend that a provider prescribe naloxone

       e. Facilitate referral to an opioid treatment program


      3.54 (1.05)

      2.95 (1.39)

      3.59 (1.30)

      3.41 (1.30)

      2.90 (1.36)


      3.43 (1.16)

      2.90 (1.24)

      3.67 (1.10)

      3.71 (1.12)

      2.88 (1.17)


      0.708

      0.960

      0.959

      0.175

      0.918
      3. How willing are you to:

       a. Screen patients for risks of substance use

       b. Facilitate referral to treatment for substance use

       c. Educate patients on how to use naloxone in the ED

       d. Recommend that a provider prescribe naloxone

       e. Facilitate referral to an opioid treatment program


      3.70 (1.17)

      3.66 (1.21)

      3.77 (1.28)

      3.42 (1.32)

      3.51 (1.26)


      3.96 (1.06)

      3.86 (1.11)

      3.99 (1.15)

      3.81 (1.20)

      3.77 (1.14)


      0.176

      0.337

      0.312

      0.055

      0.203
      4. To what extent do you agree or disagree that:

       a. Addiction is a chronic medical illness

       b. Addiction results from changes in brain neurocircuitry

       c. Addiction is influenced by psychological factors

       d. Addiction is influenced by environmental factors


      3.82 (1.15)

      3.82 (0.97)

      4.33 (0.61)

      4.35 (0.72)


      4.13 (1.06)

      4.06 (0.94)

      4.41 (0.71)

      4.43 (0.69)


      0.045

      0.071

      0.233

      0.516
      5. To what extent do you agree or disagree that:

       a. I feel that there is little I can do to help drug users

       b. I am able to work with drug users as effectively as others

       c. I am inclined to feel like a failure working with drug users

       d. In general, I have less respect for drug users than others

       e. I often feel uncomfortable when working with drug users

       f. In general, I feel I can understand drug users


      2.86 (0.96)

      3.70 (1.03)

      2.06 (0.78)

      2.27 (1.04)

      2.02 (0.87)

      3.04 (0.87)


      2.78 (0.90)

      3.54 (1.08)

      2.01 (0.74)

      2.03 (0.95)

      2.15 (0.96)

      3.00 (0.79)


      0.671

      0.349

      0.785

      0.149

      0.492

      0.761
      6. How prohibitive do you find the following barriers to recommending naloxone in your routine ED practice:

       a. Lack of time during the clinical encounter

       b. Lack of adequate training to prescribe naloxone

       c. Limited knowledge of research for naloxone prescribing

       d. Lack of hospital/ED administrative support

       e. Concern about the legal aspects of naloxone prescribing




      2.57 (1.08)

      2.61 (1.15)

      2.37 (1.13)

      2.21 (1.14)

      2.47 (1.21)


      2.77 (1.05)

      2.53 (1.20)

      2.18 (1.16)

      2.09 (1.17)

      2.15 (1.13)


      0.324

      0.627

      0.281

      0.476

      0.199
      Preintervention surveys administered in September 2018; postintervention surveys administered March 26, 2019, to April 25, 2019. Likert scale ranges from 1 (“not at all”) to 5 (“completely”) from or 1 (“strongly disagree”) to 5 (“strongly agree”).
      Bold indicates statistical significance.SD, standard deviation; n, number of respondents; ED, emergency department.
      Three months following completion of the education interventions, ED providers reported significantly more confidence in their ability to prescribe naloxone (p = 0.001) and a reduction in their perceived barriers, such as inadequate training (p = 0.001) and knowledge gaps on the evidence for naloxone prescribing (p = 0.003) (Table 3). ED nurses reported significant increases in the perceived importance of prescribing naloxone in the ED (p = 0.001), screening and educating patients about substance use (p = 0.003), and referring patients with OUD to treatment programs (p = 0.038) (Table 4).

      (3) Additional Factors Affecting Naloxone Prescribing

      Chart review revealed that the most common reason patients were not prescribed naloxone was an ED visit for a complaint unrelated to opioid overdose. Additional reasons for not prescribing naloxone were identified and grouped thematically for Pareto analysis (Figure 3) and included: (1) multiple handoffs among providers, (2) patients with additional psychiatric complaints evaluated and managed primarily by the psychiatry service, (3) primary management by a nonemergency medicine “off-service” resident, and (4) “overdose” not formally documented by the ED provider despite emergency medical services or nursing notes indicating bystander reports of overdose. These five circumstances accounted for 79.5% of the instances in which an ED patient with OUD or opioid overdose did not receive a naloxone prescription.
      Figure 3:
      Figure 3This Pareto analysis graph displays reasons for failure to receive a naloxone prescription (n = 60; chart review time frame = July–August 2019). ED, emergency department.
      EMR work-aid usage data showed that 23.7% of fired BPAs (monthly range 14.1%–30.2%) resulted in opening of the ED naloxone order set, and 10.2% of fired BPAs (monthly range 5.7%–15.1%) resulted in actual naloxone prescriptions written. On review of overall BPA firing outcomes, one reason identified for apparent underutilization of the ED naloxone order set was activation of the BPA by “as needed” (PRN) naloxone orders included in patient-controlled opioid analgesia order sets to be used as an opioid reversal agent if needed for accidental overmedication or iatrogenic overdose.

      Discussion

      Implementation of EMR–based work-aids, including a Best Practice Advisory (BPA) and linked order set for prescribing naloxone resulted in a significant immediate increase in naloxone prescribing rates for ED patients with OUD or opioid overdose that was sustained for one year afterward. Although the targeted education interventions did not directly produce measurable effects, it is possible that they contributed to the effectiveness of the EMR work-aids through improving both ED nursing perceived importance of and ED provider confidence in prescribing naloxone, while reducing perceived barriers to naloxone prescribing. Furthermore, Pareto analysis attributing a significant number of missed opportunities for naloxone prescribing to “off-service” non-ED providers who did not receive this education suggests the necessity of education for increasing provider confidence in and use of the BPA and order set. The strength of using ITS is that the model is analyzed with and affected by time-varying or global confounders, including those events that occur close in time to the intervention of interest (that is, EMR work-aid go-live).
      • Bernal JL
      • Cummins S
      • Gasparrini A.
      Interrupted time series regression for the evaluation of public health interventions: a tutorial.
      Hence, our finding that the adjusted ITS model yielded a larger change in prescribing rates than the unadjusted model is consistent with ITS data analysis recommendations
      • Bernal JL
      • Cummins S
      • Gasparrini A.
      Interrupted time series regression for the evaluation of public health interventions: a tutorial.
      and reflects the fact that the contemporaneous events (ED staff education and feedback RCT) that occurred around the same time as the EMR work-aid intervention influence outcomes.
      Preliminary work by two other groups also found that a BPA recommending naloxone at discharge for ED patients with chief complaints related to OUD increased naloxone prescribing rates from 3.1% to 13.5%,
      • Hussain S.
      A multi-modal quality- improvement study of strategies for enhancing adoption of naloxone- prescribing clinical decision support (master's thesis).
      or increased dispensing of naloxone kits from 13% to 22.6%.
      • Marino R
      • Lynch M
      • Suffoletto B.
      Increasing utilization of take-home naloxone program for at-risk emergency department patients using computerized clinical decision support.
      Another group found that implementation of an EMR order set increased naloxone prescribing rates from 6.5% to 11.5%.
      • Holland WC
      • et al.
      Interrupted time series of user-centered clinical decision support implementation for emergency department–initiated buprenorphine for opioid use disorder.
      Our group achieved larger increases in naloxone prescribing rates from 5.1% to 28.7% after combined BPA and order set implementation. This greater difference may be due to our targeted education, overall increased awareness of the opioid crisis, and the concurrent implementation of a linked BPA and order set to facilitate writing of naloxone prescriptions. The use of order sets has been shown to both decrease the time to order medications and improve outcomes.
      • Gray J
      • Hutchinson J.
      How can electronic order sets reduce time taken to prescribe medications on admission to PICU?.
      ,
      • Micek ST
      • et al.
      Before-after study of a standardized hospital order set for the management of septic shock.
      For instance, an order set developed for ED patients meeting sepsis criteria led to more rigorous fluid resuscitation, greater administration of appropriate antibiotic treatment, and a lower 28-day mortality.
      • Micek ST
      • et al.
      Before-after study of a standardized hospital order set for the management of septic shock.
      Some important features of our EMR–based intervention, such as its simplicity, ease of implementation, and ease of use, make it easily transferrable to other settings in which financial and institutional barriers may prevent more complex or comprehensive interventions for OUD. Specifically, the BPA triggers encompass a very limited set of diagnosis codes, chief complaints, and end-user actions that are specific to OUD and opioid overdose, making it simple and inexpensive for EMR builders to code and increasing end-user acceptability. Similarly, the order set is fairly circumscribed, making it easy to build, navigate, and use even without provider education. Within our own institution, these features enabled the adaptation of the BPA and order set to the inpatient hospital environment. Furthermore, incorporation of newer and often more challenging evidence-based treatments, such as initiation of medication treatments for OUD (for example, buprenorphine) from the ED, can be facilitated with similar EMR–based work-aids.
      Our education interventions likewise have a low barrier to implementation due to the availability of evidence-based content through freely accessible resources,

      American College of Emergency Physicians, Trauma & Injury Prevention Section. Emergency Department Naloxone Distribution: Key Considerations and Implementation Strategies. Samuels EA, et al. 2015. Accessed Mar 8, 2021. https://prescribetoprevent.org/wp2015/wp-content/uploads/TIPSWhitePaper.pdf.

      National Institute on Drug Abuse. Naloxone for Opioid Overdose: Life-Saving Science. Mar 2017. Accessed Mar 8, 2021. https://www.drugabuse.gov/publications/naloxone-opioid-overdose-life-saving-science.

      Substance Abuse and Mental Health Services Administration. Naloxone. (Updated: Aug 19, 2020.) Accessed Mar 8, 2021. https://www.samhsa.gov/medication-assisted-treatment/medications-counseling-related-conditions/naloxone.

      ,

      US Department of Health and Human Services. How to Respond to an Opioid Overdose. Sep 25, 2020. Accessed Mar 8, 2021. https://www.hhs.gov/opioids/treatment/overdose-response/index.html.

      as well as the feasibility of incorporating brief interactive learning into existing ED staff meetings such as change of shift. After only a handful of brief education sessions over the course of a couple of months, we observed improvements in both ED provider confidence and willingness and nursing staff perceived importance toward harm reduction and naloxone prescribing for OUD three months later. Similarly, after implementation of an opioid education program in the US Department of Veterans Affairs health care system, prescribers who received the training reported feeling significantly more knowledgeable about and comfortable with prescribing naloxone compared to those who had not been trained.
      • Peckham AM
      • et al.
      A survey of prescribers’ attitudes, knowledge, comfort, and fear of consequences related to an opioid overdose education and naloxone distribution program.
      Other studies have shown similar baseline high willingness but lower levels of confidence among ED providers to prescribe naloxone,
      • Samuels EA
      • et al.
      Emergency department-based opioid harm reduction: moving physicians from willing to doing.
      ,
      • Lacroix L
      • et al.
      Emergency physicians’ attitudes and perceived barriers to the implementation of take-home naloxone programs in Canadian emergency departments.
      further emphasizing the importance of including brief education to reduce these perceived barriers and facilitate naloxone prescribing.
      Some groups have advocated that take-home naloxone kits may be more effective than naloxone prescriptions in preventing opioid overdose deaths, as it removes a potential barrier to obtaining naloxone. However, data on the mortality benefit of either of these interventions remain extremely limited and are based on indirect evidence through population-level associations with communitywide implementation efforts.
      • Gunn AH
      • et al.
      The emergency department as an opportunity for naloxone distribution.
      Both naloxone prescribing and naloxone dispensing have been associated with improved outcomes among people with OUD.
      • Walley AY
      • et al.
      Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis.
      • Clark AK
      • Wilder CM
      • Winstanley EL.
      A systematic review of community opioid overdose prevention and naloxone distribution programs.
      • Maxwell S
      • et al.
      Prescribing naloxone to actively injecting heroin users: a program to reduce heroin overdose deaths.
      • Dwyer K
      • et al.
      Opioid education and nasal naloxone rescue kits in the emergency department.
      Furthermore, as many as 76% of primary care patients who received naloxone prescriptions picked up the medication from their pharmacy,
      • Watson A
      • Guay K
      • Ribis D.
      Assessing the impact of clinical pharmacists on naloxone coprescribing in the primary care setting.
      and patients with OUD were more likely than those without OUD to fill their naloxone prescription.
      • Lin LA
      • et al.
      Association of opioid overdose risk factors and naloxone prescribing in US adults.
      Thus, implementation of a simple EMR–based intervention and targeted education serves as a feasible, sustainable, and efficacious way for EDs to address the opioid epidemic and improve provision of naloxone.

      Lessons Learned for Improving the Intervention

      Our postintervention process review through chart review, BPA and order set usage data, and Pareto analysis revealed several lessons learned. The first is that the BPA and education intervention should include the importance of prescribing naloxone to all patients with OUD regardless of reason for ED visit. At the start of our intervention, we focused on increasing naloxone prescribing primarily for patients with opioid overdose due to the initial ED workflow and knowledge barriers to wider naloxone prescribing. This limited focus likely contributed to the lower-than-expected 28.7% postintervention prescribing rate. Since then, increased ED provider awareness, acceptability, and feasibility through this initial work has enabled subsequent updates advocating for broader naloxone prescribing to all ED patients with OUD.
      The second lesson is that the BPA should exclude PRN naloxone orders as a trigger. Because these orders are typically used for in-hospital or in-ED opioid-based pain treatment that may not reflect outpatient opioid exposure or overdose risk, eliminating PRN naloxone as a trigger will help reduce BPA fatigue and order set underutilization.
      The third lesson is that transitions of ED care, specifically shift change and handoff with non-ED providers, which are typically high-risk times for patient care,
      • Riesenberg LA
      • et al.
      Residents’ and attending physicians’ handoffs: a systematic review of the literature.
      are particularly problematic for patients with OUD. Important discharge interventions, including naloxone prescribing, may be incompletely communicated between the off-going and on-coming teams and may have contributed to the lower than desired prescribing rates. BPA settings should be used that ensure reactivation of the BPA by the final treatment team at the time of ED discharge. One option could be to add a decline reason of “patient too altered,” which is set to only temporarily suppress the BPA for two hours and reactivate later when the patient is more able to engage in naloxone education.

      Limitations

      Limitations to this QI study include a prescribing feedback RCT that ran concurrently with the last six months of the intervention study period. However, despite the potential confounding effect, the ITS analysis found no measurable impact from this intervention on naloxone prescribing rates. Although it may have contributed to sustaining the BPA–order set improvements through continued awareness and education similar to our education intervention, it was delivered to only half of ED providers (with the other half in the control “no feedback” group) and none of the “off-service” physicians, further diminishing this possible contribution.
      Another possible confounding factor is the increased global awareness of the opioid epidemic across the duration of the study period. Since 2017, multiple organizations, including the World Health Organization and the American College of Emergency Physicians, have released policy statements encouraging naloxone prescriptions and take-home naloxone. However, we note that there was no significant trend over time before or after implementation of the EMR–based work-aids, suggesting general increased awareness was not a significant contributor to naloxone prescribing rates.
      Using ICD-10 diagnosis codes to trigger the BPA and order set is limited by the capacity, reliability, and timeliness of ED providers coding for opioid overdose and OUD in their diagnosis documentation. Substance use disorders and overdoses are known to be undercoded in ED visits, leading to decreased capture of the full at-risk population.
      • Rowe C
      • et al.
      Performance measures of diagnostic codes for detecting opioid overdose in the emergency department.
      Similarly during our chart review, we noted that restricting inclusion criteria to opioid overdose diagnosis codes only (T40*) compared to all diagnosis codes for OUD (T40* plus F11*) would miss more than 50% of ED visits for opioid overdose. This was one reason that ordering naloxone in the ED was included as a trigger for the BPA. Future work should include increased screening and documentation of patients with OUD (including possible additional BPAs for these steps) to broaden access to important lifesaving interventions like naloxone.
      Response rates to the ED staff survey postintervention were approximately 25% lower than baseline. However, our overall response rates in both cohorts before and after the intervention were still high (> 50%) relative to typical medical provider and nursing survey response rates,
      • Grava-Gubins I
      • Scott S.
      Effects of various methodologic strategies: survey response rates among Canadian physicians and physicians-in-training.
      ,
      • VanGeest J
      • Johnson TP.
      Surveying nurses: identifying strategies to improve participation.
      suggesting that our survey results remain robust. Although the posteducation survey occurred during the first month of EMR work-aid activation, use of the order set to prescribe naloxone was unlikely to have contributed to the increased confidence in prescribing naloxone, as 43 providers and 69 nurses responded, but only 7 naloxone prescriptions were written that month. In addition, we noted change in some but not all areas of the survey after the education intervention (Tables 3 and 4). Lack of change in other domains may have been due to our primary focus on addressing barriers specific to naloxone prescribing as well as the baseline survey occurring after Lecture 1.
      Finally, this study was performed at a single site only, potentially limiting the external validity of these findings. However, the broad availability of each of our intervention components—specifically, the EMR–based work-aids are available through a commonly used EMR platform (Epic), and the education resources are available through national organization websites—increases the likelihood of generalizability to other sites.

      Conclusion

      Our findings support that ED providers and nurses are willing to prescribe naloxone to ED patients at risk for opioid overdose and that prescribing rates can be improved through a combination of targeted education and highly reliable EMR–based work-aids, including a BPA and order set for ED prescribing of naloxone. These interventions are simple, feasible, easy to use, and effective at improving prescribing rates. The spread of similar interventions to other ED settings may be key to wider staff and provider engagement in harm reduction strategies for opioid use disorder.

      Funding

      The Duke Biostatistics, Epidemiology, and Research Design Core's support was made possible by the Clinical and Translational Science Awards (CTSA) Grant (UL1TR002553) from the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH) and the NIH Roadmap for Medical Research. This report's contents are solely the responsibility of the authors and do not represent the official views of NCATS or NIH.

      Acknowledgments

      The authors would like to thank Clay Musser, MD, and David Chermak for their assistance with data acquisition; Farah Davis, LCSW, for her help with patient education materials; Madeline Thornton and the Bass Connections team for their work on provider and nursing education materials; and Erin Hall for her assistance with copyediting.

      Conflicts of Interest

      All authors report no conflicts of interest.

      Appendix. SUPPLEMENTARY MATERIALS

      • Supplemental Figure 2: Trivia contest questions and answers printed for review in the emergency department (ED) and distributed by e-mail to all ED personnel for the ED staff-wide education series.

      • Supplemental Figure 3: Emergency department (ED) provider and nursing instructions with screen shots for the ED naloxone order set, Best Practice Advisory, and patient educational materials.

      • Supplemental Table 1. Demographics of respondents to emergency department survey on opioid use disorder perceptions and barriers before the education intervention. All table values are listed as n (%). n, number of respondents.

      • Supplemental Table 2. Demographics of respondents to emergency department survey on opioid use disorder perceptions and barriers after the education intervention. All table values are listed as n (%). n, number of respondents.

      References

        • Vivolo-Kantor AM
        • et al.
        Nonfatal drug overdoses treated in emergency departments—United States, 2016–2017.
        MMWR Morb Mortal Wkly Rep. 2020 Apr 3; 69: 371-376
        • Wilson N
        • et al.
        Drug and opioid-involved overdose deaths—United States, 2017–2018.
        MMWR Morb Mortal Wkly Rep. 2020 Mar 20; 69: 290-297
        • Weiner SG
        • et al.
        One-year mortality of patients after emergency department treatment for nonfatal opioid overdose.
        Ann Emerg Med. 2020; 75: 13-17
      1. North Carolina Department of Health and Human Services, Injury and Violence Prevention Branch. North Carolina Emergency Department (ED) Visits for Opioid Overdoses: December 2020. Jan 13, 2021. Accessed Mar 8, 2021. https://www.injuryfreenc.ncdhhs.gov/DataSurveillance/StatewideOverdoseSurveillanceReports/OpioidOverdoseEDVisitsMonthlyReports/StatewideOpioidOverdoseSurveillanceEDData-Dec2020.pdf.

        • Walley AY
        • et al.
        Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis.
        BMJ. 2013 Jan 30; 346: f174
        • Clark AK
        • Wilder CM
        • Winstanley EL.
        A systematic review of community opioid overdose prevention and naloxone distribution programs.
        J Addict Med. 2014; 8: 153-163
        • Maxwell S
        • et al.
        Prescribing naloxone to actively injecting heroin users: a program to reduce heroin overdose deaths.
        J Addict Dis. 2006; 25: 89-96
        • Dwyer K
        • et al.
        Opioid education and nasal naloxone rescue kits in the emergency department.
        West J Emerg Med. 2015; 16: 381-384
      2. American College of Emergency Physicians, Trauma & Injury Prevention Section. Emergency Department Naloxone Distribution: Key Considerations and Implementation Strategies. Samuels EA, et al. 2015. Accessed Mar 8, 2021. https://prescribetoprevent.org/wp2015/wp-content/uploads/TIPSWhitePaper.pdf.

      3. National Institute on Drug Abuse. Naloxone for Opioid Overdose: Life-Saving Science. Mar 2017. Accessed Mar 8, 2021. https://www.drugabuse.gov/publications/naloxone-opioid-overdose-life-saving-science.

      4. Substance Abuse and Mental Health Services Administration. Naloxone. (Updated: Aug 19, 2020.) Accessed Mar 8, 2021. https://www.samhsa.gov/medication-assisted-treatment/medications-counseling-related-conditions/naloxone.

      5. US Food and Drug Administration. New Recommendations for Naloxone. Jul 23, 2020. Accessed Mar 8, 2021. https://www.fda.gov/drugs/drug-safety-and-availability/new-recommendations-naloxone.

        • Eswaran V
        • et al.
        Take-home naloxone program implementation: lessons learned from seven Chicago-area hospitals.
        Ann Emerg Med. 2020; 76: 318-327
        • Gunn AH
        • et al.
        The emergency department as an opportunity for naloxone distribution.
        West J Emerg Med. 2018; 19: 1036-1042
        • Drainoni M-L
        • et al.
        Why is it so hard to implement change? A qualitative examination of barriers and facilitators to distribution of naloxone for overdose prevention in a safety net environment.
        BMC Res Notes. 2016 Oct 18; 9: 465
        • Marino R
        • et al.
        Do electronic health record prompts increase take-home naloxone administration for emergency department patients after an opioid overdose?.
        Addiction. 2019; 114: 1575-1581
        • Holland WC
        • et al.
        Interrupted time series of user-centered clinical decision support implementation for emergency department–initiated buprenorphine for opioid use disorder.
        Acad Emerg Med. 2020; 27: 753-763
        • Eswaran V
        • et al.
        Development of a take-home naloxone program at an urban academic emergency department.
        J Am Pharm Assoc (2003). 2020; 60: e324-e331
        • Samuels EA
        • et al.
        Adoption and utilization of an emergency department naloxone distribution and peer recovery coach consultation program.
        Acad Emerg Med. 2019; 26: 160-173
        • Hussain S.
        A multi-modal quality- improvement study of strategies for enhancing adoption of naloxone- prescribing clinical decision support (master's thesis).
        Johns Hopkins University, 2019 (Accessed Mar 8, 2021 (abstract))
      6. US Department of Health and Human Services. How to Respond to an Opioid Overdose. Sep 25, 2020. Accessed Mar 8, 2021. https://www.hhs.gov/opioids/treatment/overdose-response/index.html.

        • Samuels EA
        • et al.
        Emergency department-based opioid harm reduction: moving physicians from willing to doing.
        Acad Emerg Med. 2016; 23: 455-465
        • Harris PA
        • et al.
        Research Electronic Data Capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inform. 2009; 42: 377-381
        • Provost LP
        • Murray SK.
        The Health Care Data Guide: Learning from Data for Improvement.
        Jossey-Bass, San Francisco2011
        • Bernal JL
        • Cummins S
        • Gasparrini A.
        Interrupted time series regression for the evaluation of public health interventions: a tutorial.
        Int J Epidemiol. 2017 Feb 1; 46: 348-355
        • Marino R
        • Lynch M
        • Suffoletto B.
        Increasing utilization of take-home naloxone program for at-risk emergency department patients using computerized clinical decision support.
        Ann Emerg Med. 2018; 72: S153
        • Gray J
        • Hutchinson J.
        How can electronic order sets reduce time taken to prescribe medications on admission to PICU?.
        Arch Dis Child. 2019; 104 (e2.8)
        • Micek ST
        • et al.
        Before-after study of a standardized hospital order set for the management of septic shock.
        Crit Care Med. 2006; 34: 2707-2713
        • Peckham AM
        • et al.
        A survey of prescribers’ attitudes, knowledge, comfort, and fear of consequences related to an opioid overdose education and naloxone distribution program.
        J Public Health Manag Pract. 2018; 24: 310-317
        • Lacroix L
        • et al.
        Emergency physicians’ attitudes and perceived barriers to the implementation of take-home naloxone programs in Canadian emergency departments.
        CJEM. 2018; 20: 46-52
        • Watson A
        • Guay K
        • Ribis D.
        Assessing the impact of clinical pharmacists on naloxone coprescribing in the primary care setting.
        Am J Health Syst Pharm. 2020 Mar 24; 77: 568-573
        • Lin LA
        • et al.
        Association of opioid overdose risk factors and naloxone prescribing in US adults.
        J Gen Intern Med. 2020; 35: 420-427
        • Riesenberg LA
        • et al.
        Residents’ and attending physicians’ handoffs: a systematic review of the literature.
        Acad Med. 2009; 84: 1775-1787
        • Rowe C
        • et al.
        Performance measures of diagnostic codes for detecting opioid overdose in the emergency department.
        Acad Emerg Med. 2017; 24: 475-483
        • Grava-Gubins I
        • Scott S.
        Effects of various methodologic strategies: survey response rates among Canadian physicians and physicians-in-training.
        Can Fam Physician. 2008; 54: 1424-1430
        • VanGeest J
        • Johnson TP.
        Surveying nurses: identifying strategies to improve participation.
        Eval Health Prof. 2011; 34: 487-511