Advertisement

Improving Handoff by Deliberate Cognitive Processing: Results from a Randomized Controlled Experimental Study

Open AccessPublished:November 17, 2020DOI:https://doi.org/10.1016/j.jcjq.2020.11.008

      ABSTRACT

      Background

      Although a number of successful handoff interventions have been reported, the handoff process remains vulnerable because it relies on memory. The aim of this study was to investigate the effect of deliberate cognitive processing (i.e., analytical, conscious, and effortful thinking) on recall of information from a simulated handoff.

      Methods

      This two-phased experiment was executed in the Netherlands in 2015. A total of 78 pediatric residents were randomly divided into an intervention group (n = 37) and a control group (n = 41). In phase 1, participants received written handoffs from 8 patients. The intervention group was asked to develop a contingency plan for each patient, deliberately processing the information. The control group received no specific instructions. In phase 2, all participants were asked to write down as much as they recalled from the handoffs. The outcome was the amount and accuracy of recalled information, calculated by scoring for idea units (single information elements) and inferences (conclusions computed by participants based on two or more idea units).

      Results

      Participants in the intervention group recalled significantly more inferences (7.24 vs. 3.22) but fewer correct idea units (21.1% vs. 25.3%) than those in the control group. There was no difference with regard to incorrectly recalled information.

      Conclusion

      Our study revealed that deliberate cognitive processing leads to creation of more correct inferences, but fewer idea units. This suggests that deliberate cognitive processing results in interpretation of the information into higher level concepts, rather than remembering specific pieces of information separately. This implies better understanding of patients’ problems.
      Handoffs pose a serious threat to patient safety. Handoff failures can lead to incorrect treatment plans, diagnostic or therapeutic delays, adverse events, patient complaints, increased length of stay, and potentially to preventable morbidity and mortality.
      • Kluger M.T.
      • Bullock M.F.
      Recovery room incidents: a review of 419 reports from the Anaesthetic Incident Monitoring Study (AIMS).
      As the number of health care providers involved in a patient's care increases, so does the number of handoffs.
      • Leape L.L.
      • et al.
      The Nature of Adverse Events in Hospitalized-Patients - Results of the Harvard Medical-Practice Study-Ii.
      Handoffs can take place verbally in face-to-face situations between health care providers, through a written document, or through a combination of the two.
      • Cook R.I.
      • Render M.
      • Woods D.D.
      Gaps in the continuity of care and progress on patient safety.
      It is a challenging task to efficiently transfer all relevant information while being careful to avoid information overload.
      • Riesenberg L.A.
      • et al.
      Residents' and Attending Physicians' Handoffs: A Systematic Review of the Literature.
      ,
      • Arora V.
      • et al.
      Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis.
      Reasons for errors in handoff procedures include difficulties with keeping patient information up to date due to unavoidable daily changes and a lack of formal instruction, supervision, or feedback on handoff performance.
      • Arora V.
      • et al.
      Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis.
      Since handoff has become a priority in multiple patient safety programs, considerable clinical and scientific attention has been given to improve the handoff procedure.
      • Wohlauer M.V.
      • et al.
      The Patient Handoff: A Comprehensive Curricular Blueprint for Resident Education to Improve Continuity of Care.
      Various studies have tested the effect of one specific intervention (e.g., the SBAR handover tool) or tested the effect of the implementation of bundles consisting of multiple interventions, such as the I-PASS bundle.
      • Abraham J.
      • et al.
      Comparative evaluation of the content and structure of communication using two handoff tools: Implications for patient safety.
      • Panesar R.S.
      • et al.
      The Effect of an Electronic SBAR Communication Tool on Documentation of Acute Events in the Pediatric Intensive Care Unit.
      • Wayne J.D.
      • et al.
      Simple Standardized Patient Handoff System that Increases Accuracy and Completeness.
      The well-known I-PASS multicenter study assessed the effect of implementation of a handoff bundle on patient safety. The bundle included the use of the I-PASS mnemonic, simulation exercises, and team communication training. The first S in the I-PASS mnemonic stands for situational awareness and contingency planning and is a recommended step in the handoff process. Results from this study showed a reduction in preventable medical errors.
      • Starmer A.J.
      • et al.
      Changes in Medical Errors after Implementation of a Handoff Program.
      ,
      • Starmer A.J.
      • et al.
      Rates of Medical Errors and Preventable Adverse Events Among Hospitalized Children Following Implementation of a Resident Handoff Bundle.
      Most studies on how to improve handoffs have focused on which information should be transferred and how it should be communicated. They aim to standardize the handoff procedure using mnemonics, specific checklists, or electronic handoff systems, with positive effects on collaboration between clinicians, teamwork, and situational awareness. Some intend to reduce distractions or interruptions, with the added benefits of achieving undivided attention but also changing cultural beliefs about handoffs.
      • Abraham J.
      • et al.
      Comparative evaluation of the content and structure of communication using two handoff tools: Implications for patient safety.
      ,
      • Wayne J.D.
      • et al.
      Simple Standardized Patient Handoff System that Increases Accuracy and Completeness.
      Despite these successful interventions, the handoff process remains vulnerable and requires more attention.
      Studies that focus on health care providers on the receiving end of the handoff are currently lacking. A better understanding of how physicians correctly memorize the relevant information provided during a handoff could provide a possible route for improvement. Recall of patient information from memory is prone to errors, but it is commonly used in daily practice and would be difficult to completely avoid. Methods such as taking notes or using an electronic handoff tool can diminish recall errors, but their usability has been criticized. Handoff tools are often inadequately integrated with other electronic health care systems and clinical tasks, and concerns have been raised regarding their sustainability and consistency.
      • Abraham J.
      • Kannampallil T.
      • Patel V.L.
      A systematic review of the literature on the evaluation of handoff tools: implications for research and practice.
      Also, poor information infrastructures reduce utility when access to patient data is restricted to static workstations. Moreover, retrieving useful information from electronic handoff tools takes longer than recall from memory.
      • Yang X.J.
      • et al.
      Effects of Information Access Cost and Accountability on Medical Residents' Information Retrieval Strategy and Performance During Prehandover Preparation: Evidence From Interview and Simulation Study.
      Therefore, it is likely that recall of patient information from memory will continue to play a key role in continuity of care.
      The aim of this study was to investigate the effects of engaging in deliberate cognitive processing of information encountered during the handoff on recall of this information. Deliberate cognitive processing involves more analytical, conscious, and effortful thinking while addressing a problem. Psychological research has long shown that elaboration on to-be-learned material fosters integration of a new concept into existing knowledge networks in memory, favoring the combination of new and existing information into chunks of information. Retrieval of information from memory is thereby easier.
      • Ormrod J.E.
      Human Learning.
      Based on this well-established principle, it would be reasonable to expect that a procedure that fosters more deliberate, in-depth processing of information during the handoff will lead to improvement in recall of information and enable correct decision making, ultimately improving handoff performance. To test this idea, we conducted the present study exploring the effects of deliberate cognitive processing of handoff information on the content and quality of its subsequent recall.

      METHODS

      Participants and Setting

      Pediatric residents from three different Dutch pediatric academic hospitals (University Medical Center Utrecht, Leiden University Medical Center, Erasmus Medical Center) who attended an educational seminar participated in the study. The seminar consisted of educational sessions about pediatrics and teambuilding activities. Handoff was not a subject. The seminar lasted a full day and took place in a conference center. Residents were cleared from their regular tasks to attend the seminar. All residents that enrolled in the seminar were contacted by one of the investigators by email a week before the seminar and invited to participate in the study. Those who volunteered were recruited as participants. Participation was voluntary, without incentives. Written informed consent was obtained from all participants. Because disclosure of the study objective at the beginning of the study would influence how participants addressed the task, the participants were told that the aim was to generate input regarding the usefulness of developed handoffs for training purposes. Participants were debriefed about the actual objective after the study.

      Study Design

      The study was a two-phased experiment in which the first phase simulated a handoff reception and the second phase assessed recall of information about the handoffs received in the first phase. In phase 1 of the experiment, participants were randomly assigned to the intervention or control group. Randomization was performed in advance of the seminar, allocated according to sign-in date, and stratified by place of residency program. In both groups, participants were given the same eight written handoff reports and asked to read them thoroughly. Additionally, participants in the intervention group were given a specific task, by which we aimed to induce deliberate cognitive processing. Participants in the control group assessed the handoff reports as they would usually do, without additional instructions. In phase 2 of the experiment, all participants were asked to perform a recall task about four of the handoff reports seen in phase 1.

      Ethical Consideration

      The study was approved by the ethics review committee of the Erasmus University.

      Procedures and Intervention

      Preceding the experiment, eight written handoffs were developed by one of the investigators (G.H.). Another investigator (M.H.) checked the written handoffs for plausibility and relevance. Handoffs were based on real-life patients with different underlying illnesses (for example, diabetic ketoacidosis, bronchiolitis) and could hypothetically take place in a general academic pediatric department. See Appendix 1 for an example. Each handoff consisted of 300 to 400 words, with 40 to 62 idea units per handoff [mean 51.3, standard deviation (SD) 8.9] presenting the following information: patient's history, signs, symptoms, results of physical examination and tests, working diagnosis, and received treatment. The handoffs did not describe instructions for following management or a contingency plan. The handoffs were presented in a booklet, together with the instructions for the task, which differed for the intervention and control group. For both groups, four versions of the booklet with different orders of handoffs were prepared to control for order effects.

      Phase 1

      Participants were assigned to two separate rooms, based on prior randomization (see above). They were informed that this phase of the experiment would last 30 minutes. All participants were given the same written handoff reports and asked to read them thoroughly. They were asked to imagine that they were the responsible resident for all cases in a hypothetical next shift. As a cover story, participants from both groups were told that they would be asked to give input regarding the usefulness of the handoffs for a new curriculum at the end of the day. Both groups were informed that they would be asked questions about the content of one of the handoffs directly after phase 1, which was expected to stimulate the participants to actually engage in the task carefully while reading the handoffs. Participants from both the intervention and control groups were asked to directly report on this handoff. This handoff was randomly selected and was the same for all participants. It was not included in phase 2.
      In this phase, the intervention group was given a specific task, by which we aimed to induce deliberate cognitive processing. The group was instructed to read each handoff one by one and write down possible threats for the patient in the coming hours. Subsequently, they were requested to explain in writing why they thought these possible threats existed and to write out a contingency plan. This task was based on the I-PASS bundle, in which situational awareness and contingency planning is a step in the mnemonic. The control group was requested to read each handoff thoroughly, without any specific additional task, and were invited to make notes.
      After 30 minutes, all participants were requested to hand in the booklets and notes. They were not able to ask clarifying questions. They then immediately were asked to recall information from one of the handoffs.
      After phase 1, the participants returned to the educational seminar. Participants were instructed not to discuss the handoffs during the day.

      Phase 2

      After approximately five hours, phase 2 of the experiment started. Four of the eight original handoffs were randomly selected for recall in phase 2. For four of the eight handoffs provided in phase 1, participants from both groups were given 30 minutes to individually write down, on a blank page, as much information as they could recall from each handoff. As a cue for the participants, each handoff was titled the same as in phase 1.

      Outcome Measures

      The primary objective of the study was to assess the amount and accuracy of recalled information by evaluating the number of idea units and inferences reported by participants in phase 2.
      The amount of recall was measured by the number of correctly reported idea units and the number of correctly reported inferences. The accuracy of recall was measured by the number of incorrectly reported idea units and the number of incorrectly reported inferences.
      Other outcome measures, providing subsequent explanatory analysis, were critical idea units, confabulations, and captured idea units (see below).
      Outcome measures were all continuous variables. For each participant, we computed the means of the outcome measures in the four recall reports from phase 2. Means for the intervention and control groups were computed and compared.

      Idea Units

      An idea unit consists of the smallest meaningful, concrete information element (for example, the patient has fever) present in the original handoff. See Appendix 2 for examples. Each idea unit reported by participants was classified as correct or incorrect. A correctly recalled idea unit is an idea unit reported by the participant in phase 2 that corresponded with an idea unit in the original handoff. For example, the idea unit ‘fever’ from the original handoff is reported as ‘fever’ in phase 2. An incorrectly recalled idea unit is one reported in phase 2 that does not correspond to the original handoff. The idea unit ‘fever’ is, for example, now reported as ‘no fever.’
      Participants could also fail in reporting idea units that were present in the original handoffs. This was calculated as the remainder of the total idea units minus correct and incorrect idea units and was not analyzed further.
      Preceding the experiment, each handoff was split into idea units. This prepared a standard against which the recall reports were subsequently evaluated. One of the investigators (G.H.) scored all recall reports. This investigator was blinded to the experimental condition under which the report had been produced. Prior to the analysis of the data, two of the investigators (G.H. and M.H.) first independently evaluated 10% of the reports for the counts of correct and incorrect idea units to check the degree of interobserver agreement. The overall agreement was 87.9%; the Cohen's Kappa was 0.736. The idea units in the remaining reports were then counted by a single rater (G.H.).

      Inferences

      Another outcome measure is an inference, a conclusion made by the participant based on two or more idea units present in the original handoff. An inference can be reported without reporting the idea units that led to this inference. See Appendix 2 for examples.
      Two of the investigators (G.H. and M.H.), also blinded to the experimental condition under which the report had been produced, scored all reports for the presence of inferences and assessed the correctness of inferences.

      Critical Idea Units

      A critical idea unit is defined as information that was highly important to patient care. Each idea unit was valued with regard to a level of critical importance for patient care in a next shift by three experienced attending pediatricians (M.H., J.F., W.K.), each from a different academic hospital. Preceding the experiment, they independently valued the units by judging, for each single idea unit, whether it was a critical or a noncritical item. An idea unit was defined as a critical idea unit if all three pediatricians independently scored it as such.

      Confabulations

      We also measured the number of confabulations reported by participants. A confabulation is reported information that could not have been retrieved from the original handoff reports (in other words, information that was not present in the original handoff report).

      Captured Idea Units

      The final outcome measure was the number of idea units that could be captured in the inferences. A captured idea unit is an idea unit that is not specifically reported by a participant in phase 2, but captured in one or more reported inferences. For example, a participant reported that a patient's abdomen was suspicious for an ileus, without reporting the idea unit “abdominal distention.” The idea unit “abdominal distention” was scored as a captured idea unit. The analysis of captured idea units was carried out only with regard to critical idea units, rather than for all units.
      See Appendix 3 for a summary of outcome measures.
      All participants also received a questionnaire after phase 1, which provided the researchers with information about the participants’ background characteristics and beliefs regarding handoffs.

      Statistical Analysis

      Proportions of correct, incorrect, critical, and captured idea units were computed and compared between the two groups. Frequencies of correct and incorrect inferences and confabulations were also computed and compared. For parametric data (idea units, critical idea units, and captured idea units), the independent-sample students’ t-test was used; for nonparametric data (inferences and confabulations), the Mann-Whitney-U test was used. Two-tailed Pearson correlation coefficients and two-tailed Spearman's rho coefficients were calculated to evaluate correlation between participants’ characteristics and outcome measures. As far as we know, there are no previous studies describing recall of information in handoff, which makes a prior power analysis difficult. A convenience sample was therefore used. All statistical analyses were performed using SPSS 24.

      RESULTS

      Baseline Characteristics

      Of the 91 eligible participants, 13 did not show up at the seminar. (See Figure 1.) A total of 78 residents participated in the two phases of the study (intervention group, 37; control group, 41). No significant differences between the two groups were found with regard to gender, age, experience, and location of residency program (Table 1).
      Figure 1
      Figure 1Shown here is a consort flow diagram for participant inclusion.
      Table 1Baseline Characteristics
      Intervention (n = 37)Control (n = 41)All participants (N = 78)p value
      Female sex73.085.479.50.176
      Age in years (SD)31.0 (± 2.9)31.3 (± 2.6)31.1 (±2.7)0.658
      Years in residency (SD)2.7 (± 1.6)2.8 (± 1.4)2.7 (±1.5)0.820
      Experience in pediatrics (SD)44.3 (± 18.7)44.7 (± 20.5)44.5 (± 19.6)0.927
      University of residence (SD)
      Leiden18.922.020.50.910
      Rotterdam45.941.543.60.910
      Utrecht35.136.635.90.910
      SD, standard deviation.

      Outcome Measures

      All outcome measures were computed for the four handoffs assessed in phase 2.

      Idea Units

      Participants from the intervention group recalled significantly fewer correct idea units. There was no difference with regard to incorrectly recalled idea units (Table 2).
      Table 2Recall of Idea Units, Inferences, and Confabulations
      n = mean number of idea units, average number of inferences and confabulations; % = percentage of recalled idea units; p = level of significance according to independent-sample t-test for parametric data (idea units) and Mann-Whitney U test for nonparametric data (inferences and confabulations). Significant levels are in boldface type.
      Intervention n (%)Control n (%)p
      Handoff 1
       Idea units62 (100)62 (100)
       Correct idea units11.7 (18.9)15.7 (25.3)0.006
       Incorrect idea units0.5 (0.8)0.6 (1.0)0.509
       Correct inferences1.240.850.084
       Incorrect inferences0.080.020.260
       Confabulations0.490.510.810
      Handoff 2
       Idea units40 (100)40 (100)
       Correct idea units9.6 (24.1)11.3 (28.3)0.119
       Incorrect idea units0.8 (2.0)0.8 (2.0)0.832
       Correct inferences1.380.660.023
       Incorrect inferences0.050.020.499
       Confabulations0.651.050.072
      Handoff 3
       Idea units51 (100)51 (100)
       Correct idea units10.2 (20.0)12.3 (24.1)0.077
       Incorrect idea units0.7 (1.4)0.8 (1.6)0.530
       Correct inferences2.030.830.017
       Incorrect inferences0.110.100.832
       Confabulations0.620.930.247
      Handoff 4
       Idea units52 (100)52 (100)
       Correct idea units11.7 (22.4)12.6 (24.2)0.426
       Incorrect idea units0.9 (1.7)1.2 (2.3)0.257
       Correct inferences2.570.80< 0.001
       Incorrect inferences0.050.120.298
       Confabulations0.810.730.937
      Handoffs combined
       Idea units205 (100)205 (100)
       Correct idea units43.2 (21.1)51.8 (25.3)0.016
       Incorrect idea units2.8 (1.4)3.5 (1.7)0.225
       Correct inferences7.243.22< 0.001
       Incorrect inferences0.320.240.693
       Confabulations2.573.220.182
      low asterisk n = mean number of idea units, average number of inferences and confabulations; % = percentage of recalled idea units; p = level of significance according to independent-sample t-test for parametric data (idea units) and Mann-Whitney U test for nonparametric data (inferences and confabulations). Significant levels are in boldface type.

      Inferences

      Participants from the intervention group created more correct inferences than controls. There was no significant difference with regards to incorrect inferences (Table 2).

      Critical Idea Units

      A total of 23 (11.2%) idea units were defined as critical for the handoffs. There was no significant difference in recall of critical idea units between the two groups. The mean number of correctly recalled critical idea units was 8.3 (36.0%) in the intervention group and 8.8 (38.2%) in the control group (p = 0.438). The mean number of incorrectly recalled critical idea units also did not differ between groups: 0.7 (3%) in the intervention group and 0.7 (3%) in the control group (p = 0.438).

      Confabulations

      Participants from both groups reported confabulations (Table 2). No significant differences in reported confabulations were found between the participants from the intervention and control groups.

      Captured Idea Units

      Participants from the intervention group more often reported captured idea units than participants from the control group. The mean number of captured idea units was 1.14 in the intervention group and 0.37 in the control group (p < 0.05).

      Correlations

      There was a significant correlation between months of clinical experience in pediatrics and the number of correct idea units (r = 0.243, p = 0.032), correct critical idea units (r = 0.362, p = 0.001), and correct inferences (r = 0.340, p = 0.002) reported by participants in phase 2. No significant correlation was found between location of residency program and the number of correct idea units (r = -0.84, p = 0.464) or correct inferences (r = 0.017, p = 0.880) reported.

      DISCUSSION

      Our study showed that participants who engaged in deliberate cognitive processing while reading handoffs created more inferences in a free recall task regarding handoff than participants in a control condition who processed the reports as they would usually do. However, those who engaged in deliberate cognitive processing recalled fewer idea units.
      Creating more inferences shows that a participant is able to combine information regarding a patient into higher-level concepts. The participant makes connections between signs and symptoms, identifying them as a clinical manifestation of a higher concept that already exists in their memory. Thus, deliberate cognitive processing resulted in better understanding of patients’ problems, which could potentially contribute to improved patient safety.
      Participants in the control group created fewer inferences but recalled more idea units. A possible reason for this finding could be that the attention of participants who practiced deliberate cognitive processing (intervention group) shifted away from processing information to identifying potential threats for the patient.
      Furthermore, our study showed that participants from the intervention group more often reported idea units in inferences, the so-called captured idea units. This could also explain recall of fewer isolated idea units in the intervention group.
      Participants correctly recalled only a quarter of the total number of idea units present in the original handoff reports. In comparison with previous studies, this is less than expected.
      • Sutcliffe K.M.
      • Lewton E.
      • Rosenthal M.M.
      Communication failures: an insidious contributor to medical mishaps.
      The amount of correctly recalled information increased when considering only critical information, but still almost 63% of these critical idea units were not reported by the participants.
      Participants from both groups reported information that was not stated in the handoffs seen in phase 1 of the experiment, which we defined as confabulations.
      • Metcalf K.
      • Langdon R.
      • Coltheart M.
      Models of confabulation: a critical review and a new framework.
      In some cases, this information could be retrieved from other handoffs, with the participant blending information from one handoff in the recall assessment of another handoff. Sometimes information could not have been retrieved at all, because it was not stated in the original handoff, but it fit the clinical vignette of the patient. For example, a participant reported that the patient with a history of bronchopulmonary disease who was currently admitted for respiratory distress was fed through a nasogastric tube. This was not stated in the original handoff but is a common part of the treatment for such a patient. Confabulating information when recalling information is a common phenomenon in psychology. People make up likely sounding constructs and fill in the gaps in their memory with assumptions and speculations.
      • Gantman A.P.
      • et al.
      Why did I do that? Explaining actions activated outside of awareness.
      • Lutz S.
      • Huitt W.
      Information Processing and Memory: Theory and Applications.
      • Johnson M.K.
      • Raye C.L.
      False memories and confabulation.
      Confabulation without awareness is a potential hazard in patient safety. Confabulations could have even greater consequences, if for example a health care provider transfers confabulations in the next handoff. More research is needed to investigate causes and preventive factors of confabulating and to look for effects on patient safety in real settings.
      In our study, we observed a positive correlation between clinical experience and recall of idea units and creation of inferences. Although our sample is too small and the dispersion of the studied population not wide enough to draw a definite conclusion, this observation might point to absence of an expertise-reversal effect. An expertise-reversal effect occurs when an intervention is useful in young or less experienced doctors but could have a negative effect in expert clinicians.
      • van Merrienboer J.J.
      • Sweller J.
      Cognitive load theory in health professional education: design principles and strategies.
      An expertise-reversal effect has been observed, for example, when mnemonics, often a subject of research in handoff, are used.
      • Salzwedel C.
      • et al.
      The effect of a checklist on the quality of post-anaesthesia patient handover: a randomized controlled trial.
      ,
      • Breuer R.K.
      • et al.
      Standardizing postoperative PICU handovers improves handover metrics and patient outcomes.
      Some studies have focused on different cognitive aspects involved in handoff.
      • Young J.Q.
      • et al.
      Unpacking the Complexity of Patient Handoffs Through the Lens of Cognitive Load Theory.
      ,
      • Anderson-Montoya B.L.
      • et al.
      Running Memory for Clinical Handoffs: A Look at Active and Passive Processing.
      For example, the cognitive load theory highlights the complexity of the handoff process and elaborates on why the working memory, the bottleneck in human memory, can be simply overloaded in the handoff process. Although our study was not based on such a theory, we suggest further research should be done. The handoff can possibly be further improved by integrating our findings with other cognitive concepts and theories.

      Strengths and Limitations

      This study is particularly strong because a controlled experimental design was used. Another strength is that we investigated the effects of handoffs on the receiving physicians, which is an understudied area of research. Previous studies focused on the senders of information or both parties.
      • Starmer A.J.
      • et al.
      Changes in Medical Errors after Implementation of a Handoff Program.
      ,
      • Starmer A.J.
      • et al.
      Rates of Medical Errors and Preventable Adverse Events Among Hospitalized Children Following Implementation of a Resident Handoff Bundle.
      ,
      • Breuer R.K.
      • et al.
      Standardizing postoperative PICU handovers improves handover metrics and patient outcomes.
      ,
      • Gagnier J.J.
      • et al.
      Development, implementation and evaluation of a patient handoff tool to improve safety in orthopaedic surgery.
      We also studied a handoff procedure in which multiple patients were transferred, resembling real-life situations. Finally, the number of participants is relatively large and reflects a large proportion of the pediatric residents in the Netherlands.
      There are some limitations to this study. Participants were predominantly female, without a significant difference between the control and intervention group. This reflects the gender ratio in pediatrics in the Netherlands. Gender differences in cognitive performance, including memory, can exist.
      • Halpern D.F.
      • et al.
      The Science of Sex Differences in Science and Mathematics.
      This potentially hampers application of our results to residency programs with a different male-female ratio.
      The study setup was to investigate recall in a simulated setting. Therefore, certain aspects of clinical handoffs were not included, limiting generalizability to other types of handoffs, other levels of clinical expertise, and most importantly, real clinical settings. For example, the controlled design of our study prevented participants from discussing the cases or, for example, asking clarifying questions about the handoff, a strategy that is suggested to be important in handoff communication.
      • Patterson E.S.
      • et al.
      Handoff strategies in settings with high consequences for failure: lessons for health care operations.
      ,
      • Johnson J.K.
      • Arora V.M.
      Can we talk? The art (and science) of handoff conversation.
      During their daily clinical work, all participants carried out handoffs verbally accompanied with a written summary, instead of receiving only a written handoff as in the study.
      Despite the simulated setting, efforts to replicate a real-life clinical setting were made. The handoff reports in the first phase of the study were based on real-life patients, and the duration of the first phase (average time per patient: 3.7 minutes) also resembles a real-world setting.
      • Salzwedel C.
      • et al.
      The effect of a checklist on the quality of post-anaesthesia patient handover: a randomized controlled trial.
      ,
      • Caruso T.J.
      • et al.
      Implementation of a standardized postanesthesia care handoff increases information transfer without increasing handoff duration.
      • Joy B.F.
      • et al.
      Standardized multidisciplinary protocol improves handover of cardiac surgery patients to the intensive care unit.
      • Ilan R.
      • et al.
      Handover patterns: an observational study of critical care physicians.
      In our experimental study, participants were unable to hold onto notes, use a handoff document, check the electronic patient record, or ask other team members or even the patient for information. In everyday clinical practice, health care providers can rely on such memory aids, which may have some beneficial effects on handoffs in patient care.
      Participants also received the information from the handoffs only once, were not previously informed about the patients, and did not meet them in person. In our study, we did not examine the effect of repeated review of the information. Further research to test the added value of memory aids as well as the effect of deliberate cognitive processing in clinical settings and with physicians of different levels of expertise is needed.
      Last, while the development of a contingency plan is a recommended part of the well-known I-PASS mnemonic, the intervention and evaluation of the contingency plan alone has not been studied and is therefore not validated.
      • Starmer A.J.
      • et al.
      I-pass, a mnemonic to standardize verbal handoffs.
      ,
      • Cheung D.S.
      • et al.
      Improving handoffs in the emergency department.

      CONCLUSION

      This intervention study shows that residents who deliberately process received information during handoff are able to create more correct inferences about the transferred patients. This implies enhanced comprehension of the patients transferred in handoffs and improvement in quality of the handoff in general. Future studies are needed to investigate the effect of deliberate cognitive processing in real-life handoff procedures and for more experienced physicians.

      Declaration of Competing Interest

      All authors report no conflicts of interest.

      Appendix. SUPPLEMENTARY MATERIALS

      REFERENCES

        • Kluger M.T.
        • Bullock M.F.
        Recovery room incidents: a review of 419 reports from the Anaesthetic Incident Monitoring Study (AIMS).
        Anaesthesia. 2002; 57: 1060-1066
        • Leape L.L.
        • et al.
        The Nature of Adverse Events in Hospitalized-Patients - Results of the Harvard Medical-Practice Study-Ii.
        New England Journal of Medicine. 1991; 324: 377-384
        • Cook R.I.
        • Render M.
        • Woods D.D.
        Gaps in the continuity of care and progress on patient safety.
        British Medical Journal. 2000; 320: 791-794
        • Riesenberg L.A.
        • et al.
        Residents' and Attending Physicians' Handoffs: A Systematic Review of the Literature.
        Academic Medicine. 2009; 84: 1775-1787
        • Arora V.
        • et al.
        Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis.
        Quality & Safety in Health Care. 2005; 14: 401-407
        • Wohlauer M.V.
        • et al.
        The Patient Handoff: A Comprehensive Curricular Blueprint for Resident Education to Improve Continuity of Care.
        Academic Medicine. 2012; 87: 411-418
        • Abraham J.
        • et al.
        Comparative evaluation of the content and structure of communication using two handoff tools: Implications for patient safety.
        Journal of Critical Care. 2014; 29
        • Panesar R.S.
        • et al.
        The Effect of an Electronic SBAR Communication Tool on Documentation of Acute Events in the Pediatric Intensive Care Unit.
        American Journal of Medical Quality. 2016; 31: 64-68
        • Wayne J.D.
        • et al.
        Simple Standardized Patient Handoff System that Increases Accuracy and Completeness.
        Journal of Surgical Education. 2008; 65: 476-485
        • Starmer A.J.
        • et al.
        Changes in Medical Errors after Implementation of a Handoff Program.
        New England Journal of Medicine. 2014; 371: 1803-1812
        • Starmer A.J.
        • et al.
        Rates of Medical Errors and Preventable Adverse Events Among Hospitalized Children Following Implementation of a Resident Handoff Bundle.
        Jama-Journal of the American Medical Association. 2013; 310: 2262-2270
        • Abraham J.
        • Kannampallil T.
        • Patel V.L.
        A systematic review of the literature on the evaluation of handoff tools: implications for research and practice.
        J Am Med Inform Assoc. 2014; 21: 154-162
        • Yang X.J.
        • et al.
        Effects of Information Access Cost and Accountability on Medical Residents' Information Retrieval Strategy and Performance During Prehandover Preparation: Evidence From Interview and Simulation Study.
        Hum Factors. 2015; 57: 1459-1471
        • Ormrod J.E.
        Human Learning.
        Pearson Education International, New Jersey2009
        • Sutcliffe K.M.
        • Lewton E.
        • Rosenthal M.M.
        Communication failures: an insidious contributor to medical mishaps.
        Acad Med. 2004; 79: 186-194
        • Metcalf K.
        • Langdon R.
        • Coltheart M.
        Models of confabulation: a critical review and a new framework.
        Cogn Neuropsychol. 2007; 24: 23-47
        • Gantman A.P.
        • et al.
        Why did I do that? Explaining actions activated outside of awareness.
        Psychon Bull Rev. 2017; 24: 1563-1572
        • Lutz S.
        • Huitt W.
        Information Processing and Memory: Theory and Applications.
        2003
        • Johnson M.K.
        • Raye C.L.
        False memories and confabulation.
        Trends Cogn Sci. 1998; 2: 137-145
        • van Merrienboer J.J.
        • Sweller J.
        Cognitive load theory in health professional education: design principles and strategies.
        Med Educ. 2010; 44: 85-93
        • Salzwedel C.
        • et al.
        The effect of a checklist on the quality of post-anaesthesia patient handover: a randomized controlled trial.
        Int J Qual Health Care. 2013; 25: 176-181
        • Breuer R.K.
        • et al.
        Standardizing postoperative PICU handovers improves handover metrics and patient outcomes.
        Pediatr Crit Care Med. 2015; 16: 256-263
        • Young J.Q.
        • et al.
        Unpacking the Complexity of Patient Handoffs Through the Lens of Cognitive Load Theory.
        Teach Learn Med. 2016; 28: 88-96
        • Anderson-Montoya B.L.
        • et al.
        Running Memory for Clinical Handoffs: A Look at Active and Passive Processing.
        Hum Factors. 2017; 59: 393-406
        • Gagnier J.J.
        • et al.
        Development, implementation and evaluation of a patient handoff tool to improve safety in orthopaedic surgery.
        Int J Qual Health Care. 2016; 28: 363-370
        • Halpern D.F.
        • et al.
        The Science of Sex Differences in Science and Mathematics.
        Psychol Sci Public Interest,. 2007; 8: 1-51
        • Patterson E.S.
        • et al.
        Handoff strategies in settings with high consequences for failure: lessons for health care operations.
        Int J Qual Health Care. 2004; 16: 125-132
        • Johnson J.K.
        • Arora V.M.
        Can we talk? The art (and science) of handoff conversation.
        BMJ Qual Saf. 2016; 25: 63-65
        • Caruso T.J.
        • et al.
        Implementation of a standardized postanesthesia care handoff increases information transfer without increasing handoff duration.
        Jt Comm J Qual Patient Saf. 2015; 41: 35-42
        • Joy B.F.
        • et al.
        Standardized multidisciplinary protocol improves handover of cardiac surgery patients to the intensive care unit.
        Pediatr Crit Care Med. 2011; 12: 304-308
        • Ilan R.
        • et al.
        Handover patterns: an observational study of critical care physicians.
        BMC Health Serv Res. 2012; 12: 11
        • Starmer A.J.
        • et al.
        I-pass, a mnemonic to standardize verbal handoffs.
        Pediatrics. 2012; 129: 201-204
        • Cheung D.S.
        • et al.
        Improving handoffs in the emergency department.
        Ann Emerg Med. 2010; 55: 171-180