If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Many health care organizations want to improve their responses to patients who suffer medical injuries. Their ability to understand how well they meet patients’ needs is limited by the lack of suitable instruments for assessing injured patients’ experiences.
This study aimed to generate items for a patient experience questionnaire that medical facilities can use to assess how well resolution met patients’ needs. The Medical Injury Reconciliation Experiences Survey (MIRES) was based on findings from previous studies of New Zealand and American patients’ experiences of non-litigation resolution of medical injuries. The researchers performed a content analysis of 24 transcripts from a stratified random sample of 92 interviews from the prior studies. Themes were extracted to develop a draft questionnaire, which was revised following feedback from experts. Cognitive debriefing interviews were conducted with 24 New Zealand and American injured patients.
There were 40 items in the following domains: perceptions of communications with health care providers after the injury (15 items), perceptions of remedial gestures (11 items), indicia of the patient's overall satisfaction with the reconciliation process (4 items), the nature and impacts of the injury (5 items), and patients’ characteristics (5 items). Participants’ feedback about the questionnaire was predominantly positive. Their suggestions led to 37 revisions.
The MIRES was comprehensible and acceptable to this group of post-injury patients. While further testing is desirable, the MIRES offers promise as a practicable approach that health care organizations can use to assess how well their reconciliation processes met patients’ needs.
Medical injury can be a deeply traumatic experience for patients and family members, with serious physical, psychological, financial, and social impacts.
Traditional, adversarial processes of dispute resolution for medical injury involve considerable stress for families and care providers and are a poor mechanism for meeting injured patients’ needs for information, corrective justice, and reconciliation.
Such concerns, together with regulatory requirements for disclosure of adverse events, have led many health care organizations to develop alternative processes for reconciling with patients following medical injury. In the United States, for instance, a growing number of communication-and-resolution programs (CRPs) seek to ensure that injured patients receive a meaningful disclosure of, investigation into, and explanation of the cause of the harm, as well as an appropriate apology and other restitutive gestures.
Other countries, such as New Zealand and Sweden, operate administrative compensation schemes in which patients receive disclosures from hospitals and can obtain compensation from the government without having to show fault on the part of the provider.
Despite investment in these alternatives, patients’ experiences with medical injury reconciliation processes are not well understood. A limited literature has explored patients’ experiences with and preferences for disclosure,
but not other aspects of the reconciliation process. Many health care organizations would like to improve their institutional response to medical injury but currently lack the tools to evaluate how well they are meeting patients’ needs.
We developed and pilot tested a novel instrument for assessing patients’ and family members’ experiences with communication and reconciliation processes following medical injury, called the Medical Injury Reconciliation Experiences Survey (MIRES).* Structured questionnaires are the most common means of assessing patient experience
and have the advantages of being inexpensive to administer and amenable to comparison and benchmarking across organizations. Here, we report the development of the MIRES and provide recommendations for health care organizations that wish to use it.
We aimed to develop a questionnaire that (1) explores issues that prior research has identified as important to patients’ assessment of how they were treated by health care institutions following medical injury; (2) is practicable for health care institutions or liability insurers to administer and analyze without support from external researchers; (3) can be completed by patients in about 10 minutes; and (4) generates actionable insights for health care and insurer organizations seeking to improve how they deal with injured patients and their family members. The development of the MIRES proceeded through several steps.
Development of Initial Question Domains
To ensure content validity, the subject matter for the survey items was generated by interviewing injured patients. We conducted this work in two stages: (1) secondary analysis of interview transcripts from two prior studies
and (2) in 2018, a new round of cognitive debriefing interviewing with a subsample of patients who participated in the prior study. Prior to the interviews, we undertook a literature review. This review generated the initial domains for the interviews. The themes in the interviews reflected the themes in the literature about injury reconciliation.
Interviews with Injured Patients
To generate initial question domains and subtopics, we drew on interview transcripts from our previous studies of injured patients in New Zealand (n = 62) and the United States (n = 30). The methodology for those studies has previously been described in detail.
The studies were based in New Zealand and the United States because these jurisdictions have different regulatory frameworks for the resolution of medical injuries. In the United States, the dominant model is medical malpractice. In New Zealand, injured patients are unable to bring civil claims for personal injuries and, instead, apply for compensation and rehabilitation through a government-operated entity called the Accident Compensation Corporation (ACC).
Interviews followed a semistructured interview guide. The US version of the guide, which was adapted from the version fielded in New Zealand, is provided in Appendix 1 (available in online article). Interviews were conducted face-to-face in most cases, lasted 60 to 180 minutes and were transcribed and analyzed using standard methods of thematic content analysis. Response rates were 61% in the United States and 63% in New Zealand.
Results were quite similar for the United States and New Zealand. Interview participants identified several hospital practices that were perceived to promote reconciliation; these related to how to approach patients and family members to disclose adverse events, how to ensure meaningful communication during disclosure conversations, how to discuss patient safety improvement efforts, how compensation should be initiated, and the importance of seeking feedback on how reconciliation processes had gone.
The studies also identified specific needs that injured patients wanted hospitals to meet: being given time, space, and support to process information about the injury; being asked what their needs were; having the treating provider present during conversations about the injury; “being heard” during these conversations; having financial compensation offered proactively; receiving an authentic apology; and receiving timely information about steps the hospital would take to prevent recurrence of the event.
Generation of Questionnaire Items
To generate an initial set of question domains for the survey, we performed a secondary analysis of 24 randomly selected transcripts of patient interviews from the US and New Zealand studies that focused on aspects of the hospitals’ responses that were important to patients’ experiences. Our prior interview study suggested that 24 transcripts would be sufficient to achieve thematic saturation. Based on our review of these transcripts, we developed a set of four domains for the survey: perceptions of communication from health care providers after the injury, perceptions of remedial gestures (including compensation), overall satisfaction with the resolution process, and how the injury had affected the patient and family.
Within these domains, we then developed specific survey questions based on themes from the interview transcripts, incorporating team members’ expertise in injury resolution processes [J.S.S-M., M.B., C.J., M.M.] and patient experience questionnaire development. We added questions on patient and injury characteristics to provide background for interpreting patients’ survey responses. A cover letter explaining the purpose of the survey was also drafted.
After several rounds of revisions by the investigators to improve clarity and formatting, the draft questionnaire and accompanying letter to patients were sent to a diverse group of 15 expert reviewers. Experts were selected based on personal contacts and knowledge of relevant literature to represent several domains of expertise: patient advocacy, medical injury resolution processes, patient safety, and survey design. They were asked to read the materials and suggest any improvements they felt were advisable. Reviewers’ written feedback was compiled, discussed by two team members [J.S.S-M., M.M.] and implemented to the extent feasible. Suggestions to revise question wording were feasible to implement, whereas suggestions for additional questions that lengthened the survey were not. For example, three reviewers suggested that we should add a question about whether health care providers gave patients information about how to complain to external health complaints entities. Given that this question is not directly related to patients’ experiences of the injury and whether their providers met their needs, we decided not to add extra length to the survey by including this question.
Cognitive Debriefing Interviews
Next, the survey questionnaire was pilot tested on a subsample of patients who participated in the original interview studies. In 2018, participants took the survey with pen and paper while the interviewer timed them, and then underwent a cognitive debriefing interview. A sample size of 24 participants was selected based on experience in our prior interview study suggesting that this number was sufficient to achieve thematic saturation.
Stratified random sampling was employed using country and ethnicity as strata. We drew approximately two thirds of the pilot sample (15/24) from the United States because of the likelihood that the survey would be used widely there. To ensure representation of racial and ethnic minority perspectives, we included a one-third sample of participants who were not of European descent. To restrict the sample to individuals closest to the actual population to be surveyed (that is, people with recent injuries), we excluded persons for whom more than three years had elapsed from the date of injury, unless they were needed to ensure the one-third sample of non-Europeans.
The interview guide used for cognitive debriefing is provided in Appendix 2. An initial set of debriefing questions focused on the recruitment letter and Information for Participants sheet. Participants were asked to summarize what these materials conveyed and suggest improvements. The next set of questions asked respondents to explain the terms facility, provider, and facility representative, which are used repeatedly in the survey questions, in their own words, to ensure that they distinguished between clinical care providers and hospital administrators. Next, participants were asked to read each questionnaire item aloud and “think aloud” about how they interpreted the question and came up with their answer. For particular questions, the interviewer followed up with prespecified probes as needed to ensure that key information was elicited. For example, for questions regarding specific conversations that the patient had with hospital staff, the interviewer asked whether the participant had difficulty remembering the conversations; for questions that mentioned “medical error,” participants were asked what they thought that term meant. The interview concluded with a series of questions asking for participants’ overall impressions of the questionnaire and an invitation to contribute any other thoughts, concerns, or comments.
Cognitive debriefing interviews were undertaken face-to-face, lasted 30 to 45 minutes, and were audio-recorded and transcribed. Interviews were conducted by one female investigator [J.S.S-M.] who had also interviewed the participants in the earlier study. The Stanford University Institutional Review Board deemed this research exempt from review. All participants provided written informed consent.
Transcripts were reviewed by one investigator [J.S.S-M.]. We did not rely solely on the coding schema from our prior studies. We also undertook a new thematic analysis of the cognitive debriefing interview transcripts, using a progressive process of classifying, comparing, and refining categories. This approach was appropriate because, unlike our prior studies, the current project focused on questions about the design of the survey questionnaire. The reviewer tabulated counts of different themes and noted consistency (or otherwise) across participants’ responses to particular questions. For example, we identified responses where a significant number of participants gave similar responses to questions that asked the participants whether they had “any difficulty answering these questions.” The purpose was to consider revising the questionnaire based on these responses. One investigator [J.S.S-M.] compiled participants’ responses to each debriefing question and generated a list of action items suggested by these responses. The action items were discussed as a team, and questionnaire revisions were then implemented.
We present the final set of items that will be tested and refined in future surveys and, consequently, may be reduced in length as a result of psychometric testing. Figure 1 depicts the survey instrument development process.
Proposed Survey Domains and Questions
The initial coding for the previous studies was undertaken with a broader purpose; we assessed participants’ post-injury experiences generally. We undertook the secondary analysis with a different focus, concerned particularly with specific aspects of their experience that could be actionable by hospitals. We also undertook the secondary analysis to minimize the likelihood that something was missed in initial coding rounds. The secondary analysis of the transcripts identified four general domains of patients’ experience: communications about the injury, remedial actions they desired from the hospital, overall satisfaction with the resolution process, and how the injury had affected the patient and family.
Within the first domain, patients’ and families’ accounts suggested the need to explore both the initial disclosure conversations and subsequent resolution discussions (centered on compensation and other remedial gestures), because all were important for successful reconciliation. Patients’ accounts highlighted the importance of having support persons present during these conversations, prompting us to include a survey question about whether legal counsel or other support persons were included in the conversations. The accounts also indicated a strong need to be heard in these conversations; therefore, we drafted questions focusing on how well hospital representatives listened to and elicited patients’ concerns, as well as how effectively they conveyed information about the harm event. Finally, a number of interviewees highlighted that there should be questions relating to the appropriateness and sensitivity of language used in these conversations.
Within the second domain, patients’ and families’ accounts identified several aspects of resolution or remedial gestures that promoted reconciliation, including compensation and efforts to improve patient safety. They spoke not only to the appropriateness of being offered (or not offered) compensation, but also to the timing, generosity, and tone of the offers. These issues were also incorporated into survey questions.
Within the third domain, we included questions about the patient's overall satisfaction with the reconciliation process. Whereas other individual questions were drafted to garner detail about specific aspects of “resolution” such as compensation or apologies, the questions about satisfaction were included to reveal participants’ overall experiences. This domain also included a question about whether the injured patient would recommend that a friend or family member should seek care from the facility involved in the harm event. We included this question because overall patient satisfaction may be associated with their recommendations for care providers.
Within the fourth domain, we included questions about the effect of the medical injury on patients and families—not only because of the high frequency of such discussion in the in-depth interviews, but also because of the participants’ detailed narratives describing how the injury had affected them and the close connection to the important need to be heard. These narratives spanned financial, social, familial, and psychological effects. In addition to questions exploring those categories of effects, we drafted questions about the likelihood that patients would continue to seek care from the involved hospital and providers. Prior research found that some injured patients reported avoiding the physician/facilities involved in the error.
Whether injured patients continue to seek care from the involved providers may be an important metric for assessing how well providers have met injured patients’ needs. Therefore, we included items in our survey related to the likelihood that injured patients would continue to seek care from the providers involved in the harm events.
Patient demographic information was added to the survey questionnaire to facilitate hospitals’ analysis of the experiences of different groups of patients. Specifically, we included questions on the respondent's gender, race/ethnicity, age, and education; the clinical area to which the injury related; and the respondent's belief about the likelihood that a medical error had caused the injury.
We invited 15 experts to comment on the draft questionnaire; 14 returned feedback. Reviewers included four test site hospital representatives, four patient advocates, two patient safety experts, two communication-and-resolution program (CRP) experts, one expert in the design of questionnaires, and one safety culture expert. Two of the experts also had expertise in designing surveys relating to patient safety. The four test site hospital representatives comprised a patient liaison and other senior members of clinical safety, health care quality, and risk management teams.
The experts’ feedback was generally positive. They made a number of suggestions for improving wording and adding questions. A common suggestion across the groups of experts was to add opportunities for free-text responses. This suggestion accords with injured patients’ need to tell their stories and be heard.
Feedback from the test site hospital representatives and CRP experts focused on edits for questions concerning terminology and processes. For example, two test site hospital representatives and two CRP experts recommended avoiding the term treating doctor because patients may not know who their treating doctors are due to frequent turnover in clinicians’ shifts. In addition, given that reconciliation after medical injury requires multiple conversations, experts suggested edits to the number and content of our draft questions about patients’ discussions with health care providers. For example, they suggested referring to the disclosure conversations in the plural because several discussions often take place.
The patient advocates’ feedback primarily centered on eliciting further information about patients’ experiences and perceptions. For example, these experts suggested edits to the survey questions about patients’ needs and concerns—needs and concerns were important to differentiate because they may elicit different answers from participants. The patient advocates also encouraged revisions to better capture whether providers engaged respectfully with injured patients, as opposed to simply whether they were present during disclosure and resolution discussions.
Pilot Testing and Questionnaire Revision
Twenty-four of the patients and family members in our earlier US and New Zealand interview studies participated in cognitive debriefing interviews to pilot test the revised questionnaire. Sixteen participants were from the United States, and 8 were from New Zealand. All 24 patients who were approached agreed to participate. The participants included 22 patients and 2 family members; 13 were female and 11 were male; and the mean age was 44 years (range: 21–67 years) (Table 1). The majority (16/24) were of European descent, but others identified as Maori (2), Pacific Islander (2), Hispanic (2), and Asian (2). The median time to complete the questionnaire was 10.5 minutes (range: 7–14 minutes). None of the participants left any items in the questionnaire blank.
Table 1Patient and Family Member Participants’ Demographics (N = 24)
Our overall approach to the participants’ suggested revisions was to (1) include minor suggested revisions that would not increase the length of the questionnaire and (2) make revisions that were suggested by a substantial number of participants (six or more). The participants suggested 37 revisions; we were able to implement all of them. The revisions can be categorized as follows:
Expanding options for free-text responses to accommodate patients’ need to be heard. Our prior research highlighted the vital importance of providing injured patients with opportunities to be heard. Therefore, we undertook all suggested revisions that were consistent with this need. For example, many participants suggested that we add an option for participants to follow up with the health care institution that would administer the questionnaire.
Changing terminology and question ordering to make the questions easier for respondents to understand. For example, we reordered several questions relating to disclosure conversations and subsequent conversations to help participants differentiate between communications focused on what happened and those focused on what the hospital would do about it, and to differentiate between conversations with care providers and conversations with representatives from the insurance company or risk management.
Adding response options to particular questions. Participants suggested that we add options to questions to capture a broader range of views and experiences. For example, we edited a question to ensure that it adequately captured a range of adverse impacts experienced by patients and families after the event. Specifically, we added “. . . ability to do recreational activities.”
Editing questions to differentiate between health practitioners and organizations. These edits were suggested because participants often had different experiences with their care providers and with the hospital or insurer. For example, they may have received an apology from a physician but not an official apology from the facility.
Notwithstanding these suggested improvements, participants’ feedback about the questionnaire was predominantly positive. They appreciated the opportunity to assist with the survey design process. Several participants lamented the fact that there “was no similar survey to complete when [they were] injured” and wished they were “asked to do this then!”
Final Item Pool and Proposed Domains
The final item pool and proposed domains included 40 items in the following domains:
Perceptions of communications with health care providers after the injury (15 items)
Perceptions of remedial gestures, such as apology (11 items)
Indicia of the patient's overall satisfaction with the reconciliation process (4 items)
The nature and impacts of the injury (5 items)
Characteristics of the patient (5 items)
Figure 2 depicts examples of the survey questions about the impact of the medical injury on patients and family members. Figure 3 provides examples of the survey questions about compensation.
The MIRES is available in Appendix 3 and from the authors.
Seeking feedback from patients about their experiences in health care is fundamental to quality improvement and ensuring that care is patient centered.
This need is particularly salient for quality improvement in the domain of medical injury resolution because research and development efforts to date have focused almost exclusively on the needs and perspectives of clinicians, hospitals, and their liability insurers. However, most studies about medical injury resolution have not focused on patients’ perspectives.
To persuade health care organizations and insurers to implement CRPs, advocates of the approach have made the case that it is salutary for organizations. Research has supported their arguments that CRPs can reduce liability costs and ease moral distress among physicians who want to be forthright and caring when their patients suffer an unexpected care outcome.
Research on the extent to which these programs also serve patients’ needs has been comparatively slow to emerge. This has left current and would-be CRP adopters with little guidance for making their reconciliation processes more patient centered.
Although further research is needed—for example, to better understand differences in needs and experiences across subgroups of patients—health care organizations can also learn by assessing their own performance. To that end, we demonstrated that it is feasible to develop and administer a patient experience questionnaire that captures key lessons from prior research about what is important to patients following medical injury, including being heard and supported during disclosure conversations, being asked what their needs and desires are concerning restitutive gestures, and receiving timely compensation offers through a non-adversarial process. Such assessment can help avoid suppositions about patients’ experiences that consciously or unconsciously reflect the perspectives of providers and hospital administrators rather than patients themselves. Assessing patient experience also helps fulfill health care providers’ ethical obligation “to learn from instances when care falls short of the ideal, and to seek opportunities to improve care.”
The MIRES has potential utility in organizations with CRPs as well as other varieties of injury reconciliation processes (for example, administrative compensation schemes like New Zealand's, or mediation programs). For organizations interested in using the MIRES, we offer suggestions based on our prior research with injured patients and CRP program administrators.
First, individualize the time line for sending out the survey. Some hospital risk managers suggest a waiting period of two to three months before involving injured patients or their family members in research or feedback processes about their experience. Although many patients and family members will be emotionally and medically ready to consider these issues within two to three months of the injury, some will not. Patients and family members should have time to process the event and the subsequent communications with the health care organization before being asked to give feedback. However, waiting until all pending or possible malpractice claims have been resolved is not necessary or desirable. Some institutions may worry that eliciting feedback from patients could provoke distress, anger, or litigation, but our prior interviews with several dozen patients in two countries suggests no basis for that concern.
Second, to convey the sentiment that patients’ feedback is being sought for purposes of quality improvement, the questionnaire should be sent by the hospital's quality or patient relations office rather than risk management or the liability insurer. However, results should be shared with risk management (and the liability insurer, if the case involved compensation discussions), because patients in our research were eager to give feedback to those offices.
Third, the survey should be administered by mail or online rather than over the phone. This is consistent with how other patient experience surveys are conducted. In addition, patients in our previous studies indicated that they preferred to be contacted by letter rather than phone or in person after the initial disclosure communication had taken place, because it allowed them time to absorb and respond to the information received. In addition to completing the survey, injured patients participating in CRPs should have opportunities to discuss their experiences in person with their health care providers. This combination of written and face-to-face opportunities to process the adverse events reduces the likelihood that participants will feel that it is inappropriate to receive a survey by mail.
Fourth, organizations should have a response plan in place that can be activated if a survey response indicates a patient is experiencing ongoing anger or distress. Although our prior research suggests that asking injured patients about their experience does not cause secondary trauma,
it is ethically desirable to offer assistance if patients are still suffering from the original injury. Hospitals may wish to include a survey question asking patients if it is acceptable to contact them about their response. In our previous interview study of US patients, several patients exercised the option to have their interview transcripts shared with the hospital, and one hospital reached out in a few cases to express ongoing concern and care for the patient, reportedly with positive results. Organizations that opt to make the survey anonymous could include resource information for the patient, such as counselors available in the patient relations department.
Finally, we included several free-text response fields based on expert input, some of which invited respondents to explain foregoing responses. Such fields offer respondents a chance to narrate their experiences—a strongly felt need of injured patients—but they can complicate survey data analysis. For example, an explanation provided after a structured response may appear to conflict with the structured response, raising the question of whether the structured response should be recoded. For this reason, some institutions may prefer to omit free-text fields.
This study has several limitations. A fully validated patient experience instrument should demonstrate reliability and transferability as well as internal validity.
The MIRES has not yet reached that stage. Our pilot sample was small, and we did not conduct a larger pretest in the field following the qualitative pilot work. Unlike general patient satisfaction surveys, surveys of injured patients are difficult to field in large-scale pretesting because most patients do not experience medical injuries, and it is difficult to identify those who do. Identifying potential pretest participants requires searching risk management databases for patients whose injuries occurred during particular date ranges and who are not involved in ongoing litigation. An additional challenge to measuring validity is that, in contrast to patient experience surveys focused on clinical care, there are no other established tools for measuring patients’ experiences with reconciliation after medical injury. Health facilities that use the measure may be unable to score it, but the measure in its present form can provide a descriptive picture of patient experiences across a number of areas.
Another limitation of our study is the lack of diversity in the patient sample. This study drew from the sample in our earlier work, which unfortunately included few people of color in the United States. This limitation will be important to address in future studies because, as we have documented, reconciliation processes should be tailored to individual patients’ needs, including their cultural communication expectations.
Due to medicolegal concerns, hospitals often limit researchers to recruiting patients who can no longer bring a malpractice claim, which results in a much smaller pool of potential participants with a recent injury.
compounds the difficulty of achieving sufficient diversity in the patient sample.
An additional limitation is that patients whose adverse events occurred more recently may respond differently from the participants in our sample. The patients in our study had more time to process their injuries, which means that their experiences—and survey responses—may be different from patients who have been acutely harmed. Therefore, further research is needed to increase the likelihood that the survey will be appropriate in acute harm settings. Another limitation is that the survey was not formally reviewed for literacy level. Finally, the patients who had already participated in our initial studies may have a faster response time to survey questions.
Notwithstanding these limitations, the MIRES is based on in-depth qualitative research focusing on aspects of medical injury response that are important to patients.
—in this case, disclosure conversations, follow-up communications with facility representatives, and communications about compensation. Following best practice in the patient experience survey design field,
patients’ input was taken into account in generating the content of the question items, not only in pilot testing them. Finally, the MIRES also reflects multistakeholder, multidisciplinary expert input.
Although not validated, the MIRES survey instrument shows promise as a tool for assessing patients’ experiences. The MIRES effectively captures the issues of greatest importance to injured patients, was well accepted by patients who piloted it, and was feasible to administer quickly with pencil and paper. Incorporating the MIRES into the workflow of CRPs can help health care organizations better understand how well they are achieving their goal of meeting patients’ needs. The MIRES may be another way for health care institutions to demonstrate care and respect for patients even when things go wrong.
Data Availability Statement
Although the MIRES questionnaire is freely available for public use, no interview data used in its development are available. Interview transcripts and pilot surveys are not available for sharing because participants were promised that their responses would be kept confidential.
This research was funded by a Sylff Leadership Initiative award, a University of New South Wales Research Support Program award, and a National Health and Medical Research Council Emerging Leadership Investigator Grant.
The authors thank the patients and other interview participants, as well as the expert reviewers of the questionnaire, for their contributions to this work.