INTRODUCTION

In 2003 the Accreditation Council for Graduate Medical Education (ACGME) instituted common program requirements to restrict resident work-weeks to an average of 80 h with no more than 30 h worked consecutively. To comply with these requirements, most training programs adopted new resident schedules that resulted in an increased number of patient handoffs causing a decrease in continuity of care for patients admitted to teaching hospitals.1 Inadequate handoff of care, or sign-out, leads to interns and residents feeling unprepared for events that happen during cross cover periods and has been associated with adverse events.2,3 At the present time, both the organization and practice of sign-out are highly variable.1 Most residency programs do not have curriculum devoted to the sign-out process,1 and there is currently little prospective evidence supporting specific sign-out content or process. With further work-hour regulations adopted in the 2010 ACGE updated guidelines, safe and efficient sign-out will be a critical component of health care in the future.

The purpose of this study was twofold. First, we examined in detail the sign-out process at an institution in which cross-cover care constitutes up to 50% of an average patient’s hospital stay (see Fig. 1) and multiple sign-outs per day are common. This examination included a survey of attitudes toward and beliefs about the current sign-out process, as well as direct observation of sign-out by systems engineers to document both content and duration. Second, using an appreciative inquiry approach, we examined the sign-out process and content of five “exemplar” residents whose sign-out process was rated superior by their peers, and then used this analysis as a starting point for improving the sign-out quality at our institution. Appreciative inquiry is a highly effective change methodology that focuses on finding what is working in a system and using that as a basis for improvement.4–6 It is particularly useful in situations where best practice has not yet been established.

Figure 1
figure 1

Schematic of weekday shift coverage. Day teams that are not on call (teams 2–4) sign-out to a common day-call team (team 1) at 5 p.m. At 8 p.m., the day-call team signs out all of the teams’ patients to the night-float team, such that three of four teams’ patients are signed out sequentially after a 3-h period (team 5).

Full size image

METHODS

This study was conducted at the University of Virginia with approval of the institutional research review board. The study was conducted in three phases—first, an analysis of the existing attitudes toward and beliefs about the current and ideal sign-out process; second, analysis of the sign-out process by systems engineers; and third, an appreciative-inquiry approach to improve the sign-out methodology wherein exemplar residents were interviewed to determine best practices.

During the first phase, a voluntary, anonymous survey (using www.surveymonkey.comTM ) was conducted among all of the internal medicine residents at this institution (n = 89), as shown in Table 1. Nine questions pertained to general views toward sign-out, while five questions asked about the presence/accuracy of certain elements in the existing sign-out. These questions were directed at both the written and verbal sign-out, and were selected to understand residents’ views toward perceived inconsistencies in the sign-out process at our institution. Of note, the written sign-out was in the form of an electronic online database (mypatientyourpatient.com, Vienna, VA) that was printed prior to verbal sign-out and distributed; it was not directly linked to the electronic medical record. Responses were selected from a Likert scale (for the nine questions: agree completely, agree somewhat, neutral, disagree somewhat, disagree completely; for the five questions: always, most times, sometimes, rarely, never).

Table 1 Resident Response Rates for Pre-Existing Views Toward Sign-Out
Full size table

Phase two of the study consisted of direct observation of the verbal sign-out process by systems engineers (TP and EB) collaborating from our institution’s School of Engineering and Applied Science who had previous experience working with the sign-out practice within the Pediatrics Department.2 The systems engineers observed each stage of the sequential sign-outs during a 24-h cycle. In total, 128 consecutive, paired patient sign-outs of corresponding day (n = 64) and night (n = 64) general medicine inpatient ward sign-outs were observed to determine per-patient durations. These sign-outs were all instances of “double” sign-out, which occur when the patients of any resident team are signed out twice during a 24-h period. Double sign-outs occur when a primary team that is not on-call signs out to the day-call team (e.g. 6 p.m.), who then signs out again to the night float team (e.g., 8 p.m.), as shown in Figure 1. All sign-outs were observed and tape-recorded in the Medicine Resident Library, where sign-outs typically occur. The Remote Analysis of a Team Environment (RATE) tool was used to support analysis of sign-out recordings.7,8 The RATE tool allows the analyst to timestamp discussion of information elements, record the mode of communication, and record who is talking. Analysis included comparison of the time spent during the consecutive sign-out and characterization of the items discussed during both the day-time and night-time sign-outs (see Table 2 for specific elements recorded). To test the validity of the measurements made by the primary observer, two senior IM physicians analyzed 40 patient sign-outs to determine the level of agreement with the observer. Each rater was given a CD containing the 40 sign-outs and a checklist. The raters independently listened to each patient’s sign-out and recorded which information elements the sign-out participants discussed and which communication modes they used.

Table 2 Information Types Discussed in Consecutive Day vs. Night Sign-Out
Full size table

Phase three of the study consisted of the appreciative inquiry process to elucidate “best practice.” Residents were asked to nominate the top three “exemplars,” colleagues who in their opinion gave the best sign-out (this vote was included in the same anonymous, voluntary survey as the phase 1 survey described above). Nominations clustered on five top residents, and these residents were interviewed in depth by two interviewers. At the beginning of the interviews, each resident was asked open-ended, non-leading questions about their sign-out practice. The exemplar residents met as a group, and a list of practices for each of the residents was created and areas of overlap identified. Each of the residents was also asked specific questions and common practices were defined. The specific questions were developed by the authors based on results of the survey and perceived inconsistencies with prior sign-out practice. This group of residents then worked with a larger group of interested residents and faculty to devise a new structure and methodology for sign-out.

Statistical Analysis

Continuous data were expressed as the mean ± standard deviation. Nonparametric data were compared with the rank sum method. Categorical variables were compared by chi square. Significant results were reported at the α = 0.05 level. For analysis of the phase 2 observations, a kappa score was calculated using Fleiss’ method (1971) for measuring agreement among multiple raters.9 For each information element and the communication mode of each patient, the number of raters that marked the element as verbalized was summed. Then, for each information element, the proportion of raters that agreed was calculated. These proportions were averaged to get a raw agreement score. Fleiss’ method was used to normalize the raw agreement score into a kappa score.

RESULTS

Phase 1: Survey

The survey of pre-existing views toward sign-out was completed by 68/89 residents. Results of the survey are summarized in Table 1. Notably, residents agreed about the primary purpose of sign-out (to anticipate overnight events) but had less agreement about the organization of information (e.g., problem- vs systems-based) or extent of factual content required for written and verbal sign-out (e.g. inclusion of laboratory test values and medications).

Phase 2: Analysis of Existing Sign-Out

The time spent on consecutive day/night verbal sign-outs were 134 ± 73 s and 59 ± 41 s, respectively, amounting to a 56% reduction in time spent (p = 0.0002). The presence of data elements in verbal sign-out is categorized in Table 2. The overall kappa score for agreement among observers (internal medicine physicians versus systems engineer) was 0.664, consistent with substantial agreement.10

Phase 3: Appreciative Inquiry

The anonymous vote for the best residents at giving sign-out identified a top tier of five residents with a mean vote of 11 ± 1.6 for this group, compared with a mean vote of 1.7 ± 2.3 for the remainder. Two were PGY-2 residents and three were PGY-3 residents.

The appreciative inquiry interviews with these residents identified several areas of agreement. Each resident independently identified being concise as one of the most important aspects of their sign-out (both written and verbal). Concise sign-out was valued since it decreases inattentiveness that occurs during long, or unfocused, sign-outs. Four out of five residents reported achieving this goal by presenting the most acutely ill (or “sickest”) patient(s) first to allow adequate attention on the acute issues. Likewise, concise written sign-out is important for cross-covering residents to be able to quick identify key concepts. They also reported the importance of very brief sign-out for uncomplicated patients or those without acute issues, since this information can be easily obtained from the medical record.

Furthermore, 5/5 residents used a problem-based approach, since (1) it is more concise, (2) it is easier to organize the active issues, (3) it allows prioritization of problems, and (4) it is easier to limit to active problems. The residents all reported that they vigilantly updated the plan section on written sign-out on a daily basis to keep only active/relevant problems, which they felt was key to maintaining clarity. They also all reported that a detailed hospital chronology is not useful in this section since (1) it adds too much length and complexity to the sign-out, and (2) this information can be readily obtained from the medical record.

The group reported that the laboratory value errors on written sign-out were too common to be trustworthy, and all agreed that the laboratory test section should not be included on sign-out. They proposed that important and relevant laboratory tests should be kept only if pertinent in the active problems/plan section (e.g., the hematocrit could be listed under “acute anemia”). Likewise, they agreed that comprehensive medication lists also introduced too much potential for error (though high-risk medications, such as anticoagulants, continuous infusions, and chemotherapy could be included).

Finally, the residents also reported that “sign-in” does not often occur, or at best, occurs in a haphazard way. To address this issue, they suggested that an empty box for note-taking on overnight events be included in the written sign-out structure so that events can be organized for sign-in.

This group of residents then worked with a larger group of interested residents and faculty to devise a new template for sign-out with the following fields/attributes: (1) name and demographic information, (2) “star-rating” field, which marks the level of illness/acuity of the patient from 1–3 stars with patients sorted according to star-rating in the database, (3) chronic diagnoses, (4) important medications reported in different fields for therapeutic dose anticoagulation, chemotherapy, and continuous infusions, (5) inpatient procedures with dates performed, (6) active important problems with more critical/acute problems listed first, (7) anticipated events, (8) cross-cover tasks, and (9) empty box for notes on overnight events.

DISCUSSION

Our study attempted to characterize and improve the complex process of resident sign-out, which has become even more critical for ensuring patient safety since the ACGME work hour restrictions have resulted in more frequent shift changes. Typical of other residency programs since the ACGME standards were published, our residency program adopted a schedule that requires multiple sequential handoffs, but did not develop a curriculum for teaching patient care handoffs to improve safety. The primary findings from our study were that (1) sign-out organization and practice vary widely in the absence of an educational curriculum, (2) 40% of residents did not expect to make many decisions about patients during cross-cover periods, and (3) sign-out duration decreases by 56% for multiple sign-outs in a 24-h period, accompanied by a change in sign-out content. Furthermore, the appreciative inquiry approach was successful in identifying the common attributes of the methods of exemplar residents to guide the development a new sign-out methodology for both written and verbal sign-out.

The apparent degradation of information that occurs with multiple sequential sign-outs during a 24-h period is striking and has not been previously reported. Night-float coverage is now an essential system for meeting the ACGME 2010 updated recommendations. Duty hour restrictions have created scheduling concerns, and some teaching hospitals such as ours have experimented with interim cross-coverage in the early evening hours to allow non-call teams to go home at a reasonable time and avoid overworking the night float team. In our institution, this system resulted in an increase in the number of patient care handoffs by 70% and an increase in the average patient’s time under a cross-covering physician to 50%. Our findings of a significant reduction in time spent during sign-out from the day to night sign-out highlight that subsequent sign-outs are associated with a decrease in the amount of information conveyed. In addition, our finding that certain types of information are discussed less frequently in the night sign-out compared with the day sign-out supports that degradation of information occurs with successive transfers of information. The decrease in time spent on sign-out may not have been simply due to completion of tasks during this brief 2-h interim window, but rather a result of compressing multiple patient teams’ sign-out into one session when the night float team arrives. Resident teams may feel significant time pressure during this time of evening, while the week-day call team is also trying to complete all work for the new admissions.

An interesting finding of the survey was that many residents thought that they should not be expected to make many decisions regarding care of cross-cover patients. In addition to the risk of an increased number of patient care handoffs, the change in work structure toward shift-based care may also have the consequence of a cultural shift in views toward patient care. Our analysis did not investigate this point in detail, and this survey question may have some limitation due to potential for different interpretations. Yet, this trainees’ expectation for limited responsibility for “cross cover” patients is a finding that warrants further investigation.

The appreciative-inquiry approach was successful in identifying common attributes to improve the efficiency and organization of sign-out. A key finding of the appreciative inquiry group was that both written and verbal sign-out should be concise and organized in a systematic way by all residents. A major problem of the previous sign-out practice at our institution was that the written sign-out was often inaccurate, not updated, and not organized with a consistent methodology (see Table 1). Because multiple sequential sign-outs occur among providers who are not very familiar with the patients being covered, the written sign-out is critical to encapsulate the most important information and also to serve as a written prompt for the verbal sign-out. Clarity is critical when considering that the average time spent per patient on sign-out was only 134 s (dropping to only 59 s at night) in our study, and was even lower in another report (median of 39 s).11 To improve brevity and organization, the plan section should be limited to active and important problems, and eliminate long hospital course chronologies or the cumbersome systems-based approach. In addition, the most acute patients should be focused on preferentially by always presenting patients in order of most ill/acute to least. The group also agreed that laboratory tests and medications should not be included systematically on sign-out documents, since duplicating laboratory tests and medications is time-intensive and increases errors. Rather, important and relevant laboratory tests can be included in the written sign-out as part of the “plan” section, where it is more likely to be updated. Our group has previously shown that duplication errors were common for clinically important data, including differences in potassium, creatinine, and medications (e.g., a 28% error rate for antibiotics and a 60% error-rate for therapeutic-dose anticoagulation).12 Similar dramatic error rates have been shown at other teaching hospitals as well.13 Direct integration with the EMR significantly improves this type of error, which may allow more safe inclusion of laboratory test values and medications in the future.

Limitations

There were several limitations of our study. First, this was a single site analysis and may not be generalizable to other institutions. Second, only weekday sign-out sessions were analyzed, likely underestimating the information exchange problems that may occur during the more complex sequential sign-outs that occur over the weekend. In addition, we did not analyze the sign-in process. Third, observations of sign-out sessions were not blinded to the residents, which may have influenced behavior during sign-out. Fourth, this was an analysis of resident sign-out, which may not be generalizable to attending physician sign-out. Lastly, the exemplar physicians, while clearly distinguished by their peers as giving good sign-out, were not evaluated with patient level outcomes, and we therefore cannot link exemplary sign-out with better outcomes, as that would require further study.

Future Directions

Based on the results of the residency program’s appreciative inquiry and quality improvement initiative, we developed an educational curriculum for sign-out including didactic and interactive components, similar to programs developed at other institutions.14–16 An educational curriculum may also be critical in cultivating the notion of patient ownership among cross-covering physicians. In addition, we have developed a new software database for sign-out, in collaboration with systems engineers. The goal of the new database is not only to support the sign-out process, but also to reinforce the concepts discussed above through the structure and organization of the data entry fields.

CONCLUSION

In conclusion, we found that resident views toward the sign-out process are diverse in the absence of an educational curriculum directed toward handoff of care. Second, typical night float scheduling to accommodate resident work hour restrictions results in degradation of clinically important information following multiple sequential patient care handoffs. Finally, an appreciative inquiry approach was successful at our institution in identifying key changes to the sign-out process that may help improve the clarity and efficiency of information exchange at sign-out. Initiating an educational curriculum for sign-out at teaching hospitals is critical not only for establishing a standardized process for sign-out, but also for creating a culture of patient ownership among cross-covering physicians.