Abstract
Background: Driven by changes to improve quality in patient care and population health while reducing costs, evolvement of the health system calls for restructuring health professionals' education and aligning it with the healthcare delivery system. In response to these changes, the Accreditation Council for Graduate Medical Education's Clinical Learning Environment Review (CLER) encourages the integration of health system leadership, faculty, and residents in restructuring graduate medical education (GME). Innovative approaches to achieving this restructuring and the CLER objectives are essential.
Methods: The Alliance of Independent Academic Medical Centers National Initiative (NI) IV provided a multiinstitutional learning collaborative focused on supporting GME redesign. From October 2013 through March 2015, participants conducted relevant projects, attended onsite meetings, and participated in teleconferences and webinars addressing the CLER areas. Participants shared best practices, resources, and experiences. We designed a pre/post descriptive study to examine outcomes.
Results: Thirty-three institutions completed NI IV, and at its conclusion, the majority reported greater CLER readiness compared with baseline. Twenty-two (88.0%) institutions reported that NI IV had a great impact on advancing their efforts in the CLER area of their project focus, and 15 (62.5%) reported a great impact in other CLER focus areas. Opportunities to share progress with other teams and the national group meetings were reported to contribute to teams' success.
Conclusion: The NI IV learning collaborative prepared institutions for CLER, suggesting successful integration of the clinical and educational enterprises. We propose that national learning collaboratives of GME-sponsoring health systems enable advancement of their education mission, leading ultimately to better healthcare outcomes. This learning model may be generalizable to newfound programs for academic medical centers.
- Academic medical centers
- competency-based education
- cooperative behavior
- education–medical–graduate
- patient safety
- quality improvement
INTRODUCTION
As healthcare evolves to focus on population-based care and value- vs volume-based reimbursements, our healthcare systems are responding with transformations in the clinical, education, and research enterprises. Healthcare systems are active sponsors of graduate medical education (GME) programs, and restructuring the health professional's educational training is a priority. Aligning that training with the focus of the sponsoring institution promotes high-quality and safe care in the clinical workplace. Success within the healthcare system can be achieved once health professionals' education and healthcare delivery systems are closely aligned.1 To accomplish this goal, educational redesign, involvement of senior institutional leadership, and faculty education are paramount. Failure to develop competency training in systems-based practice and practice-based learning and improvement will ensure a skills gap in current medical teaching facilities that will be perpetuated in the physician workforce and subsequently impact quality population health.2,3
In an effort to promote a learning environment with a focus on quality improvement and patient safety, the Accreditation Council for Graduate Medical Education (ACGME) introduced the Clinical Learning Environment Review (CLER) in 2014.4 CLER requires that institutions expand their partnerships with all hospital areas to create an adequate and sustainable infrastructure for an optimal clinical learning environment. CLER focuses on 6 key areas: patient safety; transitions in care; supervision; duty hours oversight, fatigue management, and mitigation; professionalism; and quality improvement (including healthcare disparities).4,5 CLER also urges integration of GME leadership, faculty, and residents in hospital initiatives around CLER focus areas. CLER provides formative feedback on the effectiveness of an institution's engagement of residents and fellows in the 6 focus areas.5 The feedback includes insightful observations and opportunities for organizational improvement. However, as the program rolled out, understanding the initiative and educating all stakeholders about it proved to be a challenge for many institutions.
While academic centers share their experiences and best practices in many ways, the Alliance of Independent Academic Medical Centers (AIAMC) is an organization that brings together independent academic medical centers (institutions that operate independently of medical school ownership or governance while maintaining major medical school affiliations) from across the country (Figure 1) to improve educational, clinical, and research opportunities within their healthcare systems. The AIAMC's mission is to help members achieve the highest standards of patient care through the integration of medical education and research into their clinical missions. To fulfill this mission, the AIAMC organized a learning collaborative model that provides education, resources, and sharing of best practices. Collaborative learning, or situations in which forms of interaction generate learning,6 recognizes that education is an active process involving learning engagement.
The AIAMC has sponsored 4 learning collaboratives–National Initiatives (NIs)–since 2007. Fifty-five hospitals and health systems and more than 450 individuals have participated in the AIAMC NIs. This drive has produced learning and change that resulted in meaningful and sustainable outcomes in medical education and in the quality and safety of patient care.7-14 This study examines the outcomes of the AIAMC NI IV: Achieving Mastery of CLER that was created to optimize the principles in the 6 CLER focus areas. To our knowledge, ours is the first report evaluating a national learning collaborative seeking to support redesign of the GME system.
METHODS
Study Design
We used a pre/post descriptive study design to examine the outcomes of the AIAMC NI IV. This study was granted exempt status by the Scott & White Memorial Hospital Institutional Review Board based on CFR 46.101 (b)(2).
Program Structure and Participants
After a call for applications among the AIAMC member institutions in summer 2013, 34 teams were selected to participate in NI IV. The team members included designated institutional officials, GME educators, program directors, hospital C-suite members, faculty, residents, nurses, and other healthcare professionals. The NI model (the learning collaborative) provided teams the training and guidance necessary to (1) identify strengths and weaknesses across the 6 focus areas; (2) prioritize areas for improvement; (3) outline, streamline, and implement improvement strategies; (4) significantly and measurably advance the institutional level of preparedness; (5) engage the C-suite in the initiative; and (6) disseminate and share best practices. Participants attended 4 onsite learning sessions and participated in monthly networking teleconferences and educational webinars during the 18-month period of the NI from October 2013 through March 2015. Monthly team teleconference groups were structured by project area (patient safety, quality improvement, transitions of care, and professionalism) to share progress reports, network, and brainstorm about overcoming challenges in their home institutions. Educational presentations on the 6 focus areas, cross-pollination, and sharing of best practices occurred at the 4 onsite meetings (October 4-5, 2013; March 29-30, 2014; October 17-18, 2014; and March 28-29, 2015). The AIAMC staff developed a repository of critical information from institutions with CLER site visit experience (key learnings, surprises, and insider tips) that was shared with all participants. Onsite meeting sessions addressed, among other topics, how to prepare for a visit and how to better engage the C-suite. Teams were guided along the way with appropriate tools and resources, such as a project management plan and poster template.
Data Collection and Analysis
Data sources included pre/post participation surveys completed by each team. The preparticipation survey asked respondents to provide institutional data, identification of the team, and a CLER readiness assessment in the 6 focus areas via a 4-point Likert scale. Respondents described their institutional level of preparedness and activity as none, basic, intermediate, or advanced. In addition to the information requested on the preparticipation survey, the postparticipation survey included questions about scholarship outcomes and satisfaction with the NI.
We report categorical variables with counts and percentages. We used the marginal homogeneity test to compare overall pre/post intervention answers. A P-value ≤0.05 was considered significant. The statistical significance suggested changes in the distributions of the pre/post answers but was not an indicator that the change was attributable to the intervention because a control group was not included in the study design.
RESULTS
A total of 34 institutions were accepted into NI IV, but one institution dropped out during the initiative. Consequently, 33 institutions were invited to complete the postintervention assessment. Institutions participated in projects in the areas of patient safety (n=10, 30.3%), transitions in care (n=7, 21.2%), quality improvement (n=11, 33.3%), professionalism (n=2, 6.1%), and error reporting (n=3, 9.1%).
Thirty-three institutions responded to the 13 CLER focus area questions of the NI IV preintervention assessment, providing a proportion of 97.1% of all applicable questions answered in 34 questionnaires (429 of 442 total possible answers). For the postintervention assessment, 10 questions had 29 respondents, and 3 questions had 28 respondents, with a proportion of 87.2% of questions answered in the survey (374 of 429 total possible answers).
Overall, institutions participating in NI IV reported higher CLER focus area readiness in the postintervention self-assessment compared with the preintervention self-assessment (post vs pre: 28.6% vs 15.4% of total responses at the advanced level, and 24.1% vs 34.0% of total responses at the basic level) (Table 1). Marginal homogeneity tests comparing the distributions of the post/pre responses indicated differences for the following questions:
PS1. At what level do you feel your average resident is trained and given the opportunity to report errors, unsafe conditions, and near misses? (P=0.002)
QI1. How proficient is your institution at engaging residents in the use of data to improve systems of care and improve patient outcomes? (P=0.003)
QI3. How proficient is your institution at engaging residents in the overall identification and reduction of healthcare disparities? (P=0.028)
Super2. At what level does your institution maintain and oversee supervision policies concordant with ACGME requirements in the environment at the program level that assures the absence of retribution? (P=0.001)
DH1. How proficient is your institution at demonstrating effective and meaningful oversight of duty hours across all residency programs? (P=0.033)
Prof1. How well does your institution educate and monitor your residents on professionalism? (P=0.030)
The majority of participating institutions that had a CLER visit (n=18/21, 85.7%) reported a high rate of congruency between the NI IV self-assessment completed prior to enlisting in NI IV and the CLER site visit report.
The majority of participating institutions responded that the NI IV had a great impact on advancing their institutional effort in the CLER area of their project focus (n=22, 88.0%) or the other CLER areas (n=15, 62.5%) (Table 2). The NI IV program activities identified as having the most impact on the teams' progress in their CLER area were the opportunities to share progress with other teams (n=20, 83.3%) and the presentations on CLER topics (n=17, 73.9%) at the national group meetings. Additional NI IV activities, such as monthly team teleconference calls and informal collaborations with other institutions, were reported to have less impact.
A total of 59 institutional/local level presentations, 26 institutional/local level posters, 20 state/national presentations, and 39 state/national posters were the reported scholarly products of NI IV participants. In addition, participating institutions reported the generation of manuscripts as a result of NI IV, including 2 published, 4 in press, and 15 in preparation. A number of institutions reported residents as authors or coauthors on these presentations (n=17), posters (n=20), and publications (n=14). (Figures 2A through 2C).
DISCUSSION
As a national learning collaborative, AIAMC NI IV: Achieving Mastery of CLER focused on aligning medical education system redesign with the health system's focus on providing safe, effective, efficient, high-quality, and patient-centered care. The outcomes of the NI signify the role of learning collaboratives in providing leadership, support, and resources to propel movement in accomplishing objectives related to organizational needs.
With respect to preparedness outcome, we found that institutions were better prepared for CLER after participating in the learning collaborative. We offer 3 possible explanations about the structure of the collaborative for the preparedness success: (1) CLER areas were integrated throughout the planning of onsite learning sessions, teleconferences, and webinars; (2) opportunities were provided for sharing best practices and CLER site visit experiences; and (3) projects related to CLER focus areas were implemented. While we cannot isolate a single explanation, we suggest that collectively each of these NI IV components had an impact on the institutions' CLER preparedness. Further, the collaborative structure of the NI model supports the suggestion that collaboration serves as a process leading to the convergence of shared meaning among individuals.15
Further, we have shown several successes as a result of the NI IV. Engagement of GME with hospital strategic initiatives on improving patient care aligns the education and clinical enterprises. Involvement of institutional leadership officials, such as C-suite leaders, promotes change in management and reduces barriers for success. Incorporating resident participation provides unique training experiences—clinical and academic—as exemplified through the implementation of quality initiatives and scholarship opportunities. As a learning collaborative, the NI model offers a clear benefit by providing a platform for institutional sharing and learning at a national level and allows institutions to review, assess, and choose how different options can meet their local needs. By providing institutions the opportunity for knowledge sharing and best practices instead of one-size-fits-all solutions, we propose that institutions will benefit from higher satisfaction and improved implementation of programs that meet accreditation requirements.
As a consequence of the NI design, the evaluation of NI IV has limitations. Our study relies on the self-reported outcomes related to CLER areas. Further, institutions focused on different project areas and, thus, there may be variance in the specific institutional impact. The institutional teams that enrolled in the NI IV may have also had preselection bias because their leadership likely already had a deep appreciation for the importance of enhancing the clinical learning environment and was invested in a positive outcome. In addition, they may have known that their work would be enhanced by such learning collaboratives. Despite these limitations, our study retains value, justifying more in-depth studies on the impact of national learning collaboratives.
CONCLUSION
In the future, we hope to further explore the relationship between national learning collaboratives and learning efficiency and effectiveness by constructing evaluations that are specific in addressing clinical outcomes as a result of redesigning medical education curricula and programs. A longitudinal format would be valuable in ascertaining the transformational impact of aligning education and health system outcomes. Likewise, it would be of great interest to consider how specific and different changes implemented by institutions compare in their impact on said outcomes. Our opinion is that learning collaboratives such as the AIAMC NI will accelerate the process of cross-pollination among institutions and enable them to advance their mission in providing high-quality education, leading ultimately to better healthcare outcomes. The outcomes of this study suggest that organizations should consider structuring similar learning collaboratives when addressing priority issues, whether at the local, regional, or national levels, with the objectives of sharing of knowledge and best practices. In conclusion, the results from this study are important to share as academic medical centers continue to align medical education and health systems' objectives.
This article meets the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties Maintenance of Certification competencies for Systems-Based Practice and Practice-Based Learning and Improvement.
ACKNOWLEDGMENTS
The authors have no financial or proprietary interest in the subject matter of this article.
The authors are grateful to the Alliance of Independent Academic Medical Centers and its board of directors, its Committee on Integrating Academics and Quality, NI IV participating institutions, and Executive Director Kimberly Pierce-Boggs and Assistant Director Marcella de la Torre for their support of this body of work. The authors would also like to thank Yolanda Munoz Maldonado PhD for her statistical guidance and Chandler Carroll for her manuscript editorial assistance.
- © Academic Division of Ochsner Clinic Foundation