Abstract
Background: The Wayne State University Graduate Medical Education (GME) Office and Ascension Crittenton Hospital developed an educational initiative to increase resident awareness of health disparities and local community health priorities. The Plan-Do-Study-Act (PDSA) rapid-cycle performance improvement framework was used for implementation and evaluation.
Methods: During the first PDSA cycle, residents attended 5 didactic sessions. During the second PDSA cycle, residents participated in a problem-based learning (PBL) case. The following data were collected: evaluations of the didactic sessions and case, the number of appointments for diabetes self-management and education (DSME) referred by faculty and residents, and responses to questions on the annual GME surveys related to resident understanding of health disparities and the hospital’s community health needs assessment (CHNA).
Results: Eighty-eight percent of residents defined health disparities at least partially correctly in both project years. The percentage of residents who knew how to access their hospital’s CHNA increased from 25% to 29% year over year. Residents rated PBL more effective in achieving learning objectives than didactics, but the difference was not statistically significant. Six appointments for DSME were referred by program faculty and residents in the 2-month period immediately before the didactic sessions, and 6 referrals were made in a 2-month period between the didactic sessions and the PBL case. In the 2-month period immediately following the PBL case, 9 appointments for DSME were referred by residents and program faculty.
Conclusion: Residents have a good understanding of health disparities, although many may not recognize disparities that exist in their local community. PBL was more effective than didactics for resident education about local health disparities, CHNA, and DSME. Aligning GME and hospital leadership in a common vision for disparities education, as well as community engagement, is critical to successful outcomes.
INTRODUCTION
The term health and healthcare disparities refers to differences in health and healthcare between population groups.1 Despite widespread acknowledgment of their existence and health effects, health and healthcare disparities persist throughout the United States.2 The accrediting organization of graduate medical education (GME) in the United States, the Accreditation Council for Graduate Medical Education (ACGME), has prioritized healthcare disparities as an important component of healthcare quality through its Clinical Learning Environment Review (CLER) program.3
The Wayne State University (WSU) GME CLER Council is a subgroup of the WSU School of Medicine GME Committee and meets monthly, chaired by the designated institutional official. WSU family medicine, internal medicine, and transitional year residency programs are hosted by Ascension Crittenton Hospital in Rochester, MI and participate in the GME CLER Council. The GME CLER Council provides leadership and organizational structure to the residency programs for all ACGME CLER focus areas, including healthcare quality and health disparities. The GME CLER Council is also an effective venue for residency faculty, staff, residents, and hospital quality leadership to discuss and plan quality improvement (QI) projects and develop processes and procedures for effective implementation of QI projects.
Consistent with the ACGME focus on disparities, all incoming WSU residents are required to complete one online learning module related to health disparities prior to beginning their first year of residency. The module (Cultural Competency in Healthcare) is part of the American Medical Association GME Competency Education Program.4 Additional health disparities information is provided as part of new resident orientation. Individual residency programs may incorporate elements of healthcare disparities, population health, and chronic disease management concepts during regular didactic sessions. Discussion among the WSU GME Office and Crittenton-based residency programs personnel revealed that education about and awareness of healthcare disparities in the learning environment could be strengthened.
In 2015, the Alliance of Independent Academic Medical Centers (AIAMC), of which Ascension Crittenton Hospital is a member, launched a National Initiative among its membership to reduce healthcare disparities.5 As part of this initiative and in light of the identified gaps in resident education, Crittenton and the WSU GME Office designed a health disparities educational curriculum that was closely aligned with 2 of the priority areas of the hospital’s community health needs assessment (CHNA): diabetes and access to care. Crittenton conducts a CHNA every 3 years as required by federal law,6 and Crittenton’s 2016 CHNA update identified the following 3 main priorities: obesity/overweight/nutrition/diabetes, mental health, and access to care.7
The focus on health disparities and the CHNA in this resident educational initiative was expected to raise awareness of health disparities and of the hospital’s priorities identified in the CHNA. The ultimate goal of the initiative was to improve population health in the communities served by the hospital.
METHODS
The GME CLER Council, with representation from residency program leadership and faculty, residents, hospital leadership and quality personnel, and the GME Office planned and provided oversight of the educational interventions, analyzed the evaluation results, and organized efforts for sustainment of the health disparities curriculum. The curriculum was designed to increase resident awareness of heath disparities that are prevalent in the hospital community, increase resident knowledge about the hospital’s CHNA priority areas, address disparities in diabetes care, and improve population health in the community through increased utilization of diabetes self-management and education (DSME). DSME is a critical element of care for people with diabetes and improves patient outcomes.8
Engagement of residents was assumed to be an extremely important factor in the success of the project, and resident input was sought throughout planning and all other phases of the initiative.
The AIAMC fostered development and evaluation of the project components through sponsored communication with other AIAMC-member hospital residency programs from across the nation and AIAMC staff through conference calls, webinars, and conferences.
Curricula that increase resident knowledge about health disparities are an effective strategy for improving understanding about health disparities.9,10 While many curricular approaches to reducing healthcare disparities have been developed, the effectiveness of much of the training curriculum in cultural competency and healthcare disparities is unknown.11,12 In the WSU initiative, 2 different approaches were used: didactics and problem-based learning (PBL). The Plan-Do-Study-Act (PDSA) rapid-cycle performance improvement framework was used to implement the curriculum.13
Plan-Do-Study-Act Cycle I—Didactics
During the first PDSA cycle, residents attended 5 didactic sessions on health disparities, DSME, the CHNA, health disparities specifically related to diabetes, and diabetes resources available in the community. Table 1 shows the goals and expected outcomes of the sessions. Collaborative partnerships are effective in achieving communitywide behavior change and improving population-level outcomes.14 Consequently, the initiative relied on the development of community partnerships, particularly for the didactic curriculum. A partnership developed among residency program directors, hospital personnel, GME Office leadership, and the local chapter of the American Diabetes Association (ADA). Residency program faculty developed the health disparities and diabetes disparities sessions; the hospital’s director of community health developed the CHNA session; residency program faculty and the hospital’s Diabetes Center manager developed the DSME session; and a representative from the local chapter of the ADA developed the session on diabetes resources available in the community.
The lectures were delivered over the course of several weeks during regular didactic periods in spring 2016. The total time allotted was more than 4 hours. The didactic sessions were conducted with the participation of all family medicine, internal medicine, and transitional year residents (a total of 71 residents).
Predidactic and postdidactic session surveys were administered to residents to assess perceptions of gaps in knowledge and increased awareness. On the predidactic (baseline) survey, residents reported opinions about the value of DSME, self-reported rates of resident referrals for DSME, self-reported rates of resident assessments of patients with diabetes risk factors, knowledge of services available at the Diabetes Center, knowledge of existing healthcare disparities, and demographic information. The postdidactic survey included the predidactic survey items and questions about the effectiveness of the didactic presentations. Survey development, completion, and data collection were completed using Qualtrics, a web-based survey software. Two-sample t test assuming equal variances was computed to examine differences in predidactic and postdidactic resident knowledge and use of DSME. A P value ≤0.05 was considered statistically significant.
Plan-Do-Study-Act Cycle II—Problem-Based Learning
During the second PDSA cycle, residents participated in a PBL case on health disparities, the CHNA, and DSME. The case was a modification of a case focused on preventing obesity through community health programs that was originally developed at the University of California Berkeley–University of California San Francisco Joint Medical Program.15 Residents were involved in every aspect of PBL case development and delivery. A resident member of the GME CLER Council suggested using PBL as the second PDSA cycle. Another resident was instrumental in the development of the case, and 9 residents served as small group preceptors during the case discussion.
The case included an introductory section during which several components of the didactic sessions were reviewed. Preceptors were provided a case instructor’s guide, a preceptor copy of the case narrative that included discussion questions and a case group activity worksheet, a copy of the hospital’s CHNA, and a list of resources and links regarding health disparities and PBL. The purpose of the PBL case was to engage medical residents in a subset of the population health competencies for medical students16 that were modified into 3 specific learning objectives:
Discuss the role of socioeconomic, environmental, cultural, and other population-level determinants of health on the health status and healthcare of individuals in an underserved population.
Identify community assets and resources to improve the health of individuals in an underserved population.
Explain how community engagement strategies may be used to improve the health of an underserved population and to contribute to the reduction of health disparities.
The PBL case was discussed in a single 75-minute session in January 2017, approximately 9 months after completion of the first PDSA cycle. The case was tailored to the local Rochester, MI context to make it relevant to participants’ experiences as residents. The session began with small group discussion and review of Crittenton’s CHNA, with particular emphasis on obesity and its effects (eg, diabetes), an identified priority area in Crittenton’s CHNA. Residents were provided copies of the CHNA, and the hospital priority areas were identified. Participants discussed how awareness of hospital priority areas might affect their practice of medicine. The remainder of the PBL case activities centered on the health needs of a simulated family. Participants were encouraged to discuss the health disparities that may be affecting the family; community and school resources and supports that might be helpful; barriers to DSME and other aspects of comprehensive diabetes care; effective methods for educating physicians about community resources; and the connections between health disparities, individual health needs, and community health needs.
Following discussion of the case, participants completed an exercise in which they discussed a health disparity or inequity that they had witnessed or experienced. Discussions included the reason(s) for the disparity, how a CHNA might be used to address the disparity, community resources that could be deployed to mitigate the effects of the disparity, and effective methods of educating physicians about community resources that might reduce the effects of the disparity.
Residents completed a brief evaluation of the PBL case immediately following the session. Residents were asked to rate the effectiveness of the session in meeting its learning objectives; if participation in the PBL case improved understanding of health disparities, the CHNA, and resources for patients with diabetes; if participation triggered any ideas on how they could improve their practice of medicine; and if they would be interested in participating in additional PBL cases and if so, on what topic(s). Descriptive results are provided; no statistical tests were used to analyze results.
Comparison of Didactic Sessions and Problem-Based Learning Case
Two-sample t test assuming equal variances was computed to examine differences in predidactic and postdidactic resident knowledge and use of DSME and to compare the educational effectiveness of the didactic sessions and the PBL session. A P value ≤0.05 was considered statistically significant.
Graduate Medical Education Annual Survey Data
Data related to health disparities and CHNA from the GME annual surveys that were administered 4 months before and 6 months after the didactic sessions were collected and analyzed. GME annual surveys are administered through New Innovations—a residency management software system—in November to evaluate resident perceptions of program effectiveness. The survey has specific sections on education and understanding of the 6 Clinical Learning Environment focus areas designated by the ACGME.3 In the survey, residents were asked in an open-ended question to define healthcare disparities. Three independent raters were provided the Kaiser Family Foundation definition of health disparities and asked to categorize resident responses as correct, partially correct, or incorrect based on the Kaiser definition.1 Residents were also asked to give an example of a social determinant of health that has affected some of their patients, describe the purpose of the CHNA, identify how or where to access their hospital’s CHNA, and to state if they had accessed/viewed their hospital’s CHNA. Two years of survey responses to these questions were tabulated and compared. Qualitative data were evaluated by 3 reviewers including a resident. The intraclass correlation coefficient was computed to determine interrater reliability.
Diabetes Self-Management and Education Referrals
During the course of both PDSA cycles, data were collected on the number of scheduled appointments for DSME at the Diabetes Center that were referred by program faculty and residents. The number of patient appointments for DSME was collected 3 times: for a 2-month period before the didactic sessions, for a 2-month period occurring after the didactic sessions and before the PBL case, and for a 2-month period following completion of the PBL case. The Figure provides an overview of the timeline of project activities and sources of data.
Ethical Considerations
The project was reviewed using guidance provided by the WSU Institutional Review Board and was determined to be exempt. Survey responses, PBL case evaluations, and other project data were collected anonymously, and no identifiable information was collected. No patient information or Health Insurance Portability and Accountability Act–protected health information was accessed. No potential conflicts of interest were identified.
RESULTS
Predidactic and Postdidactic Surveys
Fifty-two residents completed the predidactic survey. Predidactic survey results showed that 82% of residents had referred a patient for diabetes education, 60% of residents think diabetes education is “extremely valuable” for patients, and 33% think it is “very valuable.” Fifty-two percent of residents responded that they assess patients with risk factors for diabetes “all the time” and 27% do so “often.” Twenty-six percent of residents reported that they consider the affordability of diabetes education in referring a patient for it, and 18% responded that they consider the patient’s insurance coverage when making a referral for diabetes education.
Nine residents completed the postdidactic survey. One resident (11%) rated the sessions “excellent,” 5 residents (56%) rated the sessions “good,” and 3 residents (33%) rated the sessions “average.” No residents rated the sessions as “poor.”
Comparison of predidactic and postdidactic survey responses showed no significant differences in resident opinions about the value of DSME, self-reported rates of resident referrals for DSME, or self-reported rates of resident assessments of patients with diabetes risk factors.
Problem-Based Learning Case Evaluation
Forty residents completed an evaluation immediately following the PBL case. Four residents (10%) rated it “highly effective,” 29 residents (73%) rated it “mostly effective,” 4 (10%) rated it “neutral,” 2 (5%) rated it “somewhat effective,” and 1 (3%) rated it “not effective.” Twenty-nine residents (73%) stated that participation improved their understanding of health disparities, the CHNA, and diabetes resources, while 10 (25%) stated their understanding “somewhat” improved and 1 (3%) stated that understanding did not improve. Twenty-eight (70%) residents stated that participation triggered ideas on improving their practice of medicine, 6 (15%) stated that participation “maybe” triggered ideas, and 6 (15%) stated that participation did not trigger ideas for practice improvement. Seventeen residents (43%) stated they would be interested in more PBL cases, while 15 (38%) replied “maybe” and 8 (20%) responded that they would not be interested.
Comparison of Didactic Sessions and Problem-Based Learning Case
The PBL session had a higher mean (3.83) than the didactic sessions (3.78) but not at a statistically significant level (P = 0.4).
Graduate Medical Education Survey Data
In 2015-2016, 67 residents in the family medicine, internal medicine, and transitional year residency programs completed the GME survey; in 2016-2017, 66 residents in the programs completed it. As noted previously, one of the GME survey questions asks residents to define health disparities. The raters were in agreement about the correctness of residents’ definitions. Correctness did not differ by year, with 88% of residents giving at least a partially correct definition. Interrater correlations were high, ranging from r = 0.75 to r = 0.82. The Cronbach alpha for scores was very high (alpha = 0.91), indicating the mean rating is appropriate. The overall mean incorrectness scores were nearly the same for the 2 years; the 2015-2016 mean was 1.54 and the 2016-2017 mean was 1.53 (t[118.85] = 0.08, P = 0.9379).
Each response was categorized as correct, incorrect, or incomplete by combining the ratings of the 3 raters: when 2 or more raters agreed, the agreement categorized the response. When all 3 raters disagreed, the response was categorized as incomplete. Using these category scores, 60% of responses were correct, 28% were incomplete, and 12% were incorrect. Results did not differ by year (2015-2016: 63%, 23%, 14%; 2016-2017: 58%, 33%, 9%; chi-square(2) = 1.89; P = 0.3895). Treating incomplete responses as correct did not reveal a difference between years (chi-square(1) = 0.11; P = 0.7408).
As shown in Table 2, 18% of residents in 2015-2016 and 19% of residents in 2016-2017 stated that social determinants of health had not affected any of their patients. Table 3 shows the responses provided by residents when they were asked to provide examples of social determinants of health that have affected some of their patients.
In 2015-2016, 22 residents mentioned health insurance; of these, 9 mentioned health insurance/medication issues and 3 mentioned health insurance/referrals to specialists. Eleven residents mentioned their patients’ lack of finances (independent of health insurance coverage); of these, 4 residents stated lack of finances to purchase medication or recommended tests. Other social determinants listed by residents include transportation issues and lack of patient health knowledge/understanding/compliance.
In 2016-2017, 10 residents mentioned health insurance. Twenty residents mentioned their patients’ lack of finances; of these, 10 mentioned finances related to obtaining medications. The results for 2016-2017 also show increases in residents listing language/cultural differences and social/educational services deficits and/or inadequate social supports compared to the 2015-2016 results.
Resident responses to questions about their knowledge of the hospital’s CHNA are displayed in Table 4. The percentage of residents who reported that they knew how or where to access their hospital’s CHNA increased from 25% in 2015-2016 to 29% in 2016-2017. Nineteen percent of residents stated they had accessed the CHNA in 2016-2017.
Diabetes Self-Management and Education Referrals
Predidactic and postdidactic data show no effect on the number of DSME appointments for patients referred by residents and program faculty. Six total appointments for DSME were referred by program faculty and residents for the 2-month period immediately before the didactic sessions and 6 appointments were referred for a single 2-month period that occurred between the didactic sessions and the PBL case. In the 2-month period immediately following the PBL case, 9 appointments for DSME were referred by residents and program faculty.
DISCUSSION
The WSU GME annual survey provides a wealth of information related to resident knowledge about health disparities and how social determinants of health are affecting their patients, as well as data about resident familiarity with their hospital’s CHNA. A large percentage of residents accurately defined health disparities in both the 2015-2016 and 2016-2017 GME surveys. One of the key sources of information about health disparities in the hospital community is the CHNA; however, only 25% of residents in 2015 and 29% of residents in 2016-2017 reported knowing how or where to access their hospital’s CHNA. Even fewer (19%) stated they had actually accessed it in 2016-2017. The slight increase in the number of residents knowing how to access the CHNA may be attributed to some residents retaining knowledge from the previous year’s PDSA cycle, considering that the number of new residents who did not receive previous education (37) was nearly equal to the number of residents (38) who did but had graduated and did not participate in the second PDSA cycle.
Healthcare provided by residents in these programs is affected by social determinants of health present in the community and patient population, but a substantial number of residents may not recognize the health disparities in their community. Approximately 20% of residents in both years of the survey thought social determinants of health were not a factor or were not sure. Gaps in healthcare financing (health insurance and other financial issues) among their patients, particularly as it relates to prescription drugs, are commonly encountered by residents at our hospital. The increase in residents listing language/cultural differences and inadequate social support may reflect shifting demographics in the patient population. The shift in social determinants from insurance to finances may reflect a possible decrease in the number of uninsured patients and the financial difficulties related to healthcare that persist despite having insurance coverage.
In predidactic and postdidactic surveys, no significant differences were found in resident opinions about the value of DSME, self-reported rates of resident referrals for DSME, self-reported rates of resident assessments of patients with diabetes risk factors, or knowledge about DSME. The predidactic survey results indicate that residents have varied understanding of disparities in their patient population. Only 26% of residents reported that they consider the affordability of diabetes education in referring a patient for it, and only 18% of residents consider the patient’s insurance coverage when making a referral for diabetes education.
Predidactic and postdidactic data show no effect on DSME appointments for patients referred by residents and program faculty. Following the PBL case, the number of DSME appointments increased.
While there were opportunities for discussion during the didactic sessions, residents preferred the highly interactive and small group discussion format of the PBL case, although not at a statistically significant level. Based on feedback from residents, the greater time allotted for learning during the didactic sessions did not translate into more effective education or more meaningful learning. Peer interaction during the PBL session allowed residents to convey personal experiences related to health disparities that may have contributed to a more valuable learning experience. Peer interactions during the PBL sessions were allowed to move beyond the structured guidelines, and, in fact, did so in many of the groups. Discussions were more organic, while being guided by preceptors to address key questions, and increased engagement in education. PBL was an effective strategy for teaching and learning about health disparities, the CHNA, and diabetes, at least in the short term. Using the PDSA performance improvement model provided a good framework for implementing and evaluating the curriculum.
Rather than providing further instruction on healthcare disparities in general, an educational approach that incorporates the health disparities and social determinants of health in the community aligned with the needs identified in the CHNA may provide more value and better use of limited time and resources. Experiential learning programs with similar approaches have been implemented in residency programs in various locations in the United States; however, educational and clinical outcomes are not currently available.17 Further research about the effects of experiential learning programs on resident understanding of health disparities and community health needs is needed.
Limitations
Our initiative was a single-site study at a suburban hospital with family medicine, internal medicine, and transitional year residents; similar results may not be experienced in other hospitals or settings or in other types of residency programs.
Residents may have been involved in other health disparities, CHNA, and/or diabetes educational experiences during the 18-month project period. Additionally, residents are by definition involved in an educational program and are expected to improve their practice of medicine over time. Any improvement in resident knowledge about health disparities, the CHNA, or the DSME or in referral patterns that result in better health outcomes for patients could be at least attributable to other sponsored activities and/or personal study.
Ideally, all participants in the didactic sessions would have completed both the predidactic and the postdidactic evaluation surveys. The low response rate for the postdidactic survey makes interpretation of results and evaluation of didactic sessions less reliable than if a greater number of responses had been received. The low response rate is attributable in part to the timing of the postsurvey implementation that occurred near the end of the academic year. Many residents who participated in the didactic sessions graduated at around the same time as the postdidactic survey was made available.
We cannot fully attribute the increase in scheduled DSME appointments to participation in the PBL case alone. The hospital may have sponsored other initiatives to increase DSME utilization that we are not aware of. Other factors that may explain the increase need to be investigated. Because we did not collect patient data, we cannot determine if the increased number of DSME appointments was for an underserved population or not.
Next Steps
Faculty Development
Faculty development in health disparities education or QI methods was not provided. Research suggests that the lack of faculty expertise and faculty development in cultural competency and healthcare disparities is a major barrier to effective health disparities and cultural competency education.18 Future health disparities curricula (a future PDSA cycle) at Ascension Crittenton Hospital may include a faculty development component to address this need. Faculty development in QI methodologies, including use of PDSA cycles, is also a need.19
Health Disparities Task Force
The project generated interest in a resident-led task force on health disparities and further QI projects with a health disparities or health equity focus. Plans for the task force include developing and delivering a health disparities component for new resident orientation and other events and learning experiences during residency.
Long-Term Impact of Education
Data will be collected at defined intervals to assess the impact of the initiative on long-term resident referral behaviors. The initiative was focused on all residency programs and on applying education in inpatient and outpatient settings. Using identifiers based on program (internal medicine, family medicine, and transitional year) and referrals made in the inpatient vs the outpatient setting will help focus future education on subgroups. These subgroups may benefit from reinforcement through additional educational sessions designed to address knowledge gaps and other provider barriers related to health disparities.
Experiential Learning Health Disparities Projects in the Local Context
The GME CLER Council discussed project sustainability through resident-led PDSA cycles for experiential learning about health disparities present in the local hospital community and aligned with CHNA priority areas. The GME CLER Council will continue to follow and provide input about further project activities.
CONCLUSION
The project was useful in demonstrating the effectiveness of PBL for teaching residents about health disparities and the CHNA. Identifying the hospital’s priority areas through its CHNA allowed the initiative to be geared toward health disparities present in the patient population and in the local community. The program will be modified to provide a longitudinal formal curriculum on health disparities, including resident-led PDSA cycles tied to specific outcomes. Progress will be fostered by active oversight of the GME CLER Council. The GME Office will monitor long-term educational implications for residents through the GME annual survey, ACGME surveys, and the reporting of publications and presentations through annual program evaluations and the GME Annual Institutional Review. Participation in the AIAMC National Initiative V was essential to program planning, effectiveness, and dissemination.
This article meets the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties Maintenance of Certification competencies for Patient Care, Medical Knowledge, and Practice-Based Learning and Improvement.
ACKNOWLEDGMENTS
The authors have no financial or proprietary interest in the subject matter of this article. The authors acknowledge the contributions of Lisa Dillon, Rachel Klamo, Angela DelPup, Celestial Brock, and Megan Ahee.
A poster and brief presentation describing the project were presented by Brian Benson and Dr Tsveti Markova at the Alliance of Independent Academic Medical Centers National Initiative V meeting in Amelia Island, FL on April 1, 2017.
A poster and brief presentation describing the project were presented by Dr MinhChau Ha at the Southeast Michigan Center for Medical Education Research Forum in Rochester, MI on May 24, 2017.
A poster and brief presentation describing the project were presented by Dr MinhChau Ha at the Michigan Summit on Quality Improvement and Patient Safety in Novi, MI on June 2, 2017.
- © Academic Division of Ochsner Clinic Foundation