Abstract
Background When quality improvement processes are integrated into resident education, many opportunities are created for improved outcomes in patient care. For Bethesda Family Medicine (BFM), integrating quality improvement into resident education is paramount in fulfilling the Accreditation Council for Graduate Medical Education Practice-Based Learning and Improvement core competency requirements.
Methods A resident-developed diabetes management treatment protocol that targeted 11 evidence-based measures recommended for successful diabetes management was implemented within the BFM residency and all physician practices under its parent healthcare system. This study compares diabetes management at BFM and at 2 other family medicine practices at timepoints before and after protocol implementation. We measured hemoglobin A1c (HbA1c), low-density lipoprotein (LDL) cholesterol, and systolic blood pressure (SBP) in adult diabetics and compared patient outcomes for these measures for the first and third quarters of 2009 and 2010.
Results In BFM patients, HbA1c, LDL, and SBP levels decreased, but only HbA1c improvement persisted long term. For the comparison groups, in general levels were lower than those of BFM patients but not significantly so after the first measurement period.
Conclusions A resident-led treatment protocol can improve HbA1c outcomes among residents' diabetic patients. Periodic educational interventions can enhance residents' focus on diabetes management. Residents in graduate medical education can initiate treatment protocols to improve patient care in a large healthcare system.
INTRODUCTION
Bethesda Family Medicine (BFM) is a community-based family medicine residency program operating under the parent organization of TriHealth Inc., a nonprofit health system in the greater Cincinnati, OH, area. TriHealth is part of the Catholic Health Initiative and a member of the Alliance of Independent Academic Medical Centers (AIAMC). The processes within the TriHealth system are designed to bring a uniform evidence-based approach to all outpatient primary care practices under the TriHealth umbrella. The goal is to provide the best evidence-based care to patients through evidence-based treatment protocols developed by physicians and other healthcare providers under the structure of quality committees and educational resources. To achieve this end, a link is needed to bring the latest information regarding best practices to the outpatient settings where physicians provide care on a daily basis.
Much discussion in the graduate medical education (GME) literature currently surrounds integrating quality processes and resident education.1-5 Linking the 2 provides many opportunities for improved outcomes in patient care. Additionally, teaching resident physicians about the quality process delineates the structure needed for practicing physicians to deliver the best care while adhering to constraints regarding time and available resources.
During a 2-year process, the TriHealth quality committee identified chronic diseases that required an evidence-based approach. The committee reviewed recommendations from national organizations and considered the body of evidence available for such disease states as congestive heart failure, coronary artery disease, and diabetes mellitus. The magnitude of the process revealed many opportunities for education and improvement of care. When the organization introduced an electronic medical record (EMR) system in the outpatient setting, processes for tracking patient outcomes became more feasible, and the development of treatment protocols to assist physicians seemed a logical and necessary next step.
Protocol development required resources that medical practices could not consistently devote time to while still maintaining the busy practice of medicine. The family medicine residency seemed a natural fit to assist the TriHealth outpatient primary care practices with the development of evidence-based treatment protocols for chronic diseases. This process would also fulfill the learning requirement of the Accreditation Council for Graduate Medical Education Practice-Based Learning and Improvement competency in resident education.
To this end, we developed an assignment for family medicine residents, the Chronic Disease Management Project. The purpose of the project was twofold: (1) to research and summarize the evidence and recommended guidelines for managing a chronic disease or health maintenance problem and (2) to help develop a practice-wide system for managing chronic disease. For each chronic disease, strategies for completing this project included summarizing guidelines from societies as well as evidence-based guidelines for its management and recommending the frequency of monitoring required for the disease's optimal control. The goal for each chronic disease was to evaluate and summarize recommendations from at least 3 sources and produce a concise guide for treatment. Sources included quality societies (eg, the National Committee for Quality Assurance, the US Preventive Services Task Force, and the Physician Consortium for Performance Improvement); physician professional societies (eg, the American Academy of Family Physicians, the American Congress of Obstetricians and Gynecologists, and the American Heart Association); and web-based clinical information providers such as the Centers for Disease Control and Prevention and UpToDate.
An important consideration in selecting recommendations was the level of evidence on which they were founded. For instance, the most desirable is Level 1 evidence that is based on well-designed and implemented randomized-controlled trials (RCTs). If RCTs were not available, Level 2 evidence from well-designed nonrandomized trials, case-control or cohort studies, or multiple time series was considered. The least desirable was Level 3 evidence consisting of descriptive studies, clinical experience, or expert opinion.6
This process was used for numerous chronic diseases and preventive health topics, including diabetes mellitus, hypertension, hyperlipidemia, coronary artery disease, and prostate and breast cancer. Also undertaken were pediatric topics such as asthma, obesity, and attention deficit hyperactivity disorder. When complete, each evidence-based summary was reviewed with an attending physician member of the institutional Quality Committee. The resident then presented the summary and recommendations to the full Quality Committee for discussion and a vote for adoption within the larger health system. If approved, these treatment protocols were incorporated into the health system and identified as best practice alerts within the system's electronic medical record system.
The Chronic Disease Management Protocol for Adult Diabetic Patients was developed within the BFM residency by a single resident working with a practicing physician. Evidence-based recommendations addressed outcomes with respect to 11 clinical measures: hemoglobin A1c (HbA1c) levels <7%, blood pressure (BP) <130/80 mmHg, an annual eye examination, smoking status, an annual fasting lipids test, low-density lipoprotein (LDL) level <100 mg/dL, LDL 100-130 mg/dL, annual nephropathy assessment, annual monofilament foot examination, HbA1c >9%, and LDL >130 mg/dL. We hypothesized that patients of third-year family medicine residents who had received education on this protocol would have better clinical outcomes than patients of 2 other TriHealth physician practices, one with a BFM faculty member and one with no resident involvement. After implementation of the treatment protocol within the TriHealth system, the research team collected data to compare patient outcomes in 3 TriHealth physician practices: the residency and 2 other practices.
METHODS
We determined that this diabetes management program fit very well into the AIAMC's National Initiative Phase II (NI-II), the goal of which was to demonstrate the link between GME and quality patient care. Consequently, we assembled a team to develop a project for NI-II. Team members included the vice president of academic affairs and designated institutional officer (team leader), a chief resident in the family medicine residency program, a faculty member and medical director of the family medicine residency program, the program director of the family medicine residency program and chair of the TriHealth Physician Enterprise Corporation Quality Committee, and a doctoral-level research specialist (project manager). From a number of quality improvement projects conducted in the family medicine residency program, the recently initiated diabetes management protocol seemed to fit best into the NI-II paradigm.
For the NI-II project we chose 3 from among the 11 evidence-based measures7-10 recommended for successful diabetes management listed earlier: HbA1c levels, LDL cholesterol levels, and systolic blood pressure (SBP). Specifically, we were interested in the degree to which patients served by the TriHealth Physician Partners (TPP) offices were compliant with recommended levels for these measures (ie, HbA1c <7%, LDL <100 mg/dL, and SBP <130 mmHg).
After receiving the approval of the TriHealth Institutional Review Board, the study team collected patient outcome data for a 2-year period (January 2009-December 2010). These data included patient outcomes before and after an educational intervention that acquainted the family medicine residents with the evidence-based diabetes management treatment protocol. The index educational intervention consisted of an hour-long lecture delivered in April 2009 by the resident who developed the protocol, held during the family medicine residency's weekly didactic session. In this lecture, she presented the 11 evidence-based measures and their optimal outcome levels. Following this educational intervention, the treatment protocol became the standard of care to which family medicine residents were expected to conform. These didactic sessions instruct residents on a variety of medical, practical, and professional topics. This educational intervention in particular was delivered to enable the residents to provide improved care to their patients with respect to diabetes management. It was expected that improved care, afforded by the educational intervention, would lead to improved outcomes for their patients.
During the course of the AIAMC NI-II project period, the family medicine chief resident delivered an additional educational intervention to reinforce to residents and teach the new interns the outcomes for the 11 evidence-based measures for diabetes management. Again, this session consisted of an hour-long lecture presented in August 2010 during weekly didactics. We invited all physicians within TriHealth, including those participating in our project, to each of the educational interventions via a flyer. None, other than full-time family medicine faculty members, attended. This result was perhaps caused in part by the midafternoon scheduling of didactic sessions. Another reason may be that physicians had been using pay-for-performance guidelines that are very similar to those we targeted, a fact we did not learn until later.
In addition to determining the outcome measures for residents' patients, we were also interested in comparing these outcome measures with those of other TPP physicians' patients. We hypothesized that because of the recency and exigency of education regarding diabetes management, the residents would perform better on these measures than physicians in practice longer and further removed from an active educational environment. We compared the residents' results to those of 2 other physicians: a GME faculty member and a nonfaculty physician. For selection of the specific comparison groups, TPP offices were divided into 2 groups, one that employed physicians who acted as faculty to the family medicine residency (n=7) and one in which physicians were not involved in resident education (n=8). From each of those groups, one practice was chosen at random. At each of those offices, the first physician approached regarding participation in the study accepted our invitation. Patients of those 2 physicians constituted the participants in the faculty and nonfaculty groups. Following the end of calendar year 2010, we analyzed the data.
Participants
Participants were adult diabetic patients aged 18-75 years from the BFM (residents), White Oak (faculty), and Blue Ash (nonfaculty) offices of the TPP, who were seen by a TPP third-year resident or physician during calendar years 2009 and 2010. We excluded pregnant women and patients on dialysis because of possible confounding by those conditions on the primary outcome measures of this study. Demographics of participants are displayed in the table. Group differences were ascertained using one-way analysis of variance (ANOVA) for continuous variables; Pearson's chi square was used for categorical variables.
Design
This study compared patient data on 3 diabetic scorecard items; hence, the design for this study consisted of 3 between-subjects analyses. A between-subjects analysis compares data for 2 or more independent groups of subjects and/or treatment levels. In this study, 1 between-subjects factor was a measurement with 4 levels: 1 measurement period prior to implementation of the protocol and 3 measurement periods after implementation of the protocol. The 4 levels were first quarter 2009, third quarter 2009, first quarter 2010, and third quarter 2010. First quarter 2009 data included patient outcomes prior to implementation of the diabetes treatment protocol. Data for the third quarter of 2009 and for both quarters in 2010 consisted of patient outcomes after implementation of the diabetes treatment protocol. The other between-subjects factor involved the 3 levels of TPP providers (residents, faculty physician, and nonfaculty physician). The dependent variables were HbA1c levels, LDL levels, and SBP levels. We also collected demographic data. Data for this analysis came from the Clarity data repository from the Epic electronic medical record system (Epic Systems Corporation, Verona, WI).
Procedure
We collected data for this study via 6 Microsoft Excel (Microsoft, Redmond, WA) spreadsheets (3 offices × 2 years). We then imported these data files to IBM SPSS Statistics version 19.0 (SPSS Inc., an IBM Company, Chicago, IL) for analysis. First, for each of the 3 offices, we merged the 2009 and 2010 data files using the patient identification number as the matching key. Next, we merged the resident file with the faculty physician file and then merged the nonfaculty file with the resident-faculty file. We calculated descriptive statistics for HbA1c, LDL, and SBP and graphed means and standard error of the mean error bars for the 4 measurement periods. One-way ANOVA was used to determine if significant differences existed between groups at each time period. Tukey honestly significant difference post hoc tests were conducted for normally distributed data demonstrating homogeneity of variance; Games-Howell tests were used when these assumptions were violated.
RESULTS
Results are illustrated in Figure 1 (HbA1c), Figure 2 (LDL cholesterol), and Figure 3 (SBP). Our findings suggest that exposure to the chronic disease management evidence-based treatment protocol improved several outcomes in our patients with diabetes. In general, the 3 measures either showed improvement with time or remained at respective targets throughout the study period. The additional educational intervention, delivered in August 2010, seems to have influenced continually favorable outcomes for HbA1c but not for LDL or SBP.
Hemoglobin A1c levels of patients for the 3 study groups during the 4 measurement periods over 2 years. Resident levels dropped significantly after Quarter 1 2009 and were sustained throughout the study period. Nonfaculty levels were significantly lower than the other 2 groups. Error bars represent standard error of the mean.
Low-density lipoprotein (LDL) cholesterol levels of patients for the 3 study groups during the 4 measurement periods over 2 years. Resident levels dropped but not significantly after Quarter 1 2009, but the decrease was not sustained. Nonfaculty physician levels were significantly lower than the other 2 groups at baseline but trended upward over subsequent periods, losing significance by period 3. Error bars represent standard error of the mean.
Systolic blood pressure levels for the 3 study groups during the 4 measurement periods over 2 years. Resident levels dropped, although not significantly, after Quarter 1 2009, but the decrease was not sustained. Faculty and nonfaculty levels did not change appreciably, although increases for residents led to a significant difference in periods 3 and 4. Error bars represent standard error of the mean.
HbA1c
HbA1c levels dropped significantly after baseline for residents' patients, reflecting improved care following treatment protocol implementation (P=0.021); this effect was sustained through period 4 (Figure 1). HbA1c levels for patients of the faculty and nonfaculty physicians did not change appreciably, although one-way ANOVAs revealed that the nonfaculty physician's patients performed significantly better than the other 2 groups in all measurement periods (both P≤0.035). Games-Howell post hoc tests confirmed that in comparison to the residents, patients of the nonfaculty physician had significantly lower HbA1c levels at periods 1, 2, and 4 (P=0.002, 0.044, and 0.003, respectively). Additionally, patients of the nonfaculty physician had lower HbA1c levels than those of the faculty physician at period 2 (P=0.03).
LDL Cholesterol
LDL levels dropped, although not significantly, for residents' patients after treatment protocol implementation, but this change was not sustained throughout the 2-year study period (P=0.572). A series of one-way ANOVAs revealed a significant difference between groups only at period 1 (P=0.007). The Games-Howell test confirmed that levels of patients of the nonfaculty physician were significantly lower at baseline compared to the other 2 groups (P=0.038). Although this measure trended upward over the remaining study period and lost significance by period 3, it remained below the clinical target of 100 mg/dL (Figure 2).
SBP
SBP dropped, although not significantly, for residents' patients after the treatment protocol implementation (P=0.603), but again this change was not sustained through the end of the study. SBP levels were lower, although not significantly, at baseline for patients of the faculty and the nonfaculty physicians (P=0.252). SBP levels did not change appreciably for patients of the faculty physician or the nonfaculty physician. However, the residents' patients' increases at periods 3 and 4 represented significantly higher levels than the other 2 groups at period 3 (both P<0.001) and period 4 (P=0.002 and 0.003, respectively). Still, these levels remained below 135 mmHg, which we found encouraging because this level places the patients in the middle of the high normal category for SBP (Figure 3).
DISCUSSION
Our results demonstrate that HbA1c levels for patients of third-year family medicine residents improved significantly and were sustained throughout the 2-year study period following implementation of a diabetes management treatment protocol. On the other hand, LDL and SBP levels improved initially but then crept back up. We are encouraged that lower levels for the principal marker for diabetes control, HbA1c levels, were sustained. Going forward, however, we believe that conducting periodic educational interventions for residents would continually raise awareness and thereby reinforce their focus of attention on the additional markers for diabetes management.
We found noteworthy the fact that the faculty and nonfaculty physicians received financial incentives to encourage better compliance with these diabetes measures during the study period. Unfortunately, we did not learn until the study was well underway that a pay-for-performance incentive for TPP physicians had been in place since 2005. Had we known about this incentive during the study design phase, we would have chosen a more appropriate comparison group. With the benefit of hindsight, the significantly lower levels for these measures in the TPP physicians' patients are unsurprising.
Another important caveat relating to the participants in this study is that the patients of the BFM residency are of significantly lower socioeconomic status, as measured by 2010 median household income, compared to those served by the faculty and nonfaculty physicians (both P<0.001).11 We speculated that this disadvantage might relate to their generally poorer outcomes. Although a few associations were evident for each of the diabetes measures, the number of significant correlations did not exceed chance, and no distinct pattern of association emerged.
Several limitations should be taken into account when interpreting our findings. First, we did not select the participants randomly. The lack of randomization can limit the external validity of our findings (ie, our ability to generalize the findings to other programs). Also, our study design was quasiexperimental rather than a randomized controlled trial, which may limit the internal validity of the study (ie, our ability to conclude that our intervention led to the above improved outcomes). However, this design does seem superior to most continuous quality improvement projects because we included 3 study groups: an intervention group (resident physicians) and 2 control groups (faculty and nonfaculty physicians). Lastly, other ongoing quality improvement initiatives—such as those addressing issues such as coronary artery disease/stroke and smoking status (with similar outcome measures as diabetes)—were taking place during the study period. Attention to those initiatives might have increased vigilance generally, thereby leading to the improved outcomes we observed.
Two minor changes to our initial study design occurred during the project. Initially, we planned for the design to include repeated measures analysis. However, because the number of patients with data points in all measurement periods throughout the 2-year study was very low, we changed our design to between-subjects measures at quarterly time intervals (ie, a continuous quality improvement design). In addition, all of the third-year residents, rather than just 1, were used as the experimental group. Because we had too few data points when considering the patients of a single resident and because all residents were exposed to the initial educational intervention and implemented the protocol, we deemed the inclusion of all residents necessary to increase statistical power. Finally, to ensure continued exposure to the material, approximately 1 year into the study period the chief resident delivered an additional lecture regarding the diabetes management treatment protocol. Unfortunately, the timing of this second lecture may have been too close to the end of the data collection period to greatly influence the outcome measures.
CONCLUSIONS
This study demonstrates that GME can be a catalyst for improvement in patient outcomes in a large healthcare system. Although this study was small, it serves as a model for how we will focus on quality improvement in GME and the entire TriHealth organization in future work, linking residents with practicing physicians not only in primary care but in the specialties as well. As we move forward with quality improvement in our institution, we believe that our GME programs will be the catalyst for a robust process that brings evidence-based medicine to the community practices. Indeed, in the next phase, physicians will have access to evidence-based medicine through EMR technology in the examination room as they provide care to the patient. Treatment protocols, such as the one for diabetes management studied here and others developed under our GME program, formed the basis of guidelines in our EMR. Future work will likely involve larger studies across a wide variety of specialties and range of TPP offices. We cannot understate the value of the impact a residency program can have on outcomes in patient care. Periodic educational interventions should be conducted to enhance residents' focus on diabetes management.
Epitoma
Using protocols to standardize practice and improve consistency has been shown to improve the process of care in chronic disease management. As part of the Alliance of Independent Academic Medical Centers National Initiative II, the Bethesda Family Medicine residency program in Cincinnati, OH, developed a protocol and resident education program to improve outcomes among diabetic patients with regard to hemoglobin A1c, systolic blood pressure (SBP), and low-density lipoprotein (LDL). They were only able to demonstrate a significant drop in hemoglobin A1c over time as there was no significant improvement in SBP or LDL. The authors also did not report if the process of care improved with the protocol. However, this study does demonstrate that using a quality improvement methodology within a resident clinic can lead to improved outcomes with regard to diabetic care while teaching residents about Systems-Based Practice and Practice-Based Learning and Improvement.
—Guest Editor Leonard Seoane, MD
This article meets the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties Maintenance of Certification competencies for Patient Care, Medical Knowledge, Systems-Based Practice, and Practice-Based Learning and Improvement.
Patient Demographics by Group
ACKNOWLEDGMENTS
The authors extend a sincere thank you to Dr Kelly Brown and Dr Denise Stretcher for their enthusiastic participation in our study.
Footnotes
The authors have no financial or proprietary interest in the subject matter of this article.
- Academic Division of Ochsner Clinic Foundation
In this issue
Jump to section
Cited By...
- No citing articles found.