Abstract
Background: Patient perception of care is positively associated with better medical outcomes, clinician job satisfaction, and fewer malpractice claims and also has significant downstream economic impact for healthcare organizations. A sparse amount of data exists regarding provider and practice characteristics driving high levels of patient experience in the pediatric hematology-oncology (PHO) ambulatory setting. The aims of this study were to determine key drivers of high care provider ratings and of the likelihood of recommending our ambulatory PHO practice.
Methods: Patient experience was measured using the Consumer Assessment of Healthcare Providers and Systems Clinician & Group Survey (CG-CAHPS). The study outcomes were to determine the survey items most associated with top-box scores for “Rate This Provider” and for “Likelihood of Your Recommending Our Practice to Others.”
Results: The survey items Explanations the care provider gave you about your problem or condition and Concern the care provider showed for your questions or worries were most strongly correlated with high provider ratings. How well the staff worked together to care for you and Friendliness/courtesy of nurse/assistant were most strongly correlated with the likelihood of recommending the practice.
Conclusion: High provider ratings were most associated with the physician’s ability to explain problems and conditions to families. Staff teamwork and nursing attitude were most associated with patient recommendations of the PHO practice. By identifying key drivers of high provider and practice ratings by patients in the PHO ambulatory setting, a targeted approach with a focus on physician-specific communication attributes, teamwork, and nursing attitude can be deployed to improve the patient experience.
INTRODUCTION
Patient experience and perception of care are growing areas of importance in healthcare although these concepts are not novel. As early as the fifth century BC, Hippocrates is credited with writing, “Any man who is intelligent must, on considering that health is of the utmost value to human beings, have the personal understanding necessary to help himself in diseases, and be able to understand and to judge what physicians say and what they administer to his body, being versed in each of these matters to a degree reasonable for a layman.”1 Yet only recently has the healthcare field mandated quality, safety, and patient satisfaction as priorities.2 A growing body of evidence suggests that exceptional physician-patient communication is associated with high levels of patient satisfaction3 and increased patient compliance with treatment recommendations.4 In fact, studies in multiple disciplines have demonstrated a positive association between patient experience and medical outcomes, clinical quality, and patient safety measures.5-9 Additionally, a direct relationship between patient satisfaction and physician job satisfaction has been observed,10 and an inverse relationship with malpractice claims has been described in several studies.11-13
Furthermore, patient perception of care has significant downstream economic ramifications for healthcare organizations. Performance in key patient experience metrics has been shown to be predictive of patients’ recommendations of hospitals and medical practices.14 Additionally, in the value-based purchasing era, reimbursement and pay-for-performance measures are tied to patient satisfaction metrics.15 Therefore, patient perception of care is critical to organizational health more than ever before, given the increasingly competitive healthcare environment in an era of rapid information dissemination and online communication.
Despite the emerging evidence tying patient perception of care to critical medical and economic outcomes, many clinicians have not embraced the concept of using patient satisfaction data to improve care.16 In fact, eliciting opinion regarding patient perception of care (via the Consumer Assessment of Healthcare Providers and Systems Clinician & Group Survey [CG-CAHPS]) can be a valuable tool for improving care at the individual provider level, within clinical microsystems, and at large healthcare networks. The CG-CAHPS survey is standardized, validated, and applicable to both adult and pediatric patients, as well as to general and specialty practices. Prospective patients can access and use publicly available survey results when choosing providers or healthcare organizations.17
In pediatrics, the factors most responsible for patients’ overall satisfaction with care appear to be dependent on discipline and setting (ie, ambulatory, inpatient, subspecialty, emergency).18 A need for a focused oncology-oriented approach to patient satisfaction has been suggested,19 with evaluations of patient experience in medical oncology underway in London20 and Scotland.21 Expert opinion has also been solicited in an attempt to identify components of patient experience thought to be most important to medical oncology patients.22 Only one study has aimed to improve patient satisfaction in the subspecialty of pediatric hematology-oncology (PHO).23 Although improvement in patient experience was demonstrated, the key drivers most associated with favorable physician and practice ratings were not elucidated. One study identified factors associated with recommending PHO practices in a large integrated healthcare system24 but did not examine the physician-specific attributes that drive high provider ratings in this subspecialty. Theoretically, identifying key drivers of the patient experience within this particular clinical microsystem could ultimately improve care, if the data are used to spearhead quality improvement initiatives.
Identifying physician-specific characteristics and medical practice qualities important to the experience of the epidemiologically and diagnostically diverse PHO patient population is vital. In a response to an organizationwide prioritization of patient experience metrics and practice goals, our study aim was 2-fold: to determine key drivers of high care provider ratings and to determine key drivers of the likelihood that patients would recommend our midsize ambulatory PHO practice.
METHODS
Design
Using a cross-sectional descriptive survey design, the study was conducted at a midsize PHO clinic in the Midwest United States from January 2014 through May 2017. Institutional review board (IRB) approval was obtained (IRB #IM2016-063), and the study complied with the IRB ethical guidelines. Study authors completed online human subject protection training, and all surveys were de-identified of patient information and coded by CG-CAHPS number.
Survey Instrument
Patient experience was measured using the CG-CAHPS survey. The survey is a well-validated methodology and, although usually distributed randomly, was distributed to 100% of eligible patients by the third-party vendor to maximize the number of responses. All patients were eligible to receive a survey unless they had already submitted one for a visit within the preceding 90 days. The CG-CAHPS instrument included 25 items scored on a 5-point Likert type scale, as well as a “Rate This Provider” item scored on a 0- to 10-point scale, with higher scores representing a better rating.
Clinical Demographics
The clinic is a midsize semiprivate practice with 5 full-time physicians. Other clinic staff include 4 nurses, 2 patient access associates, 3 research assistants, 1 patient care technician, 2 social workers, 1 child life specialist, 1 psychologist, 1 nurse coordinator, and 1 scheduler. The clinic cares for approximately 60-70 new oncology patients per year, more than 1,000 existing oncology patients, and a large population of patients with primary hematologic disorders. The clinic receives approximately 5,000 patient visits per year. Patients are ages 0-21 years and represent a variety of ethnicities and socioeconomic backgrounds. Approximately 40% of patients have Medicaid insurance.
Analysis
The first study outcome was to determine the survey items most associated with a top-box score (defined as a rating of 9 or 10) for “Rate This Provider.” The second study outcome was to determine the survey items most associated with a top-box score (defined as a rating of 5, “very good”) for “Likelihood of Your Recommending Our Practice to Others.” Survey items were dichotomized based on a response score of 5 or a score <5.
A preliminarily data review was conducted to examine survey response missingness, internal validity, and the need to control for provider-level effects. Approximately 0%-2% of data were missing for individual questions with signs of monotonicity, and 6%-8% of data were missing for questions on visit delays and patient medications. The greater percentage of missingness for these 2 questions was presumed related to respondents interpreting questions as not applicable when they had no delays or medications to report. Missingness was then treated as missing completely at random and not imputed. Cronbach alpha was used to review the survey’s internal validity and was deemed nonconsequential. An empty multilevel model used to examine the need for provider-level random effects revealed that controlling for provider-level effects did not significantly explain additional response variability. Based on these preliminary reviews, study analyses were based on complete case data without the use of random effects.
Bivariate Spearman correlations were calculated between survey item responses to provide a general examination of the coefficients. Based on these values, collinearity was deemed a possible concern, and data were partitioned into 2 random subsets based on a 60/40 split. The larger data partition was used for feature selection via least absolute shrinkage and selection operator (LASSO) logistic regression (glmnet package: R). The lambda value in the model was tuned using 5-fold cross-validation and the one-standard-error rule for creating the most regularized model. This modeling approach functioned to address collinearity and outcome sparsity concerns.25 A post hoc nonparametric model (R package: randomForest) was also fit to this data partition to check for model dependence. This secondary feature selection process generated a variable importance list using a nonlinear model, which corroborated the selection of the same features as the LASSO model. Subsequently, a Bayesian logistic regression (PROC GENMOD, DIST = BIN, BAYES option: SAS 9.4) model was fit on the smaller data partition to generate association estimates for the selected features. In particular, the model used informative priors (ie, mean = 0.75; variance = 0.15) and Markov chain Monte Carlo sampling (ie, chains = 3; thinning = 5) based on 100,000 iterations and a 50,000 burn-in rate. The selection of the priors was influenced by reported results from Davis et al.24 Generated association estimates are presented as adjusted odds ratios (OR) with 95% credible intervals (CI). A sensitivity analysis was performed repeating the Bayesian modeling process but with the use of nonformative priors (ie, mean = 0; variance = 1,000).
RESULTS
Sample Characteristics
During the study period, 281 surveys of 1,886 eligible patient encounters (14.9%) were returned. Review of survey responses revealed strong internal validity within the CG-CAHPS instrument (ie, standardized Cronbach alpha = 0.97). Survey data were also strongly correlated between response items (ie, average Spearman rank correlation coefficient for all possible survey combinations was r = 0.54 with standard deviation = 0.15). Uncollapsed survey responses scale data are presented in Figure 1. The distribution of responses for the survey outcome measures is presented in Figure 2.
Provider Rating
In the full dataset, a top-box score (a rating of 9 or 10) for “Rate This Provider” was reported in 90.0% (253/281) of surveys. The survey items with the strongest correlation with a top-box provider rating were Explanations the care provider gave you about your problem or condition (r = 0.65) and Concern the care provider showed for your questions or worries (r = 0.65) (Table 1). All 10 care provider–specific metrics were strongly correlated with “Rate This Provider” scores. A “very good” response for provider explanations was associated with a 4.3-fold (95% CI 2.0, 7.0) greater likelihood of a physician top-box rating compared to respondents who did not mark “very good” on the explanations item. Sensitivity analysis for the item based on flat priors revealed an OR of 35.4 (95% CI 2.9, 97.2).
Recommending the Practice
In the full dataset, a top-box score (a rating of 5, “very good”) for “Likelihood of Your Recommending Our Practice to Others” was reported in 88.7% (244/275) of surveys. Six patients did not provide a rating for the survey item. The survey items with the strongest correlation with a top-box recommendation rating were How well the staff worked together to care for you (r = 0.79) and Friendliness/courtesy of the nurse/assistant (r = 0.75) (Table 2). Unlike with the provider ratings, items from multiple domains (ie, Overall Assessment, Nurse/Assistant, and Care Provider) correlated strongly with “Likelihood of Your Recommending Our Practice to Others.” A “very good” rating for How well the staff worked together to care for you was associated with a 5.1-fold (95% CI 2.3, 8.4) greater likelihood of recommending the practice compared to respondents who did not mark “very good” on the worked together item. Sensitivity analysis for the item based on flat priors revealed an OR of 6.5 (95% CI 0.5, 16.5).
DISCUSSION
Study results demonstrated that patients at the PHO clinic value provider communication attributes and teamwork among the clinic staff. The survey metrics correlating with high provider ratings not only demonstrate the significant impact the treating physician has on the patient’s perception of care but also contradict a commonly held presumption that patient ratings of superficial components within the continuum of care (ie, food quality, parking availability, amenities) significantly impact provider and practice evaluations. Few studies have analyzed the patient experience in ambulatory pediatric subspecialty clinics, although Ye et al found that physician-specific attributes were most associated with overall satisfaction among pediatric subspecialty clinics as a whole.18
To our knowledge, no prior studies have been conducted to identify key drivers of patient experience for PHO physician ratings. The patient population served by our PHO clinic highly valued physician communication attributes. Physician explanations of complex problems and medical conditions, while a vital skill in most areas of clinical medicine, are likely a critical component of patient interaction in the highly technical and scientific field of PHO. Almost of equal importance to our patients was physicians’ concern for their questions or worries. The PHO clinic is a particularly anxiety-provoking environment for many patients, even those without malignancy. While showing concern for patients is an essential physician attribute in general, the ability to alleviate anxiety and concerns is an imperative characteristic for the PHO physician at the bedside and of paramount importance to patients when evaluating their provider.
In terms of recommending the practice, patients of our PHO clinic most valued a staff that works together well, with the data demonstrating that perceived teamwork among the staff was strongly associated with patients recommending the practice to others. In 2017, Davis et al evaluated this same outcome measure for outpatient PHO clinics in a large integrated healthcare system,24 and our study independently corroborated some of the key findings from their report. Similar to the Davis et al study, we identified teamwork as one of the most critical CG-CAHPS predictors of recommending the practice. Patients in PHO practices receive care from numerous professionals who are not physicians (eg, nurses, technicians, psychologists, and social workers), and our study underscores the importance of the entire team in the lives of these patients and families. We speculate that the multidisciplinary nature of the field and the diversity of the healthcare team required to effectively care for these complex patients may explain the importance of teamwork to this patient population.
The importance of the perception of teamwork to patients’ evaluations of ambulatory pediatric subspecialty practices does not appear to be unique to hematology-oncology. A study in outpatient pediatric cardiology, a discipline with an acuity and scientific complexity similar to hematology-oncology, demonstrated that teamwork and explanations of problems/conditions were the CG-CAHPS metrics most associated with practice recommendations, metrics that had a comparable impact in our study.26 In a study of the ambulatory pediatric neurology setting, staff teamwork was also identified as being strongly associated with recommending the practice to others.27 Our study underscores the importance of both actual and perceived teamwork among physicians, nurses, and other staff members.
Davis et al also identified cheerfulness of practice as being a top predictor of practice recommendations, although this metric was institution-specific and not a standard CG-CAHPS or Press Ganey survey item.24 Similarly, our results suggest that staff attitudes significantly influence patients’ perception of the practice, as Friendliness/courtesy of the nurse/assistant was the second most important predictor of patient recommendations. Our study confirms that patients in the ambulatory PHO setting, who often identify closely with members of the nursing team, place considerable value on nursing attributes when evaluating the PHO practice.
In contrast to the Davis et al study, we did not identify wait time as a strong predictor of patient perceptions of the practice.24 In fact, in our study, patient flow metrics were least correlated with recommendations of the practice. This finding may be attributable to differences in actual wait times, interventions already in place affecting perceived wait times, or individual practice characteristics. More research needs to be done to quantify the impact of patient flow metrics on patient perceptions of care in PHO clinics.
By identifying the key drivers of high provider and practice ratings, a targeted approach with a focus on physician-specific communication attributes and teamwork among physicians, nurses, and staff can be deployed to improve the patient experience. These data could be used to spearhead quality improvement programs targeting communication and empathy training for physicians, and other interventions could focus on improving team dynamics and care coordination. Additionally, given the vital role of nursing care in ambulatory PHO, interventions could be put in place to augment the abilities of this critical component of the care team.
In the field of PHO, significant financial and workforce resources are allocated to novel drug development, laboratory research, and national and international clinical trials. Significant capital is invested to incrementally improve disease outcomes, reduce toxicities, and decrease morbidity. Optimizing patients’ perception of care may be a crucial and underused component of care delivery in this population. Future research should be aimed at measuring compliance, morbidity, and cost in relation to patient experience in PHO.
Several factors could have restricted our study findings. Because of anonymity, unique patient demographic characteristics were not available for use in analyses or reporting. Survey items were collapsed into binary groups because of sparse variability in item response groups. Additionally, the sample was taken from a convenience data source with candidate variable items and outcomes collected at the same cross-section of time from the same data source. Collinearity was a concern between individual survey item questions and warranted the use of a regularization based model for feature selection. Given the study period length, individual respondents could have theoretically completed more than one survey during the study period, resulting in possible response dependencies. However, this likelihood is low because each visit is supposed to be considered a unique encounter by the respondent, and patients were ineligible to complete more than one survey within 90 days.
Precision measures for association estimates were wide given the sparsity of respondents providing low scores on instrument items and because of the sample size constraint. Intervals were also large in the flat prior models, revealing shrinkage related to use of informed priors. Also, only a small percentage of patients in the sample were dissatisfied with their care, creating outcome variable imbalances. Additionally, although the CG-CAHPS survey is considered the gold standard for measurement of patient perception of care, the use of any survey data can include a potential nonresponse bias. The return rate for the PHO clinic ranged from 6.8%-31.8% monthly and was 14.9% overall. The national CG-CAHPS return rate for PHO clinics is not known (Michael Corrothers, The CAHPS Database, email communication, March 2018); however, our response rate is in line with national expected Press Ganey return rates of 18.4%-19.3% (Joseph Foli, Press Ganey Associates, Inc., email communication, January 2018), a comparable methodology that overlaps our institutional CG-CAHPS survey. Nonetheless, the relatively low return rate is a potentially limiting factor in this study.
CONCLUSION
This study examined our patients’ perception of care in the PHO ambulatory setting and determined that this patient cohort most strongly associated a high provider rating with the provider’s ability to explain problems/conditions to families and show concern for patient questions or worries. Study data also associated the likelihood of recommending the practice to others most strongly with staff teamwork and nursing attitude.
This article meets the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties Maintenance of Certification competencies for Patient Care, Medical Knowledge and Interpersonal and Communication Skills.
ACKNOWLEDGMENTS
These data were presented at a meeting of the American Society of Pediatric Hematology-Oncology, on April 26, 2017, in Montreal, Canada. The authors have no financial or proprietary interest in the subject matter of this article.
Thank you to Paige Moore, MHA, Toni Clark, MHA, CPXP, and Ken Cheyne, MD for their assistance with this study.
- © Academic Division of Ochsner Clinic Foundation
REFERENCES
In this issue
Jump to section
Cited By...
- No citing articles found.