Background

Burnout is an occupational condition characterized by emotional exhaustion, depersonalization, and a low sense of personal accomplishment.1,2 In healthcare, burnout is thought to contribute to poor outcomes, including worse patient safety,37 and to lower patient satisfaction.810 Burned-out employees are more likely to leave their jobs,1,1115 take sick leave, and suffer from depression and relationship problems.1618 Burnout affects nearly half of all U.S. nurses and physicians,1,19 and is significantly more prevalent among physicians than in the general U.S. population.20 For that reason, burnout has been a major topic of health services research in efforts to better understand its causes and solutions.2125

The most common measure of burnout is the Maslach Burnout Inventory (MBI), a well-validated,26 widely-used self-survey measure.27,28 The MBI comprises three scales: 1) emotional exhaustion (nine items), a state of chronic emotional and physical depletion; 2) depersonalization (five items), a sense of disconnection from coworkers and clients; and 3) diminished personal accomplishment (eight items), a negative sense of self-value and ability.29,30 The MBI has been validated among both physicians31 and nurses,32 and has been found to accurately discriminate between populations of employees who are not suffering from burnout and those clinically diagnosed with burnout.33 While the original MBI Human Services Survey comprises 22 items,34 West and colleagues have validated single items from the MBI emotional exhaustion (MBI:EE) and MBI depersonalization (MBI:DP) subscales as standalone measures.35 They found that, compared to the full scales, single items demonstrated strong psychometric validity, both in terms of meaningfully stratifying risk of high burnout35 and exhibiting strong, consistent associations with outcomes (e.g., suicidality, perceived major medical error, serious thoughts of dropping out of medical school), such that the association between burnout and the outcome was not altered by the use of the single-item MBI versus the full MBI.36 The single-item version of the MBI scales was subsequently used in a major national survey of medical residents.37

The MBI is a proprietary assessment tool and carries licensing fees, making its routine or widespread use, such as in repeated monitoring of employee burnout, potentially cost-prohibitive. An alternative non-proprietary, single-item measure of burnout exists, and has been used successfully in a variety of settings. It was first introduced in a survey of HMO physicians by Schmoldt and colleagues,38 and has also been used in the Physician Work Life Study,39 the Minimizing Error, Maximizing Outcome (MEMO) study,40 and the Mississippi Workforce Study.41 Rohland and colleagues compared this single-item version with the full MBI-HSS in a cohort of 307 Texas medical school graduates,42 and Hansen and Girgis compared the two versions in a survey of 740 Australian oncology health care professionals that included physicians and nurses, among others.43 Despite the differences in sample size and populations, the psychometric findings in these two studies were almost identical: the ANOVA R2 between the MBI:EE subscale and the non-proprietary single-item burnout measure was 0.50 in both, while the simple correlation (r) between the two measures was 0.64 in the study by Rohland and colleagues42 and 0.68 in Hansen and Girgis’ study.43 They both concluded that this single-item measure of burnout could be effectively used as an alternative to the MBI:EE to screen for burnout, especially where emotional exhaustion was the primary subscale of interest and a shorter survey was needed. However, the sample sizes in these studies were small, and to our knowledge, no one has evaluated the performance of this non-proprietary single-item measure of burnout in subsets of U.S. health care workers other than physicians, such as non-physician providers and primary care clinical staff.

Objective

We compared the non-proprietary single-item burnout measure from Rohland and colleagues to a single-item MBI:EE using a large national sample of health care workers from primary care.44 We assessed the performance of the two measures across four distinct occupational classes: providers, registered nurses (RNs), clinical associates (e.g., licensed practical nurses (LPNs), medical technicians), and administrative clerks.

Design

We conducted a cross-sectional survey, administered online. Data were collected during a six-week period in May–June 2012. This work was conducted as part of an operational quality improvement evaluation of VHA primary care. Per VHA Handbook 1058.05, we obtained signed documentation of non-research status from the VA program office overseeing the evaluation (documentation provided upon request.) The survey was voluntary, and we offered no individual-level incentives.

Participants

The target population was all Veterans Health Administration (VHA) primary care personnel, and included respondents from 630 of the 913 primary care clinics operating at the time. We focused our analyses on respondents from the four occupations that comprise primary care teams: providers (physicians, nurse practitioners, and physician assistants), RNs, clinical associates (e.g., LPNs, medical technicians), and administrative clerks. The response rate was approximately 25 %.44

Main Measures

The non-proprietary single-item burnout measure instructs respondents to define burnout for themselves: “Overall, based on your definition of burnout, how would you rate your level of burnout?” Responses are scored on a five-category ordinal scale, where 1 = “I enjoy my work. I have no symptoms of burnout;” 2 = “Occasionally I am under stress, and I don’t always have as much energy as I once did, but I don’t feel burned out;” 3 = “I am definitely burning out and have one or more symptoms of burnout, such as physical and emotional exhaustion;” 4 = “The symptoms of burnout that I’m experiencing won’t go away. I think about frustration at work a lot;” and 5 = “I feel completely burned out and often wonder if I can go on. I am at the point where I may need some changes or may need to seek some sort of help.” This item often is dichotomized as ≤2 (no symptoms of burnout) vs. ≥3 (1 or more symptoms).38,39

The single item from the MBI:EE that West and colleagues validated as a standalone burnout assessment is36: “I feel burned out from my work.” This item appears in the MBI:EE subscale of both the MBI Human Services Survey and MBI General Survey (the two variants of the MBI typically used in health services research), and responses are measured on a seven-point frequency scale ranging from 0 “Never” to 6 “Every day”. They defined “high levels of burnout” as feeling burned out at a frequency of “once a week” or more (a score greater than or equal to 4). This item and two other items from the emotional exhaustion subscale of the MBI-GS were chosen by Leiter and Shaughnessy based on results from a structural equation model in which they minimized correlations among item errors.45 The two other emotional exhaustion items were: “I feel tired when I get up in the morning and have to face another day on the job,” and “Working all day is really a strain for me.” In sensitivity analyses, we also compared the single-item burnout measure to Leiter and Shaughnessy’s three-item MBI:EE.

Respondent Characteristics

The survey also collected respondent characteristics, including tenure with the VA (eight categories), supervisory level (six categories), age (six categories), sex, race (six categories), Latino ethnicity (dichotomous), and occupation (we used four categories).

Analytical Procedure

In order to compare the performance of the non-proprietary single-item measure to the single MBI:EE item, we examined the Pearson correlation between the original ordinal scale values of the two measures as well as several measures of agreement and discrimination between the dichotomized measures (i.e., based on a positive screen for burnout): kappa values, sensitivity and specificity, and positive and negative predictive values (PPV & NPV). We also conducted a receiver operator curve analysis to calculate the area under the curve (AUC). Analyses were conducted with all respondents combined and separately within each of the four occupations: PCP, RN, clinical associate, and clerk. We calculated all statistics and conducted analyses using Stata (SE) version 12 (StataCorp., College Station, TX, USA).

Key Results

Our analysis included 5,404 respondents (1,769 providers, 1,380 RNs, 1,358 clinical associates, 557 clerks). Fifty-two percent were 50 years of age or older, 38 % had been with the VA more than 10 years, and 39 % had some supervisory responsibilities (Table 1).

Table 1 Respondent Characteristics

The prevalence of burnout was 36.7 % as measured on the single MBI:EE item and 38.5 % as measured on the non-proprietary single-item measure (Table 2). Burnout varied substantially by occupation, and was highest for providers and lowest for clinical associates. Results from the measures of agreement are presented in Table 3. The Pearson correlation between the two burnout measures was 0.79. A full cross-tabulation of the frequencies for the two single-item burnout measures is included in Table 4. In a comparison of measures of agreement on whether the respondent was burned out, the inter-rater agreement (kappa) was 0.70, indicating 70 % greater agreement between the two measures than by chance alone. Compared to the single MBI:EE item, the non-proprietary single-item measure had sensitivity of 83.2 %, specificity of 87.4 %, PPV of 79.3 %, and NPV of 90.0 %. The AUC was 0.93 (se = 0.004). When we assessed discrimination statistics separately by respondent occupation, we found similar results across occupations. We also repeated analyses with the three-item version of the MBI:EE used by Leiter and Shaughnessy,45 and found similar results (electronic supplementary material, available online).

Table 2 Cross-Tabulation of Burnout Assessed by the Single-Item Measure and the Maslach Burnout Inventory Emotional Exhaustion Scale, for the Entire Sample
Table 3 Burnout Prevalence, and Agreement and Discrimination Statistics, Overall and by Occupation
Table 4 Cross-Tabulation of Burnout Values for the Non-Proprietary Single-Item Burnout Measure and the Single-Item Maslach Burnout Inventory Emotional Exhaustion (MBI:EE) Measure*

Conclusions

We found that a non-proprietary single-item measure of burnout was a viable substitute for the one-item MBI:EE validated by West and colleagues, with a high Pearson correlation and area under the curve. We compared the burnout measures among four very different occupational classes (providers, RNs, clinical associates, and administrative clerks), with significant differences in prevalence of burnout, and found remarkably consistent results. The single-item measure has the important advantage of carrying no licensing fee, which for the MBI was $2.00 per use for small-volume administration at the time of this paper. The single-item measure is also easy to interpret, with the response scale explicitly indicating where a change in values signals symptoms of burnout (3 = “I am definitely burning out”) versus no burnout (2 = “don’t feel burned out”).

Our findings are similar to two previous, smaller studies that compared the single-item burnout measure to the full MBI:EE in a sample of physician alumni from a single institution42 and in a sample of Australian cancer care workers.43 In the latter, the investigators found a Pearson correlation of 0.66 between the two versions, with the MBI:EE producing a burnout prevalence of 32.0, vs. 28.2 % for the non-proprietary single-item measure. In other analyses, we identified elements of medical home team-based care that were associated with burnout, using both the non-proprietary single-item measure and the three-item MBI:EE, and found that the factors associated with each measure were virtually identical,44 suggesting that small absolute differences between the two measures in the prevalence of burnout would not likely bias studies of factors contributing to burnout.

Limitations

The present study has three notable limitations. First, we compared the single-item measure to a one-item MBI:EE,36 not the full MBI:EE, and it is possible that different results would be achieved with the full MBI:EE. We did not have the full MBI:EE available in this dataset, but did have a three-item MBI:EE (that included the one MBI:EE item tested), and using it produced very similar results.

Second, because we were comparing single-item measures, we could not test internal consistency reliability, which requires scales to have three or more items to assess. Until test–retest analyses are conducted, the reliability of these items will not be known. However, the primary threat from poor reliability is that it introduces measurement error, biasing our results towards the null (i.e., toward lower correlation between the measures).

Finally, because the surveys had to be anonymous, our sampling frame was estimated, and it is impossible to know for certain who participated in the survey and, consequently, what the true response rate was. We estimated the response rate at 25 % based on administrative data.44 There may have been important differences between those who participated and those who did not, which could have introduced selection bias. There may be concerns that our results do not generalize beyond this sample. However, the demographics of our respondents were very similar to those of primary care participants in the VA All Employee Survey, which was fielded the month prior, that was sent to all VA employees and achieved a 62 % response rate.44 Our sample was also representative of primary care clinics, with respondents from 69 % of the 913 VHA primary care clinics, including sites in all 50 states. This gives us added confidence that our findings are generalizable to the VA primary care population, and possibly to other primary care employees.

Overall, we found that a non-proprietary single-item measure of burnout could serve as a valid substitute for the MBI:EE. Because it is not licensed, and its scores are easy to interpret, this assessment tool has important logistical advantages over the single-item MBI:EE.