Article Text

Download PDFPDF

Statistical tools used for analyses of frequent users of emergency department: a scoping review
  1. Yohann Chiu1,
  2. François Racine-Hemmings1,
  3. Isabelle Dufour1,
  4. Alain Vanasse1,
  5. Maud-Christine Chouinard2,
  6. Mathieu Bisson1,
  7. Catherine Hudon1
  1. 1 Department of Family Medicine and Emergency Medicine, Université de Sherbrooke, Sherbrooke, Quebec, Canada
  2. 2 Department of Health Sciences, Université du Québec à Chicoutimi, Chicoutimi, Quebec, Canada
  1. Correspondence to Yohann Chiu; yohann.chiu{at}usherbrooke.ca

Abstract

Objective Frequent users represent a small proportion of emergency department users, but they account for a disproportionately large number of visits. Their use of emergency departments is often considered suboptimal. It would be more efficient to identify and treat those patients earlier in their health problem trajectory. It is therefore essential to describe their characteristics and to predict their emergency department use. In order to do so, adequate statistical tools are needed. The objective of this study was to determine the statistical tools used in identifying variables associated with frequent use or predicting the risk of becoming a frequent user.

Methods We performed a scoping review following an established 5-stage methodological framework. We searched PubMed, Scopus and CINAHL databases in February 2019 using search strategies defined with the help of an information specialist. Out of 4534 potential abstracts, we selected 114 articles based on defined criteria and presented in a content analysis.

Results We identified four classes of statistical tools. Regression models were found to be the most common practice, followed by hypothesis testing. The logistic regression was found to be the most used statistical tool, followed by χ2 test and t-test of associations between variables. Other tools were marginally used.

Conclusions This scoping review lists common statistical tools used for analysing frequent users in emergency departments. It highlights the fact that some are well established while others are much less so. More research is needed to apply appropriate techniques to health data or to diversify statistical point of views.

  • Frequent users
  • Statistical methods

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • First overview of statistical tools used in frequent users analysis.

  • Follows a well-defined methodological framework in an extensive body of literature.

  • Quality assessment is not performed in a scoping review.

  • Studies in other languages than English or French might have been missed.

Background

Emergency department (ED) ‘frequent users’ are a sub-group of ED users that make repeated, multiple visits during a given amount of time. Though there is no consensus about definition for frequent users, thresholds in the literature range from two to more than 10 ED visits per year,1 2 while the most common one is more than four ED visits per year.1 2 Frequent users represent a small proportion of ED users but account for a large number of visits.3–5 They often display complex characteristics such as low socioeconomic status combined with physical and mental health issues.6 As such, their ED use is considered suboptimal,7 as the best strategy would be to identify those patients at an earlier stage in their health problem trajectory, in order to treat them more efficiently.8 Furthermore, frequent users’ visits may lead to overcrowding in EDs and decreased quality of care.2 Identifying factors that best describe those users and predict their ED use is therefore an essential task to improve ED care as well as frequent users’ health problems. Adequate statistical tools are needed to that end. Although they are numerous, no literature review has been published yet about statistical tools used for analysing ED frequent users. Therefore, the aim of our study was to draw up a list of statistical tools used in identifying variables associated with frequent use or predicting the risk of becoming a frequent user.

Methods

In order to list the statistical tools used in describing variables associated with and prediction of frequent ED use, we conducted a scoping review. We followed the 5-stage methodology of Arksey and O’Malley9 adapted by Levac et al.10 The methodological framework of a scoping review allows ‘mapping rapidly the key concepts underpinning a research area and the main sources and types of evidence available’,11 thus allowing us to identify gaps in the literature and future research opportunities.

Stage 1: Identifying the research question

We defined our research question as follows: What statistical tools are used in the identification of variables associated with frequent ED users and in their prediction?

Stage 2: Identifying relevant studies

We searched PubMed, CINAHL and Scopus databases in February 2019, using search strategies developed with the help of an information specialist (see the online supplementary appendix for the complete search strategy). Keywords included variants of ‘frequent users’, ‘emergency departments’ and ‘statistical tools’.

Supplemental material

There were no restriction regarding the population age or sex, health conditions, study period or country.

Stage 3: Study selection

Articles written in French or in English were included using the following criteria:

  • The study must focus on frequent users of EDs (studies focusing on re-visits or on frequent visits other than in EDs were excluded).

  • The study must have an explicit definition of frequent users, such as four visits in 1 year (reviews were excluded).

  • The study must use at least one statistical tool that is classified as inferential (not descriptive, as defined by The Cambridge Dictionary of Statistics12), such as hypothesis tests, regression models, decision trees or others.

  • The study’s objectives must include identifying variables associated with frequent use or predicting the risk of becoming a frequent user.

We collected 4534 potential abstracts (figure 1). Of those, 32 were duplicates and 4344 were excluded by an investigator (YC) after reading the title and the abstract. At this stage, studies were discarded if it was explicit from the title and the abstract that they were unfit for the scoping review (for instance studies about frequent use of inpatient services, systematic reviews, etc). In case of uncertainty, studies were kept for complete reading. Then, YC and FRH or ID independently evaluated the remaining 158 full text articles, of which 109 matched the above criteria. A third evaluator was consulted in case of discrepancy. Reasons for exclusion were: not in French or English (1), duplicate (3), systematic review (4), no inferential statistics (5), no explicit definition of frequent users (5), focus not on ED (14), no description or prediction of frequent users (17). A reference search among the references of the 109 included articles yielded five relevant articles. Thus, 114 articles were included in this study, of which YC, ID and MB examined the full texts.

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram. ED, emergency department.

Stage 4: Charting the data

YC, MB and ID independently extracted the corresponding data. Reported characteristics were the first (two) author(s), the publication year, the study location, the population, the frequent users’ definition, the objectives, the sample size and the statistical tools used concerning the research question.

Stage 5: Collating, summarising and reporting the results

The results are reported via a content analysis.13

Patient and public involvement

Patients or public were not involved in this study.

Results

The studies’ main characteristics are presented in table 1. Out of 114 studies, 65 were conducted in the USA, 17 in Canada and 8 in Australia (figure 2). The various statistical tools were classified into four main categories: regression, hypothesis testing, machine learning and other tools.

Table 1

Main characteristics of the 86 included studies

Figure 2

Number of studies by country.

Regression

Regression tools consist of a set of processes aimed at quantifying the relationships between a dependent variable and other explanatory variables.14 They are useful for description and prediction. Some regression models may be regularised, which in this case means avoiding overfitting with too many explanatory variables, or zero-truncated, which means that the model is not allowed to take null values.

Out of the four categories (regression, hypothesis testing, machine learning and other tools), the most reported tool was the logistic regression (90 studies,3–5 15–101 two of which are regularised by LASSO or elastic net techniques), followed by the binomial regression (13 studies,18 46 55 73 76 77 82 89 102–106 2 of which are zero-truncated). To a lesser extent, the Poisson regression (seven studies,77 107–112 one of which is zero-truncated), the linear regression (six studies74 76 102 113–115), the analysis of variance (six studies44 59 73 96 103 104), the Cox regression (four studies87 93 105 116) and hierarchical models (one study90) were also used. In those studies, the results are often associated with ORs. The mixed-effects models were mentioned three times.39 91 117 Regression parameters were estimated by generalised estimating equations in four studies18 103 106 118 while parameter confidence intervals were estimated by the bootstrap procedure (two studies25 67) and the Clopper-Pearson method (one study25). The receiver operating characteristic curve, or equivalently the sensitivity, specificity or area under the curve (‘c-statistic’), was computed in 10 studies.4 36 48 64 75 83 88 107 117 119 Finally, two studies performed imputation to account for missing data (Markov chain Monte Carlo and multiple imputations78 90).

Hypothesis testing

Statistical tests aim at testing a specific hypothesis about data and rely on probability distributions.120 In the selected studies, the tests aimed mainly at comparing two samples (frequent users and non-frequent users).

The most common statistical tests were the χ2 test (53 studies17 28 31 34 36–38 40–42 47 49 52 54 56 58 60 62–69 72–74 76 77 79–82 85 88 89 94 96 97 101–104 109 110 112 115 119 121–124) and the t-test (24 studies40 45 47 49 62 63 65 66 74 77 79 81 85 89 94 95 97 98 109 114 115 122 124 125) which measured association between variables or goodness-of-fit. As an alternative to the χ2 test for association, five studies used the Fisher exact test.63 72 94 98 119 Sample mean differences were assessed by 23 studies with the Mann-Whitney U test (also called the Wilcoxon rank-sum test20 23 31 47 58 66 77 98 110 119 121 123–125), its variant for dependent samples the Wilcoxon signed rank test,40 101 or the Kruskal-Wallis test.23 37 42 68 73 76 102 The difference in proportions test,126 Mantel-Haenszel test (test for differences in contingency tables, two studies44 117), the likelihood ratio test (significance test for nested models, two studies64 67), the Hosmer-Lemeshow test (goodness-of-fit for logistic regression, two studies67 70), the Wald test (significance test for regression coefficients, two studies30 96) and the Breslow-Day test (test for homogeneity in contingency tables OR53) were also used to a lesser degree. Finally, one study checked the assumption of normality with the Kolmogorov-Smirnov test.66

Machine learning

Machine learning tools are a set of algorithms that can learn and adapt to data in order to classify or predict, for instance.127 In the selected studies, the machine learning tools aimed mainly at classifying users (frequent vs non-frequent).

Two studies used random forests31 36 along with support vector machines. Decision trees, which include classification and regression trees, were implemented by five studies.5 31 36 61 113 Adaptive boosting, or AdaBoost, is a meta-algorithm that combines with other algorithms and helps for better performances. It was computed in two studies.36 61

Other tools

Two studies used survival analysis,50 116 while another one fitted a non-parametric distribution to their data.25 Finally, maximum likelihood monotone coarse classifier algorithm was used as a binning method91 and non-negative matrix factorisation as a clustering technique.115

Discussion

The most exploited statistical tools arguably came from regression analysis. This may be because regression is well established in medical statistics or also because it is the most natural tool when trying to find significant variables to explain a dependent variable (in this case, to be a frequent user). Moreover, it allows predicting easily the risk of a new user becoming a frequent user, depending on its covariates. Other tools from hypothesis testing or machine learning also proved to be popular, although to a much lesser extent. Combining these statistical techniques may help in discovering significant and complementary patterns, compared with using tools from one class only. In our scoping review, two studies mixed statistical tools from regression, hypothesis testing and machine learning.31 36 In those studies, the author evaluated various performance criteria. While logistic regression performed well, other techniques such as random forests or LASSO regression were also competitive. Besides the fact that logistic regression can display modest performances,128 random forests and LASSO regression can complete logistic regression. The first technique can be used to assess the importance of each independent variable in the model, while the second technique can be useful for automatic selection of features. Likewise, using a variety of statistical tools can help complete or confirm results obtained with established methodologies. Different tools from one class can also be mixed in order to achieve different stages of the analysis (for instance, different types of regression82).

The analysis of frequent ED users could benefit from using more machine learning techniques. Those were found to be not as common as regression or hypothesis testing, although they are especially appropriate when dealing with classification, prediction or big data. Tools such as support vector machines (which were used by two studies in this scoping review31 36), artificial neural networks or Bayesian networks are common classifiers and predictors in the artificial intelligence community.129 They are popular for instance in cancer diagnostic and prognosis, which strongly rely on classification and prediction.130–132 In particular, support vector machines, decision trees or self-organising maps can deal with binary outcomes, which is usually the case for frequent use outcomes. They usually require large datasets in order to overcome overfitting, but this is becoming less and less of an issue in health sciences.133 Nevertheless, machine learning tools often use a black box approach as there are many intermediary steps leading to the final solution. While each step usually consists of simple arithmetic operations, their multiple interactions can be more difficult to interpret. In spite of this opacity, they still display good performances in classifying and predicting. In some cases, they may be more accurate than the widely used logistic regression.134 Those methods would thus turn out to be less useful in data exploration.135 Machine learning tools are getting popular in other fields in health sciences, such as critical care,136 cardiology137 or emergency medicine.138 The authors state that their fields would benefit from this growing popularity, though results need to be analysed and interpreted in collaboration with clinicians.

Other tools exist that may also be suitable for describing the associated variables or the prediction of frequent ED users but were not reported in the literature. Among those, principal component analysis (PCA) is a dimensional reduction and visualisation technique, sometimes used with cluster or discriminant analysis.139 Based on all the original explanatory variables, PCA constructs new ones by summing and weighing them differently. More weight is given to relevant variables so that those latter become dominant in the new constructions while still including all variables. For instance, Burgel et al 140 built chronic obstructive pulmonary disease clinical phenotypes by constructing new relevant variables with PCA and by grouping similar subjects in this new space with cluster analysis.140 Moreover, PCA has already been used for the construction of questionnaires and diagnosis tools in a medical context,141 142 both of which can prove useful in the identification of frequent users.

As mentioned, regression techniques were common in the selected studies. Yet, quantile regression (QR)143 was not mentioned. QR is a generalisation of mean regression in the sense that its focus is not only the mean of the dependent variable distribution (such as in classical linear regression) but any quantile of it. QR thus represents an alternative to define frequent users by the high quantiles of ED visit distribution (eg, the 90th quantile). Eight studies25 27 46 48 51 54 62 121 defined frequent users with quantiles, but they did not use QR. QR would allow for finer investigations in the different quantiles of ED users in relationship to the explanatory variables. For instance, the association between age and the number of ED visits may be significantly different across the 10th (low users) and 90th (frequent users) quantiles. Such a heterogeneous association would be uncovered by QR, while usually unseen with a classical mean regression. Ding et al 144 used QR to characterise waiting room and treatment times in EDs.144 They explored the lowest, median and highest of those times and highlighted predictors that were significant only in particular quantiles. Usually, QR requires a continuous dependent variable as opposed to a logistic regression, though it is possible to combine these two regressions.145 Furthermore, defining frequent users by quantiles would allow for better comparison between studies as there is no common definition for frequent users.

Strengths and limitations

To the best of our knowledge, this scoping review is the first to list statistical tools that are used in the identification of variables associated with frequent ED use and the prediction of frequent users. Besides, it was conducted following a well-defined methodological framework. The search strategies were designed with an information specialist in three different databases. Two independent evaluators selected the articles and extracted the data while a third independent evaluator settled disagreements, ensuring that all included studies were relevant. One limitation of our study is that quality assessment is not performed in a scoping review. However, this should not alter the results, since the aim was to list which statistical tools have been applied in the literature. Moreover, the majority of articles were in English which may introduce a selection bias (for instance, one excluded article was in Spanish). More than half of the reviewed studies were indeed conducted in the USA, making the results difficult to compare to other countries.

Conclusions

Frequent ED users represent a complex issue, and their analysis require adequate statistical tools. In this context, this scoping review shows that some tools are well established, such as logistic regression and χ2 test, while others such as support vector machines are less so, though they would deserve to get more attention. It also outlines some research opportunities with other tools not yet explored.

Acknowledgments

We would like to thank information specialist Josée Toulouse for her help in defining the search strategies and Tina Wey (PhD) for revising the text.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61.
  62. 62.
  63. 63.
  64. 64.
  65. 65.
  66. 66.
  67. 67.
  68. 68.
  69. 69.
  70. 70.
  71. 71.
  72. 72.
  73. 73.
  74. 74.
  75. 75.
  76. 76.
  77. 77.
  78. 78.
  79. 79.
  80. 80.
  81. 81.
  82. 82.
  83. 83.
  84. 84.
  85. 85.
  86. 86.
  87. 87.
  88. 88.
  89. 89.
  90. 90.
  91. 91.
  92. 92.
  93. 93.
  94. 94.
  95. 95.
  96. 96.
  97. 97.
  98. 98.
  99. 99.
  100. 100.
  101. 101.
  102. 102.
  103. 103.
  104. 104.
  105. 105.
  106. 106.
  107. 107.
  108. 108.
  109. 109.
  110. 110.
  111. 111.
  112. 112.
  113. 113.
  114. 114.
  115. 115.
  116. 116.
  117. 117.
  118. 118.
  119. 119.
  120. 120.
  121. 121.
  122. 122.
  123. 123.
  124. 124.
  125. 125.
  126. 126.
  127. 127.
  128. 128.
  129. 129.
  130. 130.
  131. 131.
  132. 132.
  133. 133.
  134. 134.
  135. 135.
  136. 136.
  137. 137.
  138. 138.
  139. 139.
  140. 140.
  141. 141.
  142. 142.
  143. 143.
  144. 144.
  145. 145.

Footnotes

  • Contributors YC and CH designed the study with FR-H, ID and AV. YC, ID, CH and MB collected and analysed the data. YC and CH wrote the first draft of the manuscript. FR-H, ID, AV, M-CC and MB contributed to the writing of the manuscript. All authors read and approved the final manuscript.

  • Funding This work was financed by grants from the Fonds de recherche du Québec – Santé and the Centre de recherche du Centre hospitalier universitaire de Sherbrooke. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement There are no unpublished additional data from the study.

  • Patient consent for publication Not required.