APM perspective

Measuring Resident Hours by Tracking Interactions with the Computerized Record

To explore the use of electronic health record (EHR) data to estimate surgery resident duty hours and monitor real time workload.

Retrospective analysis of resident duty hours logged using a voluntary global positioning system (GPS)-based smartphone application compared to duty hour estimates by an EHR-based algorithm. The algorithm estimated duty hours using EHR activity data and operating room logs. A dashboard was developed through Plan-Do-Study-Act cycles for real-time monitoring of workload.

Single tertiary/quaternary medical center general surgery residency program with approximately 90 categorical, preliminary, and integrated residents at eight clinical sites.

Categorical, preliminary, and integrated surgery residents of all clinical years who volunteered to pilot a GPS application to track duty hours.

Of 2,623 work periods by 59 residents were logged with both methods. EHR-estimated work periods started later than GPS logs (median 0.3 hours, interquartile range [IQR] -0.1 - 0.3); EHR-estimated work periods ended earlier than GPS logs (median 0.1 hours, IQR -0.7 - 0.3); and EHR-estimated duty hour totals were less than totals logged by GPS (median -0.3 hours, IQR -0.8 - +0.1). Overall correlation between weekly duty hours logged by EHR and GPS was 0.79. Correlations between the 2 systems stratified from PGY-1 through PGY-5 were 0.76, 0.64, 0.82, 0.87, and 0.83, respectively. The algorithm identified six 80-hour workweek violations (averaged over 4 weeks), while GPS logs identified 8. EHR-based duty hours and operational data were integrated into a dashboard to enable real time monitoring of resident workloads.

EHR-based estimation of surgical resident duty hours has good correlation with GPS-based assessment of duty hours and identifies most workweek duty hour violations. This approach allows for dynamic workload monitoring and may be combined with operational data to anticipate and prevent duty hour violations, thereby optimizing learning.

The Accreditation Council for Graduate Medical Education specifies strict requirements for clinical work hours during residency training, with serious consequences for violations. Self-reporting of work hours by trainees can be inaccurate due to recall bias, giving program directors limited data to influence change. We aimed to assess the impact of a smart-phone based geofencing application on submission rates for work hours and reported violations in a general surgery residency program at a university-based medical center. We also examined resident perceptions surrounding implementation and use of the application.

We compared clinical work hours submitted and violations reported during the pilot period (October-November 2019) with the months prior to the launch of the application (July-August 2019). PGY1 and PGY2 residents were eligible to use the application during and after this pilot period. Semi-structured interviews were used to assess resident perceptions. A retrospective review was conducted to compare reporting during the same time period from the prior academic year (2018-2019) for historical reference. Paired t-tests were used to analyze the data.

Twenty-six residents (15 PGY1, 11 PGY2) were eligible for the intervention and 23 residents (88%) used the application. The mean number of violations reported decreased significantly during the pilot period compared with the months prior to the intervention (4.5 vs. 11, p = 0.04). The total rate of submissions was not significantly different after the intervention (85% vs. 82%, p = 0.42). The PGY1 mean submission rate decreased during the pilot period (91%-75%, p = 0.21) while the PGY2 submission rate increased (77%-91%, p = 0.07). Compared with historical data, there was an increase in overall total submission rates between academic years 2018/2019 and 2019/2020 (74% vs. 79%, p = 0.047) and an associated decrease in the mean number of monthly violations (14 vs. 6.25, p = 0.004). Thirteen (50%) residents (8 PGY1, 5 PGY2) volunteered for semi-structured interviews. Most participants found the application useful for recording and reporting clinical work hours. They noted an ease in the administrative burden as well as more accurate reporting associated with automated logging. Use of the application was not perceived to limit engagement with patient care; however, there were privacy concerns and some technical barriers were identified. The messaging regarding the application's use was identified as critical for implementation.

The “real-time” data provided by a geofencing application in our program helped to reduce the number of work-hour violations reported and did not diminish resident engagement with patient care. Decreasing the administrative burden of recording work hours coupled with improving transparency and accuracy of submissions may be important mechanisms.

Implementation of resident duty hour policies has resulted in a need to document work hours accurately. We compared the number of self-reported duty hour violations identified through an anonymous, resident-administered survey to that obtained from a standardized, ACGME-sanctioned electronic tracking system.

10 cross-sectional surveys were administered to general surgery residents over five years. A resident representative collected and de-identified the data.

A median of 54 residents (52% male) participated per cohort. 429 responses were received (79% response rate). 111 violations were reported through the survey, while the standardized electronic system identified 76, a trend significantly associated with PGY-level (p < 0.001) and driven by first-year residents (n = 81 versus 37, p = 0.001).

An anonymous, resident-run mechanism identifies significantly more self-reported violations than a standardized electronic tracking system alone. This argues for individual program evaluation of duty hour tracking mechanisms to correct systematic issues that could otherwise lead to repeated violations.