Testing technical skill via an innovative “bench station” examination*
Background
A new approach to testing operative technical skills, the Objective Structured Assessment of Technical Skill (OSATS), formally assesses discrete segments of surgical tasks using bench model simulations. This study examines the interstation reliability and construct validity of a large-scale administration of the OSATS.
Methods
A 2-hour, eight-station OSATS was administered to 48 general surgery residents. Residents were assessed at each station by one of 48 surgeons who evaluated the resident using two methods of scoring: task-specific checklists and global rating scales.
Results
lnterstation reliability was 0.78 for the checklist score, and 0.85 for the global score. Analysis of variance revealed a significant effect of training for both the checklist score, F(3,44) = 20.08, P < 0.001, and the global score, F(3,44) = 24.63, P < 0.001.
Conclusions
The OSATS demonstrates high reliability and construct validity, suggesting that we can effectively measure residents' technical ability outside the operating room using bench model simulations.
References (8)
- ReznickRK
Teaching and testing technical skills
Am J Surg
(1993) - van der VleutenCPM et al.
Assessment of clinical skills with standardized patients: state of the art
Teach Learn Med
(1990) - MartinJA et al.
An objective structured assessment of technical skill for surgical residents
- KoptaJA
An approach to the evaluation of operative skills
Surgery
(1971)
Cited by (855)
Automated measurement extraction for assessing simple suture quality in medical education[Formula presented]
2024, Expert Systems with ApplicationsPracticing open-wound suturing requires feedback, but evaluation of each suture practice takes instructors’ time and is often subjective. Recent work showed that deep-learning models can automate the evaluation by analyzing an image or a video of suturing into either pass or fail. However, they lacked fine-grained feedback offered by previous systems that required specialized devices or manual annotations. This work introduced a system that automatically extract geometric measurements from a suture practice end-product image for further evaluation. We proposed the suture instance segmentation task and a hand-crafted algorithm to extract interpretable metrics from an image. We collected two new simple suture datasets consisting of 240 images with instance segmentation and physical measurement annotations. The experiment results shows that current deep-learning model can accurately segment suture instances and the extracted measurements from our system are highly correlated with physical measurements from humans.
Development of a Sensor Technology to Objectively Measure Dexterity for Cardiac Surgical Proficiency
2024, Annals of Thoracic SurgeryTechnical skill is essential for good outcomes in cardiac surgery. However, no objective methods exist to measure dexterity while performing surgery. The purpose of this study was to validate sensor-based hand motion analysis (HMA) of technical dexterity while performing a graft anastomosis within a validated simulator.
Surgeons at various training levels performed an anastomosis while wearing flexible sensors (BioStamp nPoint, MC10 Inc) with integrated accelerometers and gyroscopes on each hand to quantify HMA kinematics. Groups were stratified as experts (n = 8) or novices (n = 18). The quality of the completed anastomosis was scored using the 10 Point Microsurgical Anastomosis Rating Scale (MARS10). HMA parameters were compared between groups and correlated with quality. Logistic regression was used to develop a predictive model from HMA parameters to distinguish experts from novices.
Experts were faster (11 ± 6 minutes vs 21 ± 9 minutes; P = .012) and used fewer movements in both dominant (340 ± 166 moves vs 699 ± 284 moves; P = .003) and nondominant (359 ± 188 moves vs 567 ± 201 moves; P = .02) hands compared with novices. Experts’ anastomoses were of higher quality compared with novices (9.0 ± 1.2 MARS10 vs 4.9 ± 3.2 MARS10; P = .002). Higher anastomosis quality correlated with 9 of 10 HMA parameters, including fewer and shorter movements of both hands (dominant, r = −0.65, r = −0.46; nondominant, r = −0.58, r = −0.39, respectively).
Sensor-based HMA can distinguish technical dexterity differences between experts and novices, and correlates with quality. Objective quantification of hand dexterity may be a valuable adjunct to training and education in cardiac surgery training programs.
Evaluation of surgical skill among otology residents during cadaveric temporal bone dissection using an objective assessment tool
2024, Medical Journal Armed Forces IndiaThe acquisition and refinement of technical skills by the surgical residents are central to surgical teaching; hence, there is increasing interest in the objective assessment of surgical competence. In the field of otology, as of now, there are limited studies for the assessment of surgical competence, also, various subjective methods are being used to assess this vital aspect of training. This study aimed to validate and use an objective assessment tool for the valuation of surgical skills in a tertiary care teaching institute in the Indian subcontinent.
Surgical competence of the residents in temporal bone dissection was assessed on a Likert scale by using a grading scale developed by Wan et al., after obtaining necessary permissions.
Junior residents in ENT who had completed one year of residency in otolaryngology were asked to perform temporal bone surgery (cortical mastoidectomy) and were marked according to the proforma by two senior experienced otologists. The Cronbach's alpha value was .86 which is an indicator of good technical validity.
In our study, we have validated a grading scoring scale used by Wan et al. to objectively assess the ability of an otology resident to perform temporal bone surgery. It is recommended for use in Indian scenario due to its good technical validity. The use of a Likert scale to individually rate each competency makes the evaluation precise and simple.
Use of Simulation For Training Advanced Colorectal Procedures
2024, Journal of Surgical EducationSimulation training for minimally invasive colorectal procedures is in developing stages. This study aims to assess the impact of simulation on procedural knowledge and simulated performance in laparoscopic low anterior resection (LLAR) and robotic right colectomy (RRC).
LLAR and RRC simulation procedures were designed using human cadaveric models. Resident case experience and simulation selfassessments scores for operative ability and knowledge were collected before and after the simulation. Colorectal faculty assessed resident simulation performance using validated assessment scales (OSATS-GRS, GEARS). Paired t-tests, unpaired t-tests, Pearson's correlation, and descriptive statistics were applied in analyses.
Barnes-Jewish Hospital/Washington University School of Medicine in St. Louis, Missouri.
Senior general surgery residents at large academic surgery program.
Fifteen PGY4/PGY5 general surgery residents participated in each simulation. Mean LLAR knowledge score increased overall from 10.0 ± 2.0 to 11.5 ± 1.6 of 15 points (p = 0.0018); when stratified, this increase remained significant for the PGY4 cohort only. Mean confidence in ability to complete LLAR increased overall from 2.0 ± 0.8 to 2.8 ± 0.9 on a 5-point rating scale (p = 0.0013); when stratified, this increase remained significant for the PGY4 cohort only. Mean total OSATS GRS score was 28 ± 6.3 of 35 and had strong positive correlation with previous laparoscopic colorectal experience (r = 0.64, p = 0.0092). Mean RRC knowledge score increased from 9.4 ± 2.2 to 11.1 ± 1.5 of 15 points (p = 0.0030); when stratified, this increase again remained significant for the PGY4 cohort only. Mean confidence in ability to complete RRC increased from 1.9 ± 0.9 to 3.2 ± 1.1 (p = 0.0002) and was significant for both cohorts.
Surgical trainees require opportunities to practice advanced minimally invasive colorectal procedures. Our simulation approach promotes increased procedural knowledge and resident confidence and offers a safe complement to live operative experience for trainee development. In the future, simulations will target trainees on the earlier part of the learning curve and be paired with live operative assessments to characterize longitudinal skill progression.
Hand Sewn Anastomosis Skill Acquisition and In Vivo Transfer Using 3D-Printed Small Bowel Simulator
2023, Journal of Surgical ResearchGeneral surgery residents need to master the hand-sewn bowel anastomosis (HSBA) technique. However, practice opportunities outside of the operating room are rare, and commercial simulators are often costly. The objective of this study is to assess the efficacy of a new, affordable silicone small bowel simulator, made with a three-dimensional (3D) printed mold, as a training tool to learn this technique.
This was a single-blinded pilot randomized controlled trial comparing two groups of eight junior surgical residents. All participants completed a pretest using an inexpensive, custom developed 3D-printed simulator. Next, participants randomized to the experimental group practiced the HSBA skill at home (eight sessions), while those randomized to the control group did not receive any hands-on practice opportunities. A posttest was done using the same simulator as for the pretest and practice sessions, and the retention-transfer test was performed on an anesthetized porcine model. Pretests, posttests and retention-transfer tests were filmed and graded by a blinded evaluator using assessments of technical skills, quality of final product, and tests of procedural knowledge.
The experimental group significantly improved after practicing with the model (P = 0.01), while an equivalent improvement was not noted in the control group (P = 0.07). Moreover, the experimental group's performance remained stable between the posttest and the retention-transfer test (P = 0.95).
Our 3D-printed simulator is an affordable and efficacious tool to teach residents the HSBA technique. It allows development of surgical skills that are transferable to an in vivo model.
Bibliometric analysis of the classic cited papers in the American Journal of Surgery: Citation recapitulates surgical history
2023, American Journal of SurgeryWe performed a bibliometric analysis of the American Journal of Surgery (AJS) to identify, characterize and place within a historical context, its published classic cited papers (CCPs).
Bibliometric data from papers published in the AJS between January 1, 1945, and December 31, 2021 was extracted from the Web of Science database. Analysis was performed utilizing Bibliometrix and VOSViewer software.
27,070 papers were published in the AJS over the study period. There were 16 CCPs, including 5 Top CCPs, identified. Review of the Top CCPs reveals that they are based on careful clinical observations, innovation and generally build on prior published work. Top CCPs usually are specific to a particular diagnosis or a commonly performed procedure, as such papers frequently present a scoring or classification system, or important details related to new operative approaches or techniques.
Bibliometric study of the AJS has allowed for identification, characterization and appreciation of many of the key changes that have occurred in the discipline throughout the history of modern surgery.
- *
This work was supported by the physicians of Ontario through a grant given by the P.S.I. foundation.