Using operative outcome to assess technical skill

Background: This study examined whether an operative product and time to completion could serve as measures of technical skill. Methods: Nine final-year (PGY5) and 11 penultimate-year (PGY4) general surgery residents participated in a 6-station bench model examination. Time to completion was recorded. Twelve faculty surgeons (2 per station) evaluated the quality of the final product using a 5-point scale. Reuslts: The mean interrater reliability was 0.59 for product quality. Interstation reliability was 0.59 for analysis of the final product and 0.72 for time to completion. There was 63% and 78% agreement between attendings’ ratings and product quality and time scores respectively. PGY5s’ mean product quality score was 4.14 ± 0.26, compared with 3.82 ± 0.33 for PGY4s (P < 0.05). PGY5s’ mean time was 110 ± 19 minutes compared with PGY4s’ 132 ± 15 (P < 0.05). Conclusions: Analysis of the operative end product and time to completion offer efficient alternatives to on-line examiner scoring for bench model examinations of technical competence