Feasibility, reliability and validity of a new assessment form for use with basic surgical trainees

Background: To establish feasibility, reliability and validity of a newly designed basic surgical training (BST) assessment form. Methods: The assessment form was evaluated among trainees on the South-east Scotland BST rotation over 18 months. Feasibility was indicated by response rate. Reliability was determined using test-retest analysis and internal consistency of each domain determined by Cronbach’s alpha and item-total correlation. Construct validity was evaluated by determining improvement in performance after 1 year of training. Results: Response rate was high (889 of 984 forms [90%]), with similar representation from all disciplines. A highly significant positive correlation was found between test-retest scores for all domains (rho: I, 0.736; II, 0.875; III, 0.671; IV, 0.826; V, 0.859; P = 0.0001 all domains, Spearman’s). Internal consistency was excellent for each domain, with Cronbach’s alpha ranging from 0.82 to 0.95. Item-total correlation coefficients were greater than the required 0.4 for all but one task. In 101 assessments carried out after 1 year of training, significant improvement in scores was found for all domains except clinical skills (time 0 to time 1 year median [interquartile range]: I, 81 [26 to 100] to 100 [100 to 100], P = 0.008; II, 17 [−19 to 52] to 72 [36 to 100], P = 0.015; III, 85 [51 to 100] to 100 [87 to 100], P = 0.018; IV, 82 [44 to 100] to 92 [69 to 100], P = 0.211; V, 27 [−33 to 64] to 76 [32 to 100], P = 0.004). Conclusions: The new assessment form is feasible, reliable and valid and would therefore be suitable for extended use with basic surgical trainees.