The reliability of an instrument for identifying and quantifying surgeons’ teaching in the operating room

Background: The operating room is an important teaching venue where surgical residents develop their operative skills and clinical judgement. The purpose of this study was to establish reliability for an observation instrument designed to quantify surgeons’ teaching behaviors in the operating room. Methods: An instrument was developed to identify operating room teaching behaviors in four categories: informing, questioning, responding, and setting tone. Two trained observers coded videotaped operations. Cronbach’s alpha was used to estimate the instrument’s internal consistency, and criterion-related reliability was established through an interobserver agreement level (IOA). Results: Results for each of the teaching behavior categories were as follows: informing (IOA = 86%, ALPHA = +0.978); questioning (IOA = 97%, ALPHA = +0.966); responding (IOA = 93%, ALPHA = +0.97); setting tone (IOA = 97%, ALPHA = +0.882). Conclusions: This instrument is reliable in identifying and quantifying surgeons’ teaching behaviors in the operating room. Identifying teaching behaviors will be valuable to describing and enhancing teaching in the operating room.