Measuring Intra- and Inter-Observer Agreement in Identifying and Localizing Structures in Medical Images

Inter- and intra-observer variability exists in any measurements made on medical images. There are two sources of variability. The first occurs when the observers identify and localize the object of interest, and the second happens when the observers make appropriate measurement on the object of interest. A number of statistical methods are available to quantify the degree of agreement between measurements made by different observers. However, little has been done to develop metrics for quantifying the variability in identifying and localizing the objects of interest prior to measurement. In this paper, we propose to use the complex wavelet structural similarity index (CW-SSIM) method to measure the variability in identifying and localizing structures on images. Performance comparisons using simulated images as well as real mammography images demonstrate the effectiveness and robustness of the CW-SSIM method.