Fusing heterogeneous features for the image-guided diagnosis of intraductal breast lesions

In the analysis of histopathological images, both holistic (e.g., architecture features) and local appearance features demonstrate excellent performance, while their accuracy may vary dramatically among different inputs. This motivates us to investigate how to fuse results from these features to further enhance the accuracy. Particularly, we employ content-based image retrieval approaches to discover morphologically relevant images for image-guided diagnosis, using both holistic and local features. However, because of the dramatically different characteristics and representations of these heterogenous features, their resulting ranks may have no intersection among the top candidates, causing difficulties for traditional fusion methods. In this paper, we employ graph-based query-specific fusion approach where multiple retrieval ranks are integrated and reordered by conducting link analysis on a fused graph. The proposed method is capable of adaptively combining the strengths of local or holistic features for different queries, and does not need any supervision. We evaluate our method on a challenging clinical problem, i.e., histopathological image-guided diagnosis of intraductal breast lesions, and it achieves 91.67% classification accuracy on 120 breast tissue images from 40 patients.