DocumentCode :
1450808
Title :
Learning Semantic and Visual Similarity for Endomicroscopy Video Retrieval
Author :
André, Barbara ; Vercauteren, Tom ; Buchner, Anna M. ; Wallace, Michael B. ; Ayache, Nicholas
Author_Institution :
Mauna Kea Technol., Paris, France
Volume :
31
Issue :
6
fYear :
2012
fDate :
6/1/2012 12:00:00 AM
Firstpage :
1276
Lastpage :
1288
Abstract :
Content-based image retrieval (CBIR) is a valuable computer vision technique which is increasingly being applied in the medical community for diagnosis support. However, traditional CBIR systems only deliver visual outputs, i.e., images having a similar appearance to the query, which is not directly interpretable by the physicians. Our objective is to provide a system for endomicroscopy video retrieval which delivers both visual and semantic outputs that are consistent with each other. In a previous study, we developed an adapted bag-of-visual-words method for endomicroscopy retrieval, called “Dense-Sift,” that computes a visual signature for each video. In this paper, we present a novel approach to complement visual similarity learning with semantic knowledge extraction, in the field of in vivo endomicroscopy. We first leverage a semantic ground truth based on eight binary concepts, in order to transform these visual signatures into semantic signatures that reflect how much the presence of each semantic concept is expressed by the visual words describing the videos. Using cross-validation, we demonstrate that, in terms of semantic detection, our intuitive Fisher-based method transforming visual-word histograms into semantic estimations outperforms support vector machine (SVM) methods with statistical significance. In a second step, we propose to improve retrieval relevance by learning an adjusted similarity distance from a perceived similarity ground truth. As a result, our distance learning method allows to statistically improve the correlation with the perceived similarity. We also demonstrate that, in terms of perceived similarity, the recall performance of the semantic signatures is close to that of visual signatures and significantly better than those of several state-of-the-art CBIR methods. The semantic signatures are thus able to communicate high-level medical knowledge while being consistent with the low-level visual signatures and much sh- rter than them. In our resulting retrieval system, we decide to use visual signatures for perceived similarity learning and retrieval, and semantic signatures for the output of an additional information, expressed in the endoscopist own language, which provides a relevant semantic translation of the visual retrieval outputs.
Keywords :
biomedical optical imaging; content-based retrieval; endoscopes; knowledge acquisition; learning (artificial intelligence); medical image processing; semantic networks; video retrieval; Dense-Sift; adapted bag-of-visual-words method; computer vision technique; content-based image retrieval; cross-validation; diagnosis support; distance learning method; endomicroscopy retrieval; endomicroscopy video retrieval; endoscopy; high-level medical knowledge; in vivo endomicroscopy; intuitive Fisher-based method; low-level visual signatures; medical community; semantic detection; semantic ground truth; semantic knowledge extraction; semantic signatures; semantic similarity learning; state-of-the-art CBIR methods; support vector machine methods; visual outputs; visual similarity learning; visual-word histograms; Colonic polyps; Computer aided instruction; Image retrieval; Medical diagnostic imaging; Semantics; Visualization; Bag-of-visual-words (BoW); content-based image retrieval (CBIR); endomicroscopy; semantic and visual similarity; semantic gap; similarity learning; Algorithms; Artificial Intelligence; Capsule Endoscopy; Colonic Neoplasms; Humans; Image Enhancement; Image Interpretation, Computer-Assisted; Information Storage and Retrieval; Microscopy, Video; Pattern Recognition, Automated; Radiology Information Systems; Reproducibility of Results; Semantics; Sensitivity and Specificity; Subtraction Technique;
fLanguage :
English
Journal_Title :
Medical Imaging, IEEE Transactions on
Publisher :
ieee
ISSN :
0278-0062
Type :
jour
DOI :
10.1109/TMI.2012.2188301
Filename :
6153380
Link To Document :
بازگشت