Robust Object Recognition via Visual Pathway Feedback

Object recognition, which consists of classification and detection, has two important attributes for robustness: (1) Closeness: detection windows should be close to object locations, and (2) Adaptiveness: object matching should be adaptive to object variations in classification. It is difficult to satisfy both attributes by considering classification and detection separately, thus recent studies combine them based on confidence contextualization and foreground modeling. However, these combinations neglect feature saliency and object structure, which are important for recognition. In fact, object recognition originates in the mechanism of "what" and "where" pathways in human visual systems, and more importantly, these pathways have feedback to each other, which provides a probable way to improve closeness and adaptiveness. Inspired by the feedback, we propose a robust object recognition framework by designing a computational model of the feedback mechanism. In the "what" feedback, the feature saliency from classification is exploited to rectify detection windows for better closeness, while in the "where" feedback, object parts from detection are used to model object matching of object structure for better adaptiveness. Experiments show that the "what" and "where" feedback can be effective to improve closeness and adaptiveness for robust object recognition, and encouraging results are obtained on the challenging PASCAL VOC 2007 dataset.