Title :
Learning semantic relationships for better action retrieval in images
Author :
Vignesh Ramanathan;Congcong Li;Jia Deng;Wei Han;Zhen Li;Kunlong Gu;Yang Song;Samy Bengio;Chuck Rossenberg;Li Fei-Fei
Author_Institution :
Stanford University, 450 Serra Mall, California 94305, United States
fDate :
6/1/2015 12:00:00 AM
Abstract :
Human actions capture a wide variety of interactions between people and objects. As a result, the set of possible actions is extremely large and it is difficult to obtain sufficient training examples for all actions. However, we could compensate for this sparsity in supervision by leveraging the rich semantic relationship between different actions. A single action is often composed of other smaller actions and is exclusive of certain others. We need a method which can reason about such relationships and extrapolate unobserved actions from known actions. Hence, we propose a novel neural network framework which jointly extracts the relationship between actions and uses them for training better action retrieval models. Our model incorporates linguistic, visual and logical consistency based cues to effectively identify these relationships. We train and test our model on a largescale image dataset of human actions. We show a significant improvement in mean AP compared to different baseline methods including the HEX-graph approach from Deng et al. [8].
Keywords :
"Visualization","Semantics","Predictive models","Training","Neural networks","Animals","Marine vehicles"
Conference_Titel :
Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on
Electronic_ISBN :
1063-6919
DOI :
10.1109/CVPR.2015.7298713