Max-margin discriminative random fields for multimodal human action recognition

Proposed is a max-margin discriminative random fields model for multimodal human action recognition. To incorporate multiple modalities for joint modelling, a specific graphical structure with parallel sequential observations and related hidden-state layers is designed. Moreover, the corresponding potential functions for model formulation are designed. For model learning, the max-margin learning method is proposed to discover both latent correlation among multimodal data and temporal context within individual modality. A comparison experiment shows that the proposed model can boost the performance of human action recognition by taking advantage of complementary characteristics from multiple modalities.