Hierarchical Control Models for Multimodal Process Modeling

The multimodal and hierarchical structure characteristics of a system make process modeling quite difficult. In this paper, we present a hierarchical control model (HCM) for hierarchically multimodal processing. From multiple streams, a control layer extracts the inherent group process that denotes the evolution of the system and controls the evolution of every modality. HCMs model the influences of the group on modalities and represent the hierarchical structure of the system by a multilayer network. To estimate the state order of the model, we also present a new information criterion that corrects the preference of traditional criteria for more complex models and proves the rationality of HCMs. Comparisons with other models on multiagent activity recognition show that HCMs are reliable and efficient.