A System for Multicamera Task Recognition and Summarization for Structured Environments

In this paper, we propose a novel system for visual recognition and summarization of pick and place tasks that may be executed in settings such as an industrial assembly line. Our novel approach is based on the utilization of hidden Markov models for online task recognition as well as on the use of prior knowledge via a Hopfield-based optimization scheme. To facilitate offline analysis, we extract summaries of the captured content based on these features. We extract the motion energy using the norms of the Zernike moments, looking for local minima and maxima that indicate distinctive visual events and thus key-frames. The proposed scheme is not threshold-dependent, and, therefore, the number of extracted key-frames varies according to the complexity of motion energy variation. We validate our system by experimenting on two datasets.