Modeling video-based anomaly detection using deep architectures: Challenges and possibilities

We are looking to perform anomaly detection in video streams, within the fastest time possible, and without the need to hand-engineer features to suit for particular scenes. In any scene captured by surveillance camera, there could be single or multiple persons (agents) and activities ongoing concurrently, with or without human-object and/or human-human interactions. These characteristics lead to a very interesting problem, which involves techniques and insights from a number of domains-anomaly detection, activity recognition, sequence modeling, and deep learning. First, we need to know how to represent video frames as a set of features, then model the temporal sequence and the spatio-temporal relations in the sequence, followed by training the system using some machine learning algorithm on the training set of sequences. The trained system would be able to tell when there is an anomaly in the input stream. However, this is very challenging due to large variations in environment and human movement, and also due to the vague definition of anomaly in the domain of video surveillance. In this paper, we would like to give informational insights on how techniques from the four domains above can be applied to perform video-based anomaly detection.