On the impact of wireless multimedia network for multi-modal activity recognition

Due to the rapid development of sensor devices, many types of multimedia sensor data can be collected and analyzed to develop useful multimedia applications such as human activity recognition. Our objective of this research work is to study and analyze the impact of the wireless multimedia network quality on human activity recognition accuracy in an end-to-end networked multimedia system environment - Ocha House. We utilized the new improved Generalized Time Warping (GTW) algorithm and its variants for temporally aligning multi-modal sequences from multiple subjects performing similar human activities. In Ocha House, we captured the human activity data with camera sensors which are transmitted via wireless network for server processing and analysis. We evaluated the human activity recognition accuracy using GTW and its variants by taking into consideration of the wireless network data packet loss phenomenon due to common wireless network protocol behaviors as a result of network flow control and congestion management. We discovered that GTW and its variants are sensitive to wireless network packet loss and observed interesting characteristics resulting in deteriorated human activity recognition accuracy - when wireless network packet loss rate is nominal, the human activity recognition accuracy decreases dramatically.