MADIS: A Multimedia-Aided Disaster information Integration System for emergency management

With the proliferation of smart devices, disaster responders and community residents are capturing footage, pictures and video of the disaster area with mobile phones and wireless tablets. This multimedia disaster situation information is critical for assisting emergency management (EM) personnel to effectively respond in a timely manner. Currently, however the data is not integrated in incident command systems where situation reports, incidence action plans, etc. are being held. Therefore, we have designed and developed a Multimedia-Aided Disaster information Integration System (MADIS), which utilizes advanced data mining techniques to analyze situation reports and pictures as well as text captured in the field and automatically link the reports directly to relevant multimedia content. Specifically, a dynamic hierarchical image classification approach is proposed to categorize disaster images into different subjects by fusing image and text information. Situation reports are analyzed using advanced document processing techniques and then associated with processed multimedia data. In order to seamlessly incorporate user interactive activities for improving information integration, a user feedback processing scheme is proposed to refine the association between situation reports and images as well as the affinity among images. The system is developed on Apple´s mobile operating system (iOS) and runs on iPad tablets, and its usefulness is evaluated by domain experts from the local EM department.