Title :
New applications of multimodal human-computer interfaces
Author :
Czyzewski, Andrzej
Author_Institution :
Multimedia Syst. Dept., Gdansk Univ. of Technol., Gdansk, Poland
Abstract :
Multimodal computer interfaces and examples of their applications to education software and for the disabled people are presented. The proposed interfaces include the interactive electronic whiteboard based on video image analysis, application for controlling computers with gestures and the audio interface for speech stretching for hearing impaired and stuttering people. Application of the eye-gaze tracking system to awareness evaluation is demonstrated. The proposed method assumes analysis of visual activity of patients remaining in vegetative state. The scent emitting multimo-dal computer interface is an important supplement of the polysen-soric stimulation process, playing an essential role in education and therapy of children with developmental disorders. A new approach to diagnosing Parkinson´s disease is shown. The progression of the disease can be measured by the UPDRS (Unified Parkinson Disease Rating Scale) scale which is used for evaluating motor and behavioral symptoms of Parkinson´s disease, employing the multimo-dal interface called Virtual-Touchpad (VTP) to support medical diagnosis. The paper is concluded with some general remarks concerning the role of multimodal computer interfaces applied to learning, therapy and everyday usage of computerized devices.
Keywords :
computer aided instruction; diseases; gaze tracking; gesture recognition; handicapped aids; human computer interaction; interactive systems; medical diagnostic computing; medical disorders; patient diagnosis; speech-based user interfaces; touch sensitive screens; video signal processing; Parkinson´s disease diagnosis; UPDRS scale; VTP; audio interface; awareness evaluation; behavioral symptoms; computerized devices; developmental disorders; disabled people; education software; eye-gaze tracking system; gestures; hearing impaired people; interactive electronic whiteboard; medical diagnosis; motor symptoms; polysensoric stimulation process; scent emitting multimodal human-computer interfaces; speech stretching; stuttering people; unified Parkinson disease rating scale; vegetative state; video image analysis; virtual-touchpad; visual activity analysis; Algorithm design and analysis; Auditory system; Cameras; Signal processing algorithms; Speech; Streaming media; Training; multimodal interfaces; speech processing; video processing;
Conference_Titel :
New Trends in Audio & Video and Signal Processing: Algorithms, Architectures, Arrangements, and Applications (NTAV/SPA), 2012 Joint Conference
Conference_Location :
Lodz
Print_ISBN :
978-8-3728-3502-4