Multimodal desktop interaction: The face - object - gesture - voice example

Author

Vidakis, Nikolas ; Vlasopoulos, Anastasios ; Kounalakis, Tsampikos ; Varchalamas, Petros ; Dimitriou, Michalis ; Kalliatakis, Gregory ; Syntychakis, Efthimios ; Christofakis, John ; Triantafyllidis, Georgios

Author_Institution

Appl. Inf. & Multimedia Dept., Technol. Educ. Inst. of Crete, Heraklion, Greece

fYear

2013

fDate

1-3 July 2013

Firstpage

1

Lastpage

8

Abstract

This paper presents a natural user interface system based on multimodal human computer interaction, which operates as an intermediate module between the user and the operating system. The aim of this work is to demonstrate a multimodal system which gives users the ability to interact with desktop applications using face, objects, voice and gestures. These human behaviors constitute the input qualifiers to the system. Microsoft Kinect multi-sensor was utilized as input device in order to succeed the natural user interaction, mainly due to the multimodal capabilities offered by this device. We demonstrate scenarios which contain all the functions and capabilities of our system from the perspective of natural user interaction.

Keywords

face recognition; gesture recognition; human computer interaction; object recognition; sensor fusion; speech recognition; user interface management systems; Microsoft Kinect multisensor; desktop applications interaction; human behaviors; input qualifiers; intermediate module; multimodal human computer interaction; natural user interface system; Databases; Face; Face recognition; Joints; Speech; Speech recognition; MS-Kinect; Multimodal Input; Multimodal Interaction; Natural Interaction;

fLanguage

English

Publisher

ieee

Conference_Titel

Digital Signal Processing (DSP), 2013 18th International Conference on

Conference_Location

Fira

ISSN

1546-1874

Type

conf

DOI

10.1109/ICDSP.2013.6622782

Filename

6622782