DocumentCode :
3263637
Title :
Predicting listener back-channels for human-agent interaction using neuro-dynamical model
Author :
Sano, Shotaro ; Nishide, Shun ; Okuno, Hiroshi G. ; Ogata, Tetsuya
Author_Institution :
Dept. of Intell. Sci. & Technol., Kyoto Univ., Kyoto, Japan
fYear :
2011
fDate :
20-22 Dec. 2011
Firstpage :
18
Lastpage :
23
Abstract :
The goal of our work is to create natural verbal interaction between humans and speech dialogue agents. In this paper, we focus on generations of back-channel for speech dialogue agents the same way humans do. To create such a system, the system needs to predict the appropriate timing of back-channel on the basis of the human´s speech. For the prediction model, we use a neuro-dynamical system called a multiple timescale recurrent neural network (MTRNN). The model is trained using an actual corpus of a poster session of the IMADE project using the presenter´s prosodic and visual information as features. Using the model, we conducted back-channel timing prediction experiments. The results showed that our system could predict back-channel timing about 0.5 seconds before generation of back-channel response. Comparing the results with the actual back-channel timing in the corpus, the system showed 37.1% of recall, 31.7% of precision, and 34.2% of F-measure. These results show the model to effectively predict and generate back-channel responses.
Keywords :
human computer interaction; interactive systems; recurrent neural nets; speech processing; IMADE project; MTRNN model; back-channel response; back-channel timing prediction; human speech; human-agent interaction; listener back-channels; multiple timescale recurrent neural network; natural verbal interaction; neuro-dynamical model; neuro-dynamical system; prediction model; speech dialogue agent; visual information; Humans; Predictive models; Silicon; Speech; Timing; Training; Visualization;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
System Integration (SII), 2011 IEEE/SICE International Symposium on
Conference_Location :
Kyoto
Print_ISBN :
978-1-4577-1523-5
Type :
conf
DOI :
10.1109/SII.2011.6147412
Filename :
6147412
Link To Document :
بازگشت