DocumentCode :
1668668
Title :
Extracting spoken and acoustic concepts for multimedia event detection
Author :
van Hout, Julien ; Akbacak, Murat ; Castan, Diego ; Yeh, Edmund ; Sanchez, Miriam
Author_Institution :
Speech Technol. & Res. Lab., SRI Int., Menlo Park, CA, USA
fYear :
2013
Firstpage :
3657
Lastpage :
3661
Abstract :
Because of the popularity of online videos, there has been much interest in recent years in audio processing for the improvement of online video search. In this paper, we explore using acoustic concepts and spoken concepts extracted via audio segmentation/recognition and speech recognition respectively for Multimedia Event Detection (MED). To extract spoken concepts, a segmenter trained on annotated data from user videos segments the audio into three classes: speech, music, and other sounds. The speech segments are passed to an Automatic Speech Recognition (ASR) engine, and words from the 1-best ASR output, as well as posterior-weighted word counts collected from ASR lattices, are used as features to an SVM based classifier. Acoustic concepts are extracted using the 3-gram lattice counts of two Acoustic Concept Recognition (ACR) systems trained on 7 broad classes. MED results are reported on a subset of the NIST 2011 TRECVID data. We find that spoken concepts using lattices yield a 15% relative improvement in Average Pmiss (APM) over 1-best based features. Further, the proposed spoken concepts gave a 30% relative gain in APM over the ACR-based MED system using 7 classes. Lastly, we obtain an 8% relative APM improvement after score-level fusion of both concept types, showing the effective coupling of both approaches.
Keywords :
acoustic signal detection; audio signal processing; multimedia communication; speech recognition; support vector machines; video signal processing; 3-gram lattice count; ACR system; ACR-based MED system; ASR engine; NIST 2011 TRECVID data; SVM based classifier; acoustic concept extraction; acoustic concept recognition system; audio processing; audio recognition; audio segmentation; automatic speech recognition engine; multimedia event detection; online video search; posterior-weighted word count; speech segments; spoken concept extraction; Acoustics; Feature extraction; Lattices; Multimedia communication; Speech; Speech recognition; Videos; Multimedia event detection; acoustic event recognition; lattice N-gram counts; segmentation; speech recognition;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on
Conference_Location :
Vancouver, BC
ISSN :
1520-6149
Type :
conf
DOI :
10.1109/ICASSP.2013.6638340
Filename :
6638340
Link To Document :
بازگشت