Fusion of magnitude and phase-based features for objective evaluation of TTS voice

This paper analyzes the distance-based objective measures for evaluation of Text-to-Speech (TTS) systems (which is generally used objective measures). In this paper, we discuss some aspects of evaluation of speech quality of synthesized speech. Some of the limitations and issues of subjective evaluation are discussed and importance of objective measures is presented. Traditional objective measure using Dynamic Time Warping (DTW) distance is used in this work. We have used magnitude and phase-based features as well as auditory features to check effectiveness of objective measures for predicting quality of TTS voice. In particular, Mel Frequency Cepstral Coefficients (MFCC) features and phase-based Modified Group Delay-based Cepstral Coefficients (MGDCC) alone have no good correlation with subjective scores. However, feature-level fusion of MFCC and MGDCC gives better correlation than all other feature sets. With this fusion, we obtained value of correlation coefficient, -0.3 and -0.32 for Blizzard Challenge databases 2010 and 2011, respectively. The results also show significance of phase-based features for objective measures when used along with magnitude-based features. The experimental results show that distance-based measures still do not work well with Blizzard Challenge databases and need more general objective measures for measuring quality of TTS voice.