TTS evaluation: Double-ended objective quality measures

We address the problem of TTS speech quality evaluation and propose a double-ended objective measure in the form of average spectral distortion between time-aligned reference and synthesized speech, where the reference signal is made available as the speech of the text input to the TTS spoken by the same speaker as the unit-database. We detail the time-aligned spectral distortion measure calculated via dynamic time-warping and apply this measure for comparison of the effectiveness of 5 different automatic segmentation techniques for annotating the unit-database for two Indian languages, Tamil and Kannada. We also show that the proposed measure yields more meaningful scores than the PESQ measure which is popular for codec evaluation, but likely to be less suited for TTS evaluation due to the lack of a rigorous time-alignment as is done in the proposed spectral distortion measure here.