Combining HMM spectrum models and ANN prosody models for speech synthesis of syllable prominent languages

In this paper, an approach that combines HMM spectrum models and ANN prosody models is proposed to construct a speech synthesis system. Currently, a Mandarin corpus is used to show the feasibility of this approach. We hope that this approach can be used in other syllable prominent languages like Min-Nan and Hakka. In the training phase, DCC (discrete cepstrum coefficients) are computed for each frame of the training corpus and used as spectral parameters. Multiple utterances of a syllable are first grouped into a few clusters according to their DTW paths. Then, each cluster´s syllable utterances are used to train an HMM. In the synthesis phase, for each syllable of a sentence, an HMM of the syllable is selected first according to this syllable´s contextual data. Then, a duration ANN and duration means of the HMM states are used to determine how many frames an HMM state should be assigned. To achieve the goal of real-time synthesis, we propose an interpolation method to generate DCC coefficients for each frame. Next, speech signal is synthesized by using the DCC coefficients and the pitch contour generated by another ANN to control an HNM (harmonic plus noised model) based signal synthesizer. The results of perception tests show that our interpolation method obtains slightly more natural synthetic speech than the MLE method. Also, the duration ANN can have more natural synthetic speech than the duration means of HMM states.