Multi-objective optimization for semi-supervised discriminative language modeling

A method for semi-supervised language modeling, which was designed to improve the robustness of a language model (LM) obtained from manually transcribed (labeled) data, is proposed. The LM is implemented as a log-linear model, which employs a set of linguistic features derived from word or phoneme n-grams. The proposed method is formulated as a multi-objective optimization programming problem (MOP), which consists of two objective functions based on expected risks for labeled lattices and automatic speech recognition (ASR) lattices as unlabeled training data. The model is trained in a discriminative manner and acquired as a solution to the problem. In transcribing Japanese broadcast programs, the proposed method reduced word error rate by 6.3% compared with that achieved by a conventional trigram LM.