Study of Semi supervised Approaches to Improving English Mandarin Code Switching Speech Recognition
In this paper, we present our overall efforts to improve the performance of a code switching speech recognition system using semi supervised training methods from lexicon learning to acoustic modeling, on the South East Asian Mandarin English (SEAME) data. We first investigate semi supervised lexicon learning approach to adapt the canonical lexicon, which is meant to alleviate the heavily accented pronunciation issue within the code switching conversation of the local area. As a result, the learned lexicon yields improved performance. Furthermore, we attempt to use semi supervised training to deal with those transcriptions that are highly mismatched between human transcribers and ASR system. Specifically, we conduct semi supervised training assuming those poorly transcribed data as unsupervised data. We found the semi supervised acoustic modeling can lead to improved results. Finally, to make up for the limitation of the conventional n gram language models due to data sparsity issue, we perform lattice rescoring using neural network language models, and significant WER reduction is obtained.
INTERSPEECH