  Recently, Conformer based CTC/AED model has become a mainstream architecture
for ASR. In this paper, based on our prior work, we identify and integrate
several approaches to achieve further improvements for ASR tasks, which we
denote as multi-loss, multi-path and multi-level, summarized as "3M" model.
Specifically, multi-loss refers to the joint CTC/AED loss and multi-path
denotes the Mixture-of-Experts(MoE) architecture which can effectively increase
the model capacity without remarkably increasing computation cost. Multi-level
means that we introduce auxiliary loss at multiple level of a deep model to
help training. We evaluate our proposed method on the public WenetSpeech
dataset and experimental results show that the proposed method provides
12.2%-17.6% relative CER improvement over the baseline model trained by Wenet
toolkit. On our large scale dataset of 150k hours corpus, the 3M model has also
shown obvious superiority over the baseline Conformer model. Code is publicly
available at https://github.com/tencent-ailab/3m-asr.

Sound;Computation and Language;Audio and Speech Processing