A method of attention-based end-to-end (E2E) automatic speech recognition (ASR) training includes performing cross entropy training of a model based on one or more input features of a speech signal; performing a beam search of the model on which the cross entropy training is performed to generate an n-good hypothesis list of output hypotheses; and determining an optimal hypothesis in the generated n-optimal hypothesis list. The method further includes determining a character-based gradient and a word-based gradient based on a model on which cross entropy training is performed and a loss function that maximizes a distance between the reference sequence and the determined one optimal hypothesis; and performing back propagation of the determined character-based gradient and the determined word-based gradient to the model to update the model.
Large interval tracking for attention-based end-to-end speech recognition