Label-Synchronous Neural Transducer for E2E Simultaneous Speech Translation
<|reference_start|>Label-Synchronous Neural Transducer for E2E Simultaneous Speech Translation: While the neural transducer is popular for online speech recognition, simultaneous speech translation (SST) requires both streaming and re-ordering capabilities. This paper presents the LS-Transducer-SST, a label-synchronous neural transducer for SST, which naturally possesses these two properties. The LS-Transducer-SST dynamically decides when to emit translation tokens based on an Auto-regressive Integrate-and-Fire (AIF) mechanism. A latency-controllable AIF is also proposed, which can control the quality-latency trade-off either only during decoding, or it can be used in both decoding and training. The LS-Transducer-SST can naturally utilise monolingual text-only data via its prediction network which helps alleviate the key issue of data sparsity for E2E SST. During decoding, a chunk-based incremental joint decoding technique is designed to refine and expand the search space. Experiments on the Fisher-CallHome Spanish (Es-En) and MuST-C En-De data show that the LS-Transducer-SST gives a better quality-latency trade-off than existing popular methods. For example, the LS-Transducer-SST gives a 3.1/2.9 point BLEU increase (Es-En/En-De) relative to CAAT at a similar latency and a 1.4 s reduction in average lagging latency with similar BLEU scores relative to Wait-k.<|reference_end|>
@article{deng2024label-synchronous,
    title={Label-Synchronous Neural Transducer for E2E Simultaneous Speech
  Translation},
    author={Keqi Deng, Philip C. Woodland},
    journal={arXiv preprint arXiv:2406.04541},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.04541},
    primaryClass={cs.CL eess.AS}
}