How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise on Machine Translation
<|reference_start|>How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise on Machine Translation: The massive amounts of web-mined parallel data contain large amounts of noise. Semantic misalignment, as the primary source of the noise, poses a challenge for training machine translation systems. In this paper, we first study the impact of real-world hard-to-detect misalignment noise by proposing a process to simulate the realistic misalignment controlled by semantic similarity. After quantitatively analyzing the impact of simulated misalignment on machine translation, we show the limited effectiveness of widely used pre-filters to improve the translation performance, underscoring the necessity of more fine-grained ways to handle data noise. By observing the increasing reliability of the model's self-knowledge for distinguishing misaligned and clean data at the token-level, we propose a self-correction approach which leverages the model's prediction distribution to revise the training supervision from the ground-truth data over training time. Through comprehensive experiments, we show that our self-correction method not only improves translation performance in the presence of simulated misalignment noise but also proves effective for real-world noisy web-mined datasets across eight translation tasks.<|reference_end|>
@article{meng2024how,
    title={How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise
  on Machine Translation},
    author={Yan Meng, Di Wu, Christof Monz},
    journal={arXiv preprint arXiv:2407.02208},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02208},
    primaryClass={cs.CL cs.AI}
}