  Most existing word alignment methods rely on manual alignment datasets or
parallel corpora, which limits their usefulness. Here, to mitigate the
dependence on manual data, we broaden the source of supervision by relaxing the
requirement for correct, fully-aligned, and parallel sentences. Specifically,
we make noisy, partially aligned, and non-parallel paragraphs. We then use such
a large-scale weakly-supervised dataset for word alignment pre-training via
span prediction. Extensive experiments with various settings empirically
demonstrate that our approach, which is named WSPAlign, is an effective and
scalable way to pre-train word aligners without manual data. When fine-tuned on
standard benchmarks, WSPAlign has set a new state-of-the-art by improving upon
the best-supervised baseline by 3.3~6.1 points in F1 and 1.5~6.1 points in AER.
Furthermore, WSPAlign also achieves competitive performance compared with the
corresponding baselines in few-shot, zero-shot and cross-lingual tests, which
demonstrates that WSPAlign is potentially more practical for low-resource
languages than existing methods.

Computation and Language