Cross-lingual paraphrase identification
<|reference_start|>Cross-lingual paraphrase identification: The paraphrase identification task involves measuring semantic similarity between two short sentences. It is a tricky task, and multilingual paraphrase identification is even more challenging. In this work, we train a bi-encoder model in a contrastive manner to detect hard paraphrases across multiple languages. This approach allows us to use model-produced embeddings for various tasks, such as semantic search. We evaluate our model on downstream tasks and also assess embedding space quality. Our performance is comparable to state-of-the-art cross-encoders, with only a minimal relative drop of 7-10% on the chosen dataset, while keeping decent quality of embeddings.<|reference_end|>
@article{fedorova2024cross-lingual,
    title={Cross-lingual paraphrase identification},
    author={Inessa Fedorova and Aleksei Musatow},
    journal={arXiv preprint arXiv:2406.15066},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.15066},
    primaryClass={cs.CL}
}