PARAPHRASUS : A Comprehensive Benchmark for Evaluating Paraphrase Detection Models
<|reference_start|>PARAPHRASUS : A Comprehensive Benchmark for Evaluating Paraphrase Detection Models: The task of determining whether two texts are paraphrases has long been a challenge in NLP. However, the prevailing notion of paraphrase is often quite simplistic, offering only a limited view of the vast spectrum of paraphrase phenomena. Indeed, we find that evaluating models in a paraphrase dataset can leave uncertainty about their true semantic understanding. To alleviate this, we release paraphrasus, a benchmark designed for multi-dimensional assessment of paraphrase detection models and finer model selection. We find that paraphrase detection models under a fine-grained evaluation lens exhibit trade-offs that cannot be captured through a single classification dataset.<|reference_end|>
@article{michail2024paraphrasus,
    title={PARAPHRASUS : A Comprehensive Benchmark for Evaluating Paraphrase
  Detection Models},
    author={Andrianos Michail, Simon Clematide, Juri Opitz},
    journal={arXiv preprint arXiv:2409.12060},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.12060},
    primaryClass={cs.CL cs.AI}
}