Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders
<|reference_start|>Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders: Existing cross-encoder re-rankers can be categorized as pointwise, pairwise, or listwise models. Pair- and listwise models allow passage interactions, which usually makes them more effective than pointwise models but also less efficient and less robust to input order permutations. To enable efficient permutation-invariant passage interactions during re-ranking, we propose a new cross-encoder architecture with inter-passage attention: the Set-Encoder. In Cranfield-style experiments on TREC Deep Learning and TIREx, the Set-Encoder is as effective as state-of-the-art listwise models while improving efficiency and robustness to input permutations. Interestingly, a pointwise model is similarly effective, but when additionally requiring the models to consider novelty, the Set-Encoder is more effective than its pointwise counterpart and retains its advantageous properties compared to other listwise models. Our code and models are publicly available at https://github.com/webis-de/set-encoder.<|reference_end|>
@article{schlatt2024set-encoder:,
    title={Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise
  Passage Re-Ranking with Cross-Encoders},
    author={Ferdinand Schlatt, Maik Fr"obe, Harrisen Scells, Shengyao Zhuang,
  Bevan Koopman, Guido Zuccon, Benno Stein, Martin Potthast, Matthias Hagen},
    journal={arXiv preprint arXiv:2404.06912},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.06912},
    primaryClass={cs.IR}
}