Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism
<|reference_start|>Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism: Neural Information Retrieval (NIR) has significantly improved upon heuristic-based Information Retrieval (IR) systems. Yet, failures remain frequent, the models used often being unable to retrieve documents relevant to the user's query. We address this challenge by proposing a lightweight abstention mechanism tailored for real-world constraints, with particular emphasis placed on the reranking phase. We introduce a protocol for evaluating abstention strategies in black-box scenarios (typically encountered when relying on API services), demonstrating their efficacy, and propose a simple yet effective data-driven mechanism. We provide open-source code for experiment replication and abstention implementation, fostering wider adoption and application in diverse contexts.<|reference_end|>
@article{gisserot-boukhlef2024towards,
    title={Towards Trustworthy Reranking: A Simple yet Effective Abstention
  Mechanism},
    author={Hippolyte Gisserot-Boukhlef, Manuel Faysse, Emmanuel Malherbe,
  C'eline Hudelot, Pierre Colombo},
    journal={arXiv preprint arXiv:2402.12997},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12997},
    primaryClass={cs.IR cs.CL}
}