title: Promptriever: Instruction-Trained Retrievers Can Be Prompted Like
    Language Models
summary: Instruction-tuned language models (LM) are able to respond to imperative
  commands, providing a more natural user interface compared to their base
  counterparts. In this work, we present Promptriever, the first retrieval model
  able to be prompted like an LM. To train Promptriever, we curate and release a
  new instance-level instruction training set from MS MARCO, spanning nearly 500k
  instances. Promptriever not only achieves strong performance on standard
  retrieval tasks, but also follows instructions. We observe: (1) large gains
  (reaching SoTA) on following detailed relevance instructions (+14.3 p-MRR /
  +3.1 nDCG on FollowIR), (2) significantly increased robustness to lexical
  choices/phrasing in the query+instruction (+12.9 Robustness@10 on InstructIR),
  and (3) the ability to perform hyperparameter search via prompting to reliably
  improve retrieval performance (+1.4 average increase on BEIR). Promptriever
  demonstrates that retrieval models can be controlled with prompts on a
  per-query basis, setting the stage for future work aligning LM prompting
  techniques with information retrieval.
paper_authors._id: 66ea2bb2c7dd4e003cd574ce
paper_authors.name: Orion Weller
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-19T18:05:49.426Z
paper_authors._id: 6362d9712691058b19de1ba4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6362d9712691058b19de1ba4/c9QrA2oE6lcs_46ShaTY1.jpeg
paper_authors.fullname: Orion Weller
paper_authors.type: user
paper_authors.user: orionweller
paper_authors._id: 66ea2bb2c7dd4e003cd574cf
paper_authors.name: Benjamin Van Durme
paper_authors._id: 66ea2bb2c7dd4e003cd574d0
paper_authors.name: Dawn Lawrie
paper_authors._id: 66ea2bb2c7dd4e003cd574d1
paper_authors.name: Ashwin Paranjape
paper_authors._id: 66ea2bb2c7dd4e003cd574d2
paper_authors.name: Yuhao Zhang
paper_authors._id: 66ea2bb2c7dd4e003cd574d3
paper_authors.name: Jack Hessel
paper_title: Promptriever: Instruction-Trained Retrievers Can Be Prompted Like
    Language Models
paper_summary: Instruction-tuned language models (LM) are able to respond to imperative
  commands, providing a more natural user interface compared to their base
  counterparts. In this work, we present Promptriever, the first retrieval model
  able to be prompted like an LM. To train Promptriever, we curate and release a
  new instance-level instruction training set from MS MARCO, spanning nearly 500k
  instances. Promptriever not only achieves strong performance on standard
  retrieval tasks, but also follows instructions. We observe: (1) large gains
  (reaching SoTA) on following detailed relevance instructions (+14.3 p-MRR /
  +3.1 nDCG on FollowIR), (2) significantly increased robustness to lexical
  choices/phrasing in the query+instruction (+12.9 Robustness@10 on InstructIR),
  and (3) the ability to perform hyperparameter search via prompting to reliably
  improve retrieval performance (+1.4 average increase on BEIR). Promptriever
  demonstrates that retrieval models can be controlled with prompts on a
  per-query basis, setting the stage for future work aligning LM prompting
  techniques with information retrieval.
paper_ai_keywords: instruction-tuned language models retrieval model MS MARCO FollowIR nDCG InstructIR Robustness@10 BEIR hyperparameter search