  Document retrieval is a core component of many knowledge-intensive natural
language processing task formulations such as fact verification and question
answering. Sources of textual knowledge, such as Wikipedia articles, condition
the generation of answers from the models. Recent advances in retrieval use
sequence-to-sequence models to incrementally predict the title of the
appropriate Wikipedia page given a query. However, this method requires
supervision in the form of human annotation to label which Wikipedia pages
contain appropriate context. This paper introduces a distant-supervision method
that does not require any annotation to train autoregressive retrievers that
attain competitive R-Precision and Recall in a zero-shot setting. Furthermore
we show that with task-specific supervised fine-tuning, autoregressive
retrieval performance for two Wikipedia-based fact verification tasks can
approach or even exceed full supervision using less than $1/4$ of the annotated
data indicating possible directions for data-efficient autoregressive
retrieval.

Computation and Language;Artificial Intelligence;Information Retrieval;Machine Learning