title: How Does Generative Retrieval Scale to Millions of Passages?
summary: Popularized by the Differentiable Search Index, the emerging paradigm of
  generative retrieval re-frames the classic information retrieval problem into a
  sequence-to-sequence modeling task, forgoing external indices and encoding an
  entire document corpus within a single Transformer. Although many different
  approaches have been proposed to improve the effectiveness of generative
  retrieval, they have only been evaluated on document corpora on the order of
  100k in size. We conduct the first empirical study of generative retrieval
  techniques across various corpus scales, ultimately scaling up to the entire MS
  MARCO passage ranking task with a corpus of 8.8M passages and evaluating model
  sizes up to 11B parameters. We uncover several findings about scaling
  generative retrieval to millions of passages; notably, the central importance
  of using synthetic queries as document representations during indexing, the
  ineffectiveness of existing proposed architecture modifications when accounting
  for compute cost, and the limits of naively scaling model parameters with
  respect to retrieval performance. While we find that generative retrieval is
  competitive with state-of-the-art dual encoders on small corpora, scaling to
  millions of passages remains an important and unsolved challenge. We believe
  these findings will be valuable for the community to clarify the current state
  of generative retrieval, highlight the unique challenges, and inspire new
  research directions.
paper_authors._id: 646afce31556443f24ba0d73
paper_authors.name: Ronak Pradeep
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-08T12:47:16.384Z
paper_authors._id: 5ec824d4968f6028e0559f6f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5ec824d4968f6028e0559f6f/b7C7D41DTmVOa0DHJmSE9.jpeg
paper_authors.fullname: Ronak Pradeep
paper_authors.type: user
paper_authors.user: ronak
paper_authors._id: 646afce31556443f24ba0d74
paper_authors.name: Kai Hui
paper_authors._id: 646afce31556443f24ba0d75
paper_authors.name: Jai Gupta
paper_authors._id: 646afce31556443f24ba0d76
paper_authors.name: Adam D. Lelkes
paper_authors._id: 646afce31556443f24ba0d77
paper_authors.name: Honglei Zhuang
paper_authors._id: 646afce31556443f24ba0d78
paper_authors.name: Jimmy Lin
paper_authors._id: 646afce31556443f24ba0d79
paper_authors.name: Donald Metzler
paper_authors._id: 646afce31556443f24ba0d7a
paper_authors.name: Vinh Q. Tran
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-22T18:01:28.955Z
paper_authors._id: 646b7ede628e5b50b2de8d5a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/y_be_9O4FO1sbRiNKzuJd.jpeg
paper_authors.fullname: Vinh Q. Tran
paper_authors.type: user
paper_authors.user: vqtran
paper_title: How Does Generative Retrieval Scale to Millions of Passages?
paper_summary: Popularized by the Differentiable Search Index, the emerging paradigm of
  generative retrieval re-frames the classic information retrieval problem into a
  sequence-to-sequence modeling task, forgoing external indices and encoding an
  entire document corpus within a single Transformer. Although many different
  approaches have been proposed to improve the effectiveness of generative
  retrieval, they have only been evaluated on document corpora on the order of
  100k in size. We conduct the first empirical study of generative retrieval
  techniques across various corpus scales, ultimately scaling up to the entire MS
  MARCO passage ranking task with a corpus of 8.8M passages and evaluating model
  sizes up to 11B parameters. We uncover several findings about scaling
  generative retrieval to millions of passages; notably, the central importance
  of using synthetic queries as document representations during indexing, the
  ineffectiveness of existing proposed architecture modifications when accounting
  for compute cost, and the limits of naively scaling model parameters with
  respect to retrieval performance. While we find that generative retrieval is
  competitive with state-of-the-art dual encoders on small corpora, scaling to
  millions of passages remains an important and unsolved challenge. We believe
  these findings will be valuable for the community to clarify the current state
  of generative retrieval, highlight the unique challenges, and inspire new
  research directions.
paper_ai_summary: Generative retrieval scales to large corpora with significant findings on the importance of synthetic queries and the limits of parameter scaling.
paper_ai_keywords: generative retrieval sequence-to-sequence Transformer synthetic queries MS MARCO passage ranking dual encoders