title: Recommender Systems with Generative Retrieval
summary: Modern recommender systems leverage large-scale retrieval models consisting
  of two stages: training a dual-encoder model to embed queries and candidates in
  the same space, followed by an Approximate Nearest Neighbor (ANN) search to
  select top candidates given a query's embedding. In this paper, we propose a
  new single-stage paradigm: a generative retrieval model which autoregressively
  decodes the identifiers for the target candidates in one phase. To do this,
  instead of assigning randomly generated atomic IDs to each item, we generate
  Semantic IDs: a semantically meaningful tuple of codewords for each item that
  serves as its unique identifier. We use a hierarchical method called RQ-VAE to
  generate these codewords. Once we have the Semantic IDs for all the items, a
  Transformer based sequence-to-sequence model is trained to predict the Semantic
  ID of the next item. Since this model predicts the tuple of codewords
  identifying the next item directly in an autoregressive manner, it can be
  considered a generative retrieval model. We show that our recommender system
  trained in this new paradigm improves the results achieved by current SOTA
  models on the Amazon dataset. Moreover, we demonstrate that the
  sequence-to-sequence model coupled with hierarchical Semantic IDs offers better
  generalization and hence improves retrieval of cold-start items for
  recommendations.
paper_authors._id: 645b13688b0751914f60b19d
paper_authors.name: Shashank Rajput
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-01T07:17:13.247Z
paper_authors._id: 6511e6b0c915724a65367268
paper_authors.avatarUrl: /avatars/cf69b69f27f51ecbed78b1fce856ff81.svg
paper_authors.fullname: Shashank Rajput
paper_authors.type: user
paper_authors.user: shashankMosaicML
paper_authors._id: 645b13688b0751914f60b19e
paper_authors.name: Nikhil Mehta
paper_authors._id: 645b13688b0751914f60b19f
paper_authors.name: Anima Singh
paper_authors._id: 645b13688b0751914f60b1a0
paper_authors.name: Raghunandan H. Keshavan
paper_authors._id: 645b13688b0751914f60b1a1
paper_authors.name: Trung Vu
paper_authors._id: 645b13688b0751914f60b1a2
paper_authors.name: Lukasz Heldt
paper_authors._id: 645b13688b0751914f60b1a3
paper_authors.name: Lichan Hong
paper_authors._id: 645b13688b0751914f60b1a4
paper_authors.name: Yi Tay
paper_authors._id: 645b13688b0751914f60b1a5
paper_authors.name: Vinh Q. Tran
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-23T09:18:01.499Z
paper_authors._id: 646b7ede628e5b50b2de8d5a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/y_be_9O4FO1sbRiNKzuJd.jpeg
paper_authors.fullname: Vinh Q. Tran
paper_authors.type: user
paper_authors.user: vqtran
paper_authors._id: 645b13688b0751914f60b1a6
paper_authors.name: Jonah Samost
paper_authors._id: 645b13688b0751914f60b1a7
paper_authors.name: Maciej Kula
paper_authors._id: 645b13688b0751914f60b1a8
paper_authors.name: Ed H. Chi
paper_authors._id: 645b13688b0751914f60b1a9
paper_authors.name: Maheswaran Sathiamoorthy
paper_title: Recommender Systems with Generative Retrieval
paper_summary: Modern recommender systems leverage large-scale retrieval models consisting
  of two stages: training a dual-encoder model to embed queries and candidates in
  the same space, followed by an Approximate Nearest Neighbor (ANN) search to
  select top candidates given a query's embedding. In this paper, we propose a
  new single-stage paradigm: a generative retrieval model which autoregressively
  decodes the identifiers for the target candidates in one phase. To do this,
  instead of assigning randomly generated atomic IDs to each item, we generate
  Semantic IDs: a semantically meaningful tuple of codewords for each item that
  serves as its unique identifier. We use a hierarchical method called RQ-VAE to
  generate these codewords. Once we have the Semantic IDs for all the items, a
  Transformer based sequence-to-sequence model is trained to predict the Semantic
  ID of the next item. Since this model predicts the tuple of codewords
  identifying the next item directly in an autoregressive manner, it can be
  considered a generative retrieval model. We show that our recommender system
  trained in this new paradigm improves the results achieved by current SOTA
  models on the Amazon dataset. Moreover, we demonstrate that the
  sequence-to-sequence model coupled with hierarchical Semantic IDs offers better
  generalization and hence improves retrieval of cold-start items for
  recommendations.
paper_ai_summary: A generative retrieval model using Semantic IDs and a Transformer sequence-to-sequence approach improves recommendation system performance and generalization, especially for cold-start items.
paper_ai_keywords: dual-encoder model Approximate Nearest Neighbor ANN generative retrieval model autoregressive model Semantic IDs codewords RQ-VAE Transformer sequence-to-sequence Amazon dataset cold-start items