title: DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities
summary: Learned Sparse Retrieval (LSR) models use vocabularies from pre-trained
  transformers, which often split entities into nonsensical fragments. Splitting
  entities can reduce retrieval accuracy and limits the model's ability to
  incorporate up-to-date world knowledge not included in the training data. In
  this work, we enhance the LSR vocabulary with Wikipedia concepts and entities,
  enabling the model to resolve ambiguities more effectively and stay current
  with evolving knowledge. Central to our approach is a Dynamic Vocabulary (DyVo)
  head, which leverages existing entity embeddings and an entity retrieval
  component that identifies entities relevant to a query or document. We use the
  DyVo head to generate entity weights, which are then merged with word piece
  weights to create joint representations for efficient indexing and retrieval
  using an inverted index. In experiments across three entity-rich document
  ranking datasets, the resulting DyVo model substantially outperforms
  state-of-the-art baselines.
paper_authors._id: 6710f99feea7a876d49d4088
paper_authors.name: Thong Nguyen
paper_authors._id: 6710f99feea7a876d49d4089
paper_authors.name: Shubham Chatterjee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:31:33.851Z
paper_authors._id: 62118220cf354d3c41b3e1b6
paper_authors.avatarUrl: /avatars/d360a92001d7a09fcfe8290160e4dab9.svg
paper_authors.fullname: Shubham Chatterjee
paper_authors.type: user
paper_authors.user: ShubhamC
paper_authors._id: 6710f99feea7a876d49d408a
paper_authors.name: Sean MacAvaney
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:31:40.123Z
paper_authors._id: 61dc410f7813d319d7387635
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1666975288423-61dc410f7813d319d7387635.jpeg
paper_authors.fullname: Sean MacAvaney
paper_authors.type: user
paper_authors.user: macavaney
paper_authors._id: 6710f99feea7a876d49d408b
paper_authors.name: Iain Mackie
paper_authors._id: 6710f99feea7a876d49d408c
paper_authors.name: Jeff Dalton
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:31:48.759Z
paper_authors._id: 650c05f9d4a0852d3c89bfd2
paper_authors.avatarUrl: /avatars/a1b3bf4a861a8f0bd5a0ff3a076ed5a2.svg
paper_authors.fullname: dalton
paper_authors.type: user
paper_authors.user: jeffdalton
paper_authors._id: 6710f99feea7a876d49d408d
paper_authors.name: Andrew Yates
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:31:55.238Z
paper_authors._id: 5efef195ff69163f6f59e647
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1599574940380-5efef195ff69163f6f59e647.jpeg
paper_authors.fullname: Andrew Yates
paper_authors.type: user
paper_authors.user: andrewyates
paper_title: DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities
paper_summary: Learned Sparse Retrieval (LSR) models use vocabularies from pre-trained
  transformers, which often split entities into nonsensical fragments. Splitting
  entities can reduce retrieval accuracy and limits the model's ability to
  incorporate up-to-date world knowledge not included in the training data. In
  this work, we enhance the LSR vocabulary with Wikipedia concepts and entities,
  enabling the model to resolve ambiguities more effectively and stay current
  with evolving knowledge. Central to our approach is a Dynamic Vocabulary (DyVo)
  head, which leverages existing entity embeddings and an entity retrieval
  component that identifies entities relevant to a query or document. We use the
  DyVo head to generate entity weights, which are then merged with word piece
  weights to create joint representations for efficient indexing and retrieval
  using an inverted index. In experiments across three entity-rich document
  ranking datasets, the resulting DyVo model substantially outperforms
  state-of-the-art baselines.
paper_ai_keywords: Transformer Vocabulary Sparse Retrieval Entity Retrieval Wikipedia Concepts Entity Embeddings DyVo (Dynamic Vocabulary) Head Inverted Index Entity Weights Word Piece Weights Entity-Rich Document Ranking Datasets State-of-the-Art Baselines