title: Backtracing: Retrieving the Cause of the Query
summary: Many online content portals allow users to ask questions to supplement their
  understanding (e.g., of lectures). While information retrieval (IR) systems may
  provide answers for such user queries, they do not directly assist content
  creators -- such as lecturers who want to improve their content -- identify
  segments that _caused_ a user to ask those questions. We introduce the task of
  backtracing, in which systems retrieve the text segment that most likely caused
  a user query. We formalize three real-world domains for which backtracing is
  important in improving content delivery and communication: understanding the
  cause of (a) student confusion in the Lecture domain, (b) reader curiosity in
  the News Article domain, and (c) user emotion in the Conversation domain. We
  evaluate the zero-shot performance of popular information retrieval methods and
  language modeling methods, including bi-encoder, re-ranking and
  likelihood-based methods and ChatGPT. While traditional IR systems retrieve
  semantically relevant information (e.g., details on "projection matrices" for a
  query "does projecting multiple times still lead to the same point?"), they
  often miss the causally relevant context (e.g., the lecturer states "projecting
  twice gets me the same answer as one projection"). Our results show that there
  is room for improvement on backtracing and it requires new retrieval
  approaches. We hope our benchmark serves to improve future retrieval systems
  for backtracing, spawning systems that refine content generation and identify
  linguistic triggers influencing user queries. Our code and data are
  open-sourced: https://github.com/rosewang2008/backtracing.
paper_authors._id: 65e958ef063e16f1c3fa7097
paper_authors.name: Rose E. Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-07T10:04:03.149Z
paper_authors._id: 62e2047491d6d37b6bff4dfb
paper_authors.avatarUrl: /avatars/1c758de8d531aa17e516aa11c6102f4a.svg
paper_authors.fullname: Rose E Wang
paper_authors.type: user
paper_authors.user: rewang
paper_authors._id: 65e958ef063e16f1c3fa7098
paper_authors.name: Pawan Wirawarn
paper_authors._id: 65e958ef063e16f1c3fa7099
paper_authors.name: Omar Khattab
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-07T10:04:18.338Z
paper_authors._id: 5f764762f1e7ef6e919a1e0f
paper_authors.avatarUrl: /avatars/9445f6efa3f78a870e338988d9ce5573.svg
paper_authors.fullname: Omar Khattab
paper_authors.type: user
paper_authors.user: okhattab
paper_authors._id: 65e958ef063e16f1c3fa709a
paper_authors.name: Noah Goodman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:49:40.433Z
paper_authors._id: 67321274c1f20c742bcf7a8d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ltcQhre6eDRVzn6Vbbyhu.png
paper_authors.fullname: Noah D. Goodman
paper_authors.type: user
paper_authors.user: ngoodman
paper_authors._id: 65e958ef063e16f1c3fa709b
paper_authors.name: Dorottya Demszky
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-03-07T17:39:15.210Z
paper_authors._id: 606e3a4c7f3150d181025dd1
paper_authors.avatarUrl: /avatars/224ee859c1df3f6c4ab59910f2a4732b.svg
paper_authors.fullname: Dora Demszky
paper_authors.type: user
paper_authors.user: ddemszky
paper_title: Backtracing: Retrieving the Cause of the Query
paper_summary: Many online content portals allow users to ask questions to supplement their
  understanding (e.g., of lectures). While information retrieval (IR) systems may
  provide answers for such user queries, they do not directly assist content
  creators -- such as lecturers who want to improve their content -- identify
  segments that _caused_ a user to ask those questions. We introduce the task of
  backtracing, in which systems retrieve the text segment that most likely caused
  a user query. We formalize three real-world domains for which backtracing is
  important in improving content delivery and communication: understanding the
  cause of (a) student confusion in the Lecture domain, (b) reader curiosity in
  the News Article domain, and (c) user emotion in the Conversation domain. We
  evaluate the zero-shot performance of popular information retrieval methods and
  language modeling methods, including bi-encoder, re-ranking and
  likelihood-based methods and ChatGPT. While traditional IR systems retrieve
  semantically relevant information (e.g., details on "projection matrices" for a
  query "does projecting multiple times still lead to the same point?"), they
  often miss the causally relevant context (e.g., the lecturer states "projecting
  twice gets me the same answer as one projection"). Our results show that there
  is room for improvement on backtracing and it requires new retrieval
  approaches. We hope our benchmark serves to improve future retrieval systems
  for backtracing, spawning systems that refine content generation and identify
  linguistic triggers influencing user queries. Our code and data are
  open-sourced: https://github.com/rosewang2008/backtracing.
paper_ai_keywords: bi-encoder re-ranking likelihood-based methods ChatGPT semantic relevance causal relevance