title: ReALM: Reference Resolution As Language Modeling
summary: Reference resolution is an important problem, one that is essential to
  understand and successfully handle context of different kinds. This context
  includes both previous turns and context that pertains to non-conversational
  entities, such as entities on the user's screen or those running in the
  background. While LLMs have been shown to be extremely powerful for a variety
  of tasks, their use in reference resolution, particularly for
  non-conversational entities, remains underutilized. This paper demonstrates how
  LLMs can be used to create an extremely effective system to resolve references
  of various types, by showing how reference resolution can be converted into a
  language modeling problem, despite involving forms of entities like those on
  screen that are not traditionally conducive to being reduced to a text-only
  modality. We demonstrate large improvements over an existing system with
  similar functionality across different types of references, with our smallest
  model obtaining absolute gains of over 5% for on-screen references. We also
  benchmark against GPT-3.5 and GPT-4, with our smallest model achieving
  performance comparable to that of GPT-4, and our larger models substantially
  outperforming it.
paper_authors._id: 660a24f565ffe042315e231d
paper_authors.name: Joel Ruben Antony Moniz
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-13T08:25:17.390Z
paper_authors._id: 6650e6b9e29125c00a919a4d
paper_authors.avatarUrl: /avatars/b0093fa9da51e42168545e83750c3e81.svg
paper_authors.fullname: Joel Moniz
paper_authors.type: user
paper_authors.user: joelmoniz
paper_authors._id: 660a24f565ffe042315e231e
paper_authors.name: Soundarya Krishnan
paper_authors._id: 660a24f565ffe042315e231f
paper_authors.name: Melis Ozyildirim
paper_authors._id: 660a24f565ffe042315e2320
paper_authors.name: Prathamesh Saraf
paper_authors._id: 660a24f565ffe042315e2321
paper_authors.name: Halim Cagri Ates
paper_authors._id: 660a24f565ffe042315e2322
paper_authors.name: Yuan Zhang
paper_authors._id: 660a24f565ffe042315e2323
paper_authors.name: Hong Yu
paper_authors._id: 660a24f565ffe042315e2324
paper_authors.name: Nidhi Rajshree
paper_title: ReALM: Reference Resolution As Language Modeling
paper_summary: Reference resolution is an important problem, one that is essential to
  understand and successfully handle context of different kinds. This context
  includes both previous turns and context that pertains to non-conversational
  entities, such as entities on the user's screen or those running in the
  background. While LLMs have been shown to be extremely powerful for a variety
  of tasks, their use in reference resolution, particularly for
  non-conversational entities, remains underutilized. This paper demonstrates how
  LLMs can be used to create an extremely effective system to resolve references
  of various types, by showing how reference resolution can be converted into a
  language modeling problem, despite involving forms of entities like those on
  screen that are not traditionally conducive to being reduced to a text-only
  modality. We demonstrate large improvements over an existing system with
  similar functionality across different types of references, with our smallest
  model obtaining absolute gains of over 5% for on-screen references. We also
  benchmark against GPT-3.5 and GPT-4, with our smallest model achieving
  performance comparable to that of GPT-4, and our larger models substantially
  outperforming it.
paper_ai_summary: LLMs are effectively utilized to resolve various types of references, including non-conversational entities on screen, by converting reference resolution into a language modeling problem, showing significant improvements over existing systems.
paper_ai_keywords: LLMs reference resolution language modeling problem