title: RetroLLM: Empowering Large Language Models to Retrieve Fine-grained
    Evidence within Generation
summary: Large language models (LLMs) exhibit remarkable generative capabilities but
  often suffer from hallucinations. Retrieval-augmented generation (RAG) offers
  an effective solution by incorporating external knowledge, but existing methods
  still face several limitations: additional deployment costs of separate
  retrievers, redundant input tokens from retrieved text chunks, and the lack of
  joint optimization of retrieval and generation. To address these issues, we
  propose RetroLLM, a unified framework that integrates retrieval and
  generation into a single, cohesive process, enabling LLMs to directly generate
  fine-grained evidence from the corpus with constrained decoding. Moreover, to
  mitigate false pruning in the process of constrained evidence generation, we
  introduce (1) hierarchical FM-Index constraints, which generate
  corpus-constrained clues to identify a subset of relevant documents before
  evidence generation, reducing irrelevant decoding space; and (2) a
  forward-looking constrained decoding strategy, which considers the relevance of
  future sequences to improve evidence accuracy. Extensive experiments on five
  open-domain QA datasets demonstrate RetroLLM's superior performance across both
  in-domain and out-of-domain tasks. The code is available at
  https://github.com/sunnynexus/RetroLLM.
paper_authors._id: 6760ec669e797e610c5979e0
paper_authors.name: Xiaoxi Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:22:41.868Z
paper_authors._id: 66e03eace17fb5ff054b7686
paper_authors.avatarUrl: /avatars/2b739ff11e43dd9e701c647a92617f20.svg
paper_authors.fullname: Xiaoxi Li
paper_authors.type: user
paper_authors.user: lixiaoxi45
paper_authors._id: 6760ec669e797e610c5979e1
paper_authors.name: Jiajie Jin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:21:52.824Z
paper_authors._id: 6695f14df0ffd8e3a379ad61
paper_authors.avatarUrl: /avatars/5ebb7e55ee9c2d93850b279f440675b0.svg
paper_authors.fullname: Jiajie Jin
paper_authors.type: user
paper_authors.user: jinjiajie
paper_authors._id: 6760ec669e797e610c5979e2
paper_authors.name: Yujia Zhou
paper_authors._id: 6760ec669e797e610c5979e3
paper_authors.name: Yongkang Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:21:03.390Z
paper_authors._id: 62f3a590261bc5fb2e072a5f
paper_authors.avatarUrl: /avatars/d65d362ddc32aca3d6c564252d81e109.svg
paper_authors.fullname: YongkangWu
paper_authors.type: user
paper_authors.user: wuyongkang
paper_authors._id: 6760ec669e797e610c5979e4
paper_authors.name: Zhonghua Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:20:44.760Z
paper_authors._id: 67145068485b75297d6a5ec5
paper_authors.avatarUrl: /avatars/67cb59ec072783ee5e4d909c4615bd44.svg
paper_authors.fullname: LiZhongHua
paper_authors.type: user
paper_authors.user: Benen2024
paper_authors._id: 6760ec669e797e610c5979e5
paper_authors.name: Qi Ye
paper_authors._id: 6760ec669e797e610c5979e6
paper_authors.name: Zhicheng Dou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:20:11.013Z
paper_authors._id: 66f0bf59e9d50ec57febf751
paper_authors.avatarUrl: /avatars/be97941e60064e5dd806c6fe9db3c537.svg
paper_authors.fullname: Zhicheng Dou
paper_authors.type: user
paper_authors.user: douzc
paper_title: RetroLLM: Empowering Large Language Models to Retrieve Fine-grained
    Evidence within Generation
paper_summary: Large language models (LLMs) exhibit remarkable generative capabilities but
  often suffer from hallucinations. Retrieval-augmented generation (RAG) offers
  an effective solution by incorporating external knowledge, but existing methods
  still face several limitations: additional deployment costs of separate
  retrievers, redundant input tokens from retrieved text chunks, and the lack of
  joint optimization of retrieval and generation. To address these issues, we
  propose RetroLLM, a unified framework that integrates retrieval and
  generation into a single, cohesive process, enabling LLMs to directly generate
  fine-grained evidence from the corpus with constrained decoding. Moreover, to
  mitigate false pruning in the process of constrained evidence generation, we
  introduce (1) hierarchical FM-Index constraints, which generate
  corpus-constrained clues to identify a subset of relevant documents before
  evidence generation, reducing irrelevant decoding space; and (2) a
  forward-looking constrained decoding strategy, which considers the relevance of
  future sequences to improve evidence accuracy. Extensive experiments on five
  open-domain QA datasets demonstrate RetroLLM's superior performance across both
  in-domain and out-of-domain tasks. The code is available at
  https://github.com/sunnynexus/RetroLLM.
paper_ai_summary: RetroLLM is a unified framework that integrates retrieval and generation for LLMs, using constraints and decoding strategies to enhance performance and accuracy in evidence generation.
paper_ai_keywords: LLMs hallucinations retrieval-augmented generation (RAG) unified framework constrained decoding hierarchical FM-Index constraints forward-looking constrained decoding strategy extensive experiments open-domain QA datasets