title: OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs
summary: Despite the recent advancements in Large Language Models (LLMs), which have
  significantly enhanced the generative capabilities for various NLP tasks, LLMs
  still face limitations in directly handling retrieval tasks. However, many
  practical applications demand the seamless integration of both retrieval and
  generation. This paper introduces a novel and efficient One-pass Generation and
  retrieval framework (OneGen), designed to improve LLMs' performance on tasks
  that require both generation and retrieval. The proposed framework bridges the
  traditionally separate training approaches for generation and retrieval by
  incorporating retrieval tokens generated autoregressively. This enables a
  single LLM to handle both tasks simultaneously in a unified forward pass. We
  conduct experiments on two distinct types of composite tasks, RAG and Entity
  Linking, to validate the pluggability, effectiveness, and efficiency of OneGen
  in training and inference. Furthermore, our results show that integrating
  generation and retrieval within the same context preserves the generative
  capabilities of LLMs while improving retrieval performance. To the best of our
  knowledge, OneGen is the first to enable LLMs to conduct vector retrieval
  during the generation.
paper_authors._id: 66dfdaf8c8818243494a6da4
paper_authors.name: Jintian Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-10T08:46:24.769Z
paper_authors._id: 63f89fe7565506a9cadcd2cf
paper_authors.avatarUrl: /avatars/7eb449a1109dcff051cb3ba680f0c082.svg
paper_authors.fullname: Jintian Zhang
paper_authors.type: user
paper_authors.user: MikeDean
paper_authors._id: 66dfdaf8c8818243494a6da5
paper_authors.name: Cheng Peng
paper_authors._id: 66dfdaf8c8818243494a6da6
paper_authors.name: Mengshu Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-10T08:47:03.170Z
paper_authors._id: 64f6d1ed46284aa28d9abf6c
paper_authors.avatarUrl: /avatars/d6beaecfd00345e4a664862fff217427.svg
paper_authors.fullname: Sun mengshu
paper_authors.type: user
paper_authors.user: sunmengshu
paper_authors._id: 66dfdaf8c8818243494a6da7
paper_authors.name: Xiang Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-10T08:47:46.247Z
paper_authors._id: 64c073417255ff868721fb54
paper_authors.avatarUrl: /avatars/c2a00114598aa89f071b937f0af4077f.svg
paper_authors.fullname: Xiang CHEN
paper_authors.type: user
paper_authors.user: xiangchen-dvi
paper_authors._id: 66dfdaf8c8818243494a6da8
paper_authors.name: Lei Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-10T08:47:57.401Z
paper_authors._id: 64588a9caa426bae5e080567
paper_authors.avatarUrl: /avatars/76c11bfbe64058014d4b2b89d18ba2e4.svg
paper_authors.fullname: Lei Liang
paper_authors.type: user
paper_authors.user: square0083
paper_authors._id: 66dfdaf8c8818243494a6da9
paper_authors.name: Zhiqiang Zhang
paper_authors._id: 66dfdaf8c8818243494a6daa
paper_authors.name: Jun Zhou
paper_authors._id: 66dfdaf8c8818243494a6dab
paper_authors.name: Huajun Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-10T08:49:05.458Z
paper_authors._id: 64931296137833d7ec7689cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64931296137833d7ec7689cd/TBihNdp1ZwIWjhfAWjRr6.jpeg
paper_authors.fullname: Huajun Chen
paper_authors.type: user
paper_authors.user: huajunsir
paper_authors._id: 66dfdaf8c8818243494a6dac
paper_authors.name: Ningyu Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-10T07:10:27.839Z
paper_authors._id: 620b3bbb0668e435407c8d0a
paper_authors.avatarUrl: /avatars/e0fccbb2577d76088e09f054c35cffbc.svg
paper_authors.fullname: Ningyu Zhang
paper_authors.type: user
paper_authors.user: Ningyu
paper_title: OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs
paper_summary: Despite the recent advancements in Large Language Models (LLMs), which have
  significantly enhanced the generative capabilities for various NLP tasks, LLMs
  still face limitations in directly handling retrieval tasks. However, many
  practical applications demand the seamless integration of both retrieval and
  generation. This paper introduces a novel and efficient One-pass Generation and
  retrieval framework (OneGen), designed to improve LLMs' performance on tasks
  that require both generation and retrieval. The proposed framework bridges the
  traditionally separate training approaches for generation and retrieval by
  incorporating retrieval tokens generated autoregressively. This enables a
  single LLM to handle both tasks simultaneously in a unified forward pass. We
  conduct experiments on two distinct types of composite tasks, RAG and Entity
  Linking, to validate the pluggability, effectiveness, and efficiency of OneGen
  in training and inference. Furthermore, our results show that integrating
  generation and retrieval within the same context preserves the generative
  capabilities of LLMs while improving retrieval performance. To the best of our
  knowledge, OneGen is the first to enable LLMs to conduct vector retrieval
  during the generation.
paper_ai_summary: OneGen is a framework that integrates retrieval and generation in a single pass within Large Language Models, improving retrieval performance without compromising generative capabilities.
paper_ai_keywords: Large Language Models One-pass Generation retrieval tokens autoregressive generation unified forward pass RAG Entity Linking vector retrieval