title: CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation
    Generation
summary: Retrieval-Augmented Generation (RAG) has become a powerful paradigm for
  enhancing large language models (LLMs) through external knowledge retrieval.
  Despite its widespread attention, existing academic research predominantly
  focuses on single-turn RAG, leaving a significant gap in addressing the
  complexities of multi-turn conversations found in real-world applications. To
  bridge this gap, we introduce CORAL, a large-scale benchmark designed to assess
  RAG systems in realistic multi-turn conversational settings. CORAL includes
  diverse information-seeking conversations automatically derived from Wikipedia
  and tackles key challenges such as open-domain coverage, knowledge intensity,
  free-form responses, and topic shifts. It supports three core tasks of
  conversational RAG: passage retrieval, response generation, and citation
  labeling. We propose a unified framework to standardize various conversational
  RAG methods and conduct a comprehensive evaluation of these methods on CORAL,
  demonstrating substantial opportunities for improving existing approaches.
paper_authors._id: 6722ddfb2b7a133fcb7a3978
paper_authors.name: Yiruo Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-31T09:54:45.558Z
paper_authors._id: 6530eba73572cd4fb78daf36
paper_authors.avatarUrl: /avatars/4e6f0f07bb4e0ed9e3c58d49eaf78fe5.svg
paper_authors.fullname: Yiruo Cheng
paper_authors.type: user
paper_authors.user: ariya2357
paper_authors._id: 6722ddfb2b7a133fcb7a3979
paper_authors.name: Kelong Mao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:37:57.951Z
paper_authors._id: 649a65605c74a2125c22bbc1
paper_authors.avatarUrl: /avatars/e7435d3aeeb59acc6f6f43b48d6982a0.svg
paper_authors.fullname: Mao
paper_authors.type: user
paper_authors.user: kyriemao
paper_authors._id: 6722ddfb2b7a133fcb7a397a
paper_authors.name: Ziliang Zhao
paper_authors._id: 6722ddfb2b7a133fcb7a397b
paper_authors.name: Guanting Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-31T09:54:42.575Z
paper_authors._id: 61cd4b833dd34ba1985e0753
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61cd4b833dd34ba1985e0753/BfHfrwotoMESpXZOHiIe4.png
paper_authors.fullname: KABI
paper_authors.type: user
paper_authors.user: dongguanting
paper_authors._id: 6722ddfb2b7a133fcb7a397c
paper_authors.name: Hongjin Qian
paper_authors._id: 6722ddfb2b7a133fcb7a397d
paper_authors.name: Yongkang Wu
paper_authors._id: 6722ddfb2b7a133fcb7a397e
paper_authors.name: Tetsuya Sakai
paper_authors._id: 6722ddfb2b7a133fcb7a397f
paper_authors.name: Ji-Rong Wen
paper_authors._id: 6722ddfb2b7a133fcb7a3980
paper_authors.name: Zhicheng Dou
paper_title: CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation
    Generation
paper_summary: Retrieval-Augmented Generation (RAG) has become a powerful paradigm for
  enhancing large language models (LLMs) through external knowledge retrieval.
  Despite its widespread attention, existing academic research predominantly
  focuses on single-turn RAG, leaving a significant gap in addressing the
  complexities of multi-turn conversations found in real-world applications. To
  bridge this gap, we introduce CORAL, a large-scale benchmark designed to assess
  RAG systems in realistic multi-turn conversational settings. CORAL includes
  diverse information-seeking conversations automatically derived from Wikipedia
  and tackles key challenges such as open-domain coverage, knowledge intensity,
  free-form responses, and topic shifts. It supports three core tasks of
  conversational RAG: passage retrieval, response generation, and citation
  labeling. We propose a unified framework to standardize various conversational
  RAG methods and conduct a comprehensive evaluation of these methods on CORAL,
  demonstrating substantial opportunities for improving existing approaches.
paper_ai_keywords: Retrieval-Augmented Generation (RAG) large language models (LLMs) single-turn RAG multi-turn conversations large-scale benchmark information-seeking conversations Wikipedia open-domain coverage knowledge intensity free-form responses topic shifts passage retrieval response generation citation labeling conversational RAG unified framework