title: ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG
    Capabilities
summary: In this work, we introduce ChatQA 2, a Llama3-based model designed to bridge
  the gap between open-access LLMs and leading proprietary models (e.g.,
  GPT-4-Turbo) in long-context understanding and retrieval-augmented generation
  (RAG) capabilities. These two capabilities are essential for LLMs to process
  large volumes of information that cannot fit into a single prompt and are
  complementary to each other, depending on the downstream tasks and
  computational budgets. We present a detailed continued training recipe to
  extend the context window of Llama3-70B-base from 8K to 128K tokens, along with
  a three-stage instruction tuning process to enhance the model's
  instruction-following, RAG performance, and long-context understanding
  capabilities. Our results demonstrate that the Llama3-ChatQA-2-70B model
  achieves accuracy comparable to GPT-4-Turbo-2024-0409 on many long-context
  understanding tasks and surpasses it on the RAG benchmark. Interestingly, we
  find that the state-of-the-art long-context retriever can alleviate the top-k
  context fragmentation issue in RAG, further improving RAG-based results for
  long-context understanding tasks. We also provide extensive comparisons between
  RAG and long-context solutions using state-of-the-art long-context LLMs.
paper_authors._id: 669dcf02c517d804cf9ad7b2
paper_authors.name: Peng Xu
paper_authors._id: 669dcf02c517d804cf9ad7b3
paper_authors.name: Wei Ping
paper_authors._id: 669dcf02c517d804cf9ad7b4
paper_authors.name: Xianchao Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-22T07:29:45.409Z
paper_authors._id: 62075297bcffa036eb483b2b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644647204919-62075297bcffa036eb483b2b.jpeg
paper_authors.fullname: Xianchao Wu
paper_authors.type: user
paper_authors.user: wuxianchao
paper_authors._id: 669dcf02c517d804cf9ad7b5
paper_authors.name: Zihan Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-22T07:30:15.794Z
paper_authors._id: 65f33b1c9f7970ccc0234cbf
paper_authors.avatarUrl: /avatars/99fbab303912e3674663251c04279907.svg
paper_authors.fullname: Zihan Liu
paper_authors.type: user
paper_authors.user: zihanliu
paper_authors._id: 669dcf02c517d804cf9ad7b6
paper_authors.name: Mohammad Shoeybi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-22T07:30:22.073Z
paper_authors._id: 6641544c695975af2cbd0da6
paper_authors.avatarUrl: /avatars/0ad3c18dcba585259b064fe9b00a07ce.svg
paper_authors.fullname: Mohammad Shoeybi
paper_authors.type: user
paper_authors.user: shoeybi
paper_authors._id: 669dcf02c517d804cf9ad7b7
paper_authors.name: Bryan Catanzaro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-22T07:30:29.898Z
paper_authors._id: 6311021788942700629e6247
paper_authors.avatarUrl: /avatars/e7adc1632b76e80e7e4a590033d1c20a.svg
paper_authors.fullname: Bryan Catanzaro
paper_authors.type: user
paper_authors.user: ctnzr
paper_title: ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG
    Capabilities
paper_summary: In this work, we introduce ChatQA 2, a Llama3-based model designed to bridge
  the gap between open-access LLMs and leading proprietary models (e.g.,
  GPT-4-Turbo) in long-context understanding and retrieval-augmented generation
  (RAG) capabilities. These two capabilities are essential for LLMs to process
  large volumes of information that cannot fit into a single prompt and are
  complementary to each other, depending on the downstream tasks and
  computational budgets. We present a detailed continued training recipe to
  extend the context window of Llama3-70B-base from 8K to 128K tokens, along with
  a three-stage instruction tuning process to enhance the model's
  instruction-following, RAG performance, and long-context understanding
  capabilities. Our results demonstrate that the Llama3-ChatQA-2-70B model
  achieves accuracy comparable to GPT-4-Turbo-2024-0409 on many long-context
  understanding tasks and surpasses it on the RAG benchmark. Interestingly, we
  find that the state-of-the-art long-context retriever can alleviate the top-k
  context fragmentation issue in RAG, further improving RAG-based results for
  long-context understanding tasks. We also provide extensive comparisons between
  RAG and long-context solutions using state-of-the-art long-context LLMs.
paper_ai_keywords: ChatQA 2 Llama3 LLMs GPT-4-Turbo long-context understanding retrieval-augmented generation (RAG) context window continued training recipe instruction tuning process instruction-following RAG performance long-context understanding accuracy GPT-4-Turbo-2024-0409 top-k context fragmentation state-of-the-art long-context retriever RAG-based results long-context solutions state-of-the-art long-context LLMs