title: From RAG to Memory: Non-Parametric Continual Learning for Large Language
    Models
summary: Our ability to continuously acquire, organize, and leverage knowledge is a
  key feature of human intelligence that AI systems must approximate to unlock
  their full potential. Given the challenges in continual learning with large
  language models (LLMs), retrieval-augmented generation (RAG) has become the
  dominant way to introduce new information. However, its reliance on vector
  retrieval hinders its ability to mimic the dynamic and interconnected nature of
  human long-term memory. Recent RAG approaches augment vector embeddings with
  various structures like knowledge graphs to address some of these gaps, namely
  sense-making and associativity. However, their performance on more basic
  factual memory tasks drops considerably below standard RAG. We address this
  unintended deterioration and propose HippoRAG 2, a framework that outperforms
  standard RAG comprehensively on factual, sense-making, and associative memory
  tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in
  HippoRAG and enhances it with deeper passage integration and more effective
  online use of an LLM. This combination pushes this RAG system closer to the
  effectiveness of human long-term memory, achieving a 7% improvement in
  associative memory tasks over the state-of-the-art embedding model while also
  exhibiting superior factual knowledge and sense-making memory capabilities.
  This work paves the way for non-parametric continual learning for LLMs. Our
  code and data will be released at https://github.com/OSU-NLP-Group/HippoRAG.
paper_authors._id: 67b878b8f17ca6989fd21e92
paper_authors.name: Bernal Jiménez Gutiérrez
paper_authors._id: 67b878b8f17ca6989fd21e93
paper_authors.name: Yiheng Shu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T15:15:54.523Z
paper_authors._id: 60a4ebfbaa9320dbbe69e37c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60a4ebfbaa9320dbbe69e37c/QLaEohXCWaUy8YX3wKQ_w.jpeg
paper_authors.fullname: Yiheng Shu
paper_authors.type: user
paper_authors.user: yhshu
paper_authors._id: 67b878b8f17ca6989fd21e94
paper_authors.name: Weijian Qi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T09:21:41.970Z
paper_authors._id: 67169e3fd720d7d51e36a67e
paper_authors.avatarUrl: /avatars/91113e5520009a4ed709bc62b96f2150.svg
paper_authors.fullname: WeijianQi
paper_authors.type: user
paper_authors.user: WeijianQi1999
paper_authors._id: 67b878b8f17ca6989fd21e95
paper_authors.name: Sizhe Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T15:16:08.161Z
paper_authors._id: 65030fc90a57e8f2b26bcaa3
paper_authors.avatarUrl: /avatars/8c25a4a9735f268ee8541f3e2017d92c.svg
paper_authors.fullname: Sizhe Zhou
paper_authors.type: user
paper_authors.user: KevinSRR
paper_authors._id: 67b878b8f17ca6989fd21e96
paper_authors.name: Yu Su
paper_title: From RAG to Memory: Non-Parametric Continual Learning for Large Language
    Models
paper_summary: Our ability to continuously acquire, organize, and leverage knowledge is a
  key feature of human intelligence that AI systems must approximate to unlock
  their full potential. Given the challenges in continual learning with large
  language models (LLMs), retrieval-augmented generation (RAG) has become the
  dominant way to introduce new information. However, its reliance on vector
  retrieval hinders its ability to mimic the dynamic and interconnected nature of
  human long-term memory. Recent RAG approaches augment vector embeddings with
  various structures like knowledge graphs to address some of these gaps, namely
  sense-making and associativity. However, their performance on more basic
  factual memory tasks drops considerably below standard RAG. We address this
  unintended deterioration and propose HippoRAG 2, a framework that outperforms
  standard RAG comprehensively on factual, sense-making, and associative memory
  tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in
  HippoRAG and enhances it with deeper passage integration and more effective
  online use of an LLM. This combination pushes this RAG system closer to the
  effectiveness of human long-term memory, achieving a 7% improvement in
  associative memory tasks over the state-of-the-art embedding model while also
  exhibiting superior factual knowledge and sense-making memory capabilities.
  This work paves the way for non-parametric continual learning for LLMs. Our
  code and data will be released at https://github.com/OSU-NLP-Group/HippoRAG.
paper_ai_summary: HippoRAG 2 enhances RAG by integrating passage information and leveraging a more effective LLM, improving performances across factual, sense-making, and associative memory tasks.
paper_ai_keywords: retrieval-augmented generation (RAG) vector retrieval knowledge graphs Personalized PageRank passage integration non-parametric continual learning large language models (LLMs)