title: Steering Knowledge Selection Behaviours in LLMs via SAE-Based
    Representation Engineering
summary: Large language models (LLMs) can store a significant amount of factual
  knowledge in their parameters. However, their parametric knowledge may conflict
  with the information provided in the context -- this phenomenon, known as
  context-memory knowledge conflicts, can lead to undesirable model
  behaviour, such as reliance on outdated or incorrect information. Analysing the
  internal activations of LLMs, we find that they can internally register the
  signals of knowledge conflict at mid-layers. Such signals allow us to detect
  whether a knowledge conflict occurs and use inference-time intervention
  strategies to resolve it. In this work, we propose SpARE, a
  training-free representation engineering method that uses pre-trained
  sparse auto-encoders (SAEs) to control the knowledge selection behaviour of
  LLMs. SpARE identifies the functional features that control the
  knowledge selection behaviours and applies them to edit the internal
  activations of LLMs at inference time. Our experimental results show that
  SpARE can effectively control the usage of either knowledge source to
  resolve knowledge conflict in open-domain question-answering tasks, surpassing
  existing representation engineering methods (+10%) as well as contrastive
  decoding methods (+15%).
paper_authors._id: 671713db6914e88ea68a9ec6
paper_authors.name: Yu Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-23T07:34:48.086Z
paper_authors._id: 654a500d9b8bd6406d431c0d
paper_authors.avatarUrl: /avatars/a6b76441bbc6f4b71d49c52e454c9ef7.svg
paper_authors.fullname: Yu Zhao
paper_authors.type: user
paper_authors.user: yuzhaouoe
paper_authors._id: 671713db6914e88ea68a9ec7
paper_authors.name: Alessio Devoto
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-30T09:55:28.080Z
paper_authors._id: 62180288d05d07ebf7de5f31
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62180288d05d07ebf7de5f31/io2RsaRSjIDktQylXVvEB.jpeg
paper_authors.fullname: Alessio
paper_authors.type: user
paper_authors.user: alessiodevoto
paper_authors._id: 671713db6914e88ea68a9ec8
paper_authors.name: Giwon Hong
paper_authors._id: 671713db6914e88ea68a9ec9
paper_authors.name: Xiaotang Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:42:44.534Z
paper_authors._id: 63d4698d1da8c28b7b8ab04a
paper_authors.avatarUrl: /avatars/73d4f49018db03a59e72b69bb2d1dd0c.svg
paper_authors.fullname: Xiaotang Du
paper_authors.type: user
paper_authors.user: acDante
paper_authors._id: 671713db6914e88ea68a9eca
paper_authors.name: Aryo Pradipta Gema
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T11:25:25.780Z
paper_authors._id: 644f895e23d7eb05ca695054
paper_authors.avatarUrl: /avatars/3fb04dd8544b403262bf98507de05453.svg
paper_authors.fullname: Aryo Pradipta Gema
paper_authors.type: user
paper_authors.user: aryopg
paper_authors._id: 671713db6914e88ea68a9ecb
paper_authors.name: Hongru Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-30T09:55:26.414Z
paper_authors._id: 65f906e5c3dbdcae83ff7aac
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65f906e5c3dbdcae83ff7aac/mdjiVkLDJgJcGLwv0rMe4.jpeg
paper_authors.fullname: Hongru Wang
paper_authors.type: user
paper_authors.user: Merlin-Hongru
paper_authors._id: 671713db6914e88ea68a9ecc
paper_authors.name: Kam-Fai Wong
paper_authors._id: 671713db6914e88ea68a9ecd
paper_authors.name: Pasquale Minervini
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-30T09:55:24.894Z
paper_authors._id: 61001311e043e15c13412d30
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61001311e043e15c13412d30/6yAbTweYR16XtxMBEyOWl.png
paper_authors.fullname: Pasquale Minervini
paper_authors.type: user
paper_authors.user: pminervini
paper_title: Steering Knowledge Selection Behaviours in LLMs via SAE-Based
    Representation Engineering
paper_summary: Large language models (LLMs) can store a significant amount of factual
  knowledge in their parameters. However, their parametric knowledge may conflict
  with the information provided in the context -- this phenomenon, known as
  context-memory knowledge conflicts, can lead to undesirable model
  behaviour, such as reliance on outdated or incorrect information. Analysing the
  internal activations of LLMs, we find that they can internally register the
  signals of knowledge conflict at mid-layers. Such signals allow us to detect
  whether a knowledge conflict occurs and use inference-time intervention
  strategies to resolve it. In this work, we propose SpARE, a
  training-free representation engineering method that uses pre-trained
  sparse auto-encoders (SAEs) to control the knowledge selection behaviour of
  LLMs. SpARE identifies the functional features that control the
  knowledge selection behaviours and applies them to edit the internal
  activations of LLMs at inference time. Our experimental results show that
  SpARE can effectively control the usage of either knowledge source to
  resolve knowledge conflict in open-domain question-answering tasks, surpassing
  existing representation engineering methods (+10%) as well as contrastive
  decoding methods (+15%).
paper_ai_summary: A training-free method called SpARE uses pre-trained sparse auto-encoders to control knowledge selection in LLMs, effectively resolving context-memory knowledge conflicts in open-domain QA tasks.
paper_ai_keywords: LLMs parametric knowledge context-memory knowledge conflicts internal activations inference-time intervention SpARE sparse auto-encoders knowledge selection behaviour open-domain question-answering tasks contrastive decoding