title: Steering Llama 2 via Contrastive Activation Addition
summary: We introduce Contrastive Activation Addition (CAA), an innovative method for
  steering language models by modifying activations during their forward passes.
  CAA computes ``steering vectors'' by averaging the difference in residual
  stream activations between pairs of positive and negative examples of a
  particular behavior such as factual versus hallucinatory responses. During
  inference, these steering vectors are added at all token positions after the
  user's prompt with either a positive or negative coefficient, allowing precise
  control over the degree of the targeted behavior. We evaluate CAA's
  effectiveness on Llama 2 Chat using both multiple-choice behavioral question
  datasets and open-ended generation tasks. We demonstrate that CAA significantly
  alters model behavior, outperforms traditional methods like finetuning and
  few-shot prompting, and minimally reduces capabilities. Moreover, by employing
  various activation space interpretation methods, we gain deeper insights into
  CAA's mechanisms. CAA both accurately steers model outputs and also sheds light
  on how high-level concepts are represented in Large Language Models (LLMs).
paper_authors._id: 657918cb7f5da1deb6a2e9b3
paper_authors.name: Nina Rimsky
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-12-13T02:39:58.373Z
paper_authors._id: 64b80fee5c4af63e10825753
paper_authors.avatarUrl: /avatars/68af35718e61097bd1f901d85121d96c.svg
paper_authors.fullname: Nina Panickssery
paper_authors.type: user
paper_authors.user: ninapanickssery
paper_authors._id: 657918cb7f5da1deb6a2e9b4
paper_authors.name: Nick Gabrieli
paper_authors._id: 657918cb7f5da1deb6a2e9b5
paper_authors.name: Julian Schulz
paper_authors._id: 657918cb7f5da1deb6a2e9b6
paper_authors.name: Meg Tong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T10:04:50.660Z
paper_authors._id: 63272a638624baac667c8bdb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63272a638624baac667c8bdb/ylZ-FNT9PLhn8sBCD1wQm.png
paper_authors.fullname: Meg Tong
paper_authors.type: user
paper_authors.user: meg-tong
paper_authors._id: 657918cb7f5da1deb6a2e9b7
paper_authors.name: Evan Hubinger
paper_authors._id: 657918cb7f5da1deb6a2e9b8
paper_authors.name: Alexander Matt Turner
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-12-20T00:18:40.223Z
paper_authors._id: 63d03d73623a3d1d11708951
paper_authors.avatarUrl: /avatars/254d82bb0172c3a0ee90c85379fd19bd.svg
paper_authors.fullname: Alex Turner
paper_authors.type: user
paper_authors.user: TurnTrout
paper_title: Steering Llama 2 via Contrastive Activation Addition
paper_summary: We introduce Contrastive Activation Addition (CAA), an innovative method for
  steering language models by modifying activations during their forward passes.
  CAA computes ``steering vectors'' by averaging the difference in residual
  stream activations between pairs of positive and negative examples of a
  particular behavior such as factual versus hallucinatory responses. During
  inference, these steering vectors are added at all token positions after the
  user's prompt with either a positive or negative coefficient, allowing precise
  control over the degree of the targeted behavior. We evaluate CAA's
  effectiveness on Llama 2 Chat using both multiple-choice behavioral question
  datasets and open-ended generation tasks. We demonstrate that CAA significantly
  alters model behavior, outperforms traditional methods like finetuning and
  few-shot prompting, and minimally reduces capabilities. Moreover, by employing
  various activation space interpretation methods, we gain deeper insights into
  CAA's mechanisms. CAA both accurately steers model outputs and also sheds light
  on how high-level concepts are represented in Large Language Models (LLMs).
paper_ai_keywords: Contrastive Activation Addition (CAA) steering vectors residual stream activations factual responses hallucinatory responses token positions user's prompt multiple-choice behavioral question datasets open-ended generation tasks traditional methods like finetuning and few-shot prompting activation space interpretation methods high-level concepts Large Language Models (LLMs)