title: Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
summary: In human-written articles, we often leverage the subtleties of text style,
  such as bold and italics, to guide the attention of readers. These textual
  emphases are vital for the readers to grasp the conveyed information. When
  interacting with large language models (LLMs), we have a similar need -
  steering the model to pay closer attention to user-specified information, e.g.,
  an instruction. Existing methods, however, are constrained to process plain
  text and do not support such a mechanism. This motivates us to introduce PASTA
  - Post-hoc Attention STeering Approach, a method that allows LLMs to read text
  with user-specified emphasis marks. To this end, PASTA identifies a small
  subset of attention heads and applies precise attention reweighting on them,
  directing the model attention to user-specified parts. Like prompting, PASTA is
  applied at inference time and does not require changing any model parameters.
  Experiments demonstrate that PASTA can substantially enhance an LLM's ability
  to follow user instructions or integrate new knowledge from user inputs,
  leading to a significant performance improvement on a variety of tasks, e.g.,
  an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly
  available at https://github.com/QingruZhang/PASTA .
paper_authors._id: 6549e8b0137b7246f0479cf1
paper_authors.name: Qingru Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-07T15:43:26.696Z
paper_authors._id: 64a76bf4a352114d1cbe0c3a
paper_authors.avatarUrl: /avatars/966a556d86760dd9fe244cacbed9415d.svg
paper_authors.fullname: Qingru Zhang
paper_authors.type: user
paper_authors.user: zhang4562
paper_authors._id: 6549e8b0137b7246f0479cf2
paper_authors.name: Chandan Singh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-07T17:12:43.236Z
paper_authors._id: 62b0bee117fa24819dd0999c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b0bee117fa24819dd0999c/eJ7ryFQ3_ZbqV29qf4KyM.jpeg
paper_authors.fullname: Chandan Singh
paper_authors.type: user
paper_authors.user: csinva
paper_authors._id: 6549e8b0137b7246f0479cf3
paper_authors.name: Liyuan Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-07T16:12:47.916Z
paper_authors._id: 6476372871d420a1f6be3650
paper_authors.avatarUrl: /avatars/ca3492d3967e06a7785457dd0ba2582c.svg
paper_authors.fullname: Liyuan Liu
paper_authors.type: user
paper_authors.user: LiyuanLucasLiu
paper_authors._id: 6549e8b0137b7246f0479cf4
paper_authors.name: Xiaodong Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-07T16:12:54.817Z
paper_authors._id: 64b6007dfc5a8ae2a7e761ca
paper_authors.avatarUrl: /avatars/7d59d9e19ea536ce388a83764e48867d.svg
paper_authors.fullname: xiaodong liu
paper_authors.type: user
paper_authors.user: ArivnLiu
paper_authors._id: 6549e8b0137b7246f0479cf5
paper_authors.name: Bin Yu
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-02-21T14:57:41.867Z
paper_authors._id: 64679498374fe5728d4044d5
paper_authors.avatarUrl: /avatars/78516e1c9532900cce19f81483855db8.svg
paper_authors.fullname: Bin Yu
paper_authors.type: user
paper_authors.user: binyu000
paper_authors._id: 6549e8b0137b7246f0479cf6
paper_authors.name: Jianfeng Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-07T16:13:01.145Z
paper_authors._id: 641904caf9d6f1d772ec7af7
paper_authors.avatarUrl: /avatars/4a63eac71eb30f70b1a0e9d4708f26c1.svg
paper_authors.fullname: Jianfeng Gao
paper_authors.type: user
paper_authors.user: wyngjf
paper_authors._id: 6549e8b0137b7246f0479cf7
paper_authors.name: Tuo Zhao
paper_title: Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
paper_summary: In human-written articles, we often leverage the subtleties of text style,
  such as bold and italics, to guide the attention of readers. These textual
  emphases are vital for the readers to grasp the conveyed information. When
  interacting with large language models (LLMs), we have a similar need -
  steering the model to pay closer attention to user-specified information, e.g.,
  an instruction. Existing methods, however, are constrained to process plain
  text and do not support such a mechanism. This motivates us to introduce PASTA
  - Post-hoc Attention STeering Approach, a method that allows LLMs to read text
  with user-specified emphasis marks. To this end, PASTA identifies a small
  subset of attention heads and applies precise attention reweighting on them,
  directing the model attention to user-specified parts. Like prompting, PASTA is
  applied at inference time and does not require changing any model parameters.
  Experiments demonstrate that PASTA can substantially enhance an LLM's ability
  to follow user instructions or integrate new knowledge from user inputs,
  leading to a significant performance improvement on a variety of tasks, e.g.,
  an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly
  available at https://github.com/QingruZhang/PASTA .
paper_ai_keywords: large language models (LLMs) attention heads attention reweighting prompting inference time performance improvement LLAMA-7B