title: Attention Heads of Large Language Models: A Survey
summary: Since the advent of ChatGPT, Large Language Models (LLMs) have excelled in
  various tasks but remain largely as black-box systems. Consequently, their
  development relies heavily on data-driven approaches, limiting performance
  enhancement through changes in internal architecture and reasoning pathways. As
  a result, many researchers have begun exploring the potential internal
  mechanisms of LLMs, aiming to identify the essence of their reasoning
  bottlenecks, with most studies focusing on attention heads. Our survey aims to
  shed light on the internal reasoning processes of LLMs by concentrating on the
  interpretability and underlying mechanisms of attention heads. We first distill
  the human thought process into a four-stage framework: Knowledge Recalling,
  In-Context Identification, Latent Reasoning, and Expression Preparation. Using
  this framework, we systematically review existing research to identify and
  categorize the functions of specific attention heads. Furthermore, we summarize
  the experimental methodologies used to discover these special heads, dividing
  them into two categories: Modeling-Free methods and Modeling-Required methods.
  Also, we outline relevant evaluation methods and benchmarks. Finally, we
  discuss the limitations of current research and propose several potential
  future directions. Our reference list is open-sourced at
  https://github.com/IAAR-Shanghai/Awesome-Attention-Heads.
paper_authors._id: 66da5f19247166c36547fd38
paper_authors.name: Zifan Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-06T07:06:25.188Z
paper_authors._id: 656f47ba2f058b368c0b1611
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/656f47ba2f058b368c0b1611/mrmcmA8bxaDNUhuJQQ7T1.png
paper_authors.fullname: Zifan Zheng
paper_authors.type: user
paper_authors.user: fan2goa1
paper_authors._id: 66da5f19247166c36547fd39
paper_authors.name: Yezhaohui Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T17:55:35.519Z
paper_authors._id: 662dd19f9e6d371ab71b91ce
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/662dd19f9e6d371ab71b91ce/mZBPw_Zs8ZlEFGlbekAoH.jpeg
paper_authors.fullname: Yezhaohui Wang
paper_authors.type: user
paper_authors.user: HaruTeru
paper_authors._id: 66da5f19247166c36547fd3a
paper_authors.name: Yuxin Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-06T09:07:22.127Z
paper_authors._id: 669e891c49f46ef87af2457a
paper_authors.avatarUrl: /avatars/2532729bb1cc9d52452ddb7852b41d72.svg
paper_authors.fullname: Yuxin Huang
paper_authors.type: user
paper_authors.user: saythe17
paper_authors._id: 66da5f19247166c36547fd3b
paper_authors.name: Shichao Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-06T07:06:23.258Z
paper_authors._id: 656f339a5273668d5b946b33
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/656f339a5273668d5b946b33/o2nBvQiOKKP5IfDmnpHP2.jpeg
paper_authors.fullname: Shichao Song
paper_authors.type: user
paper_authors.user: Ki-Seki
paper_authors._id: 66da5f19247166c36547fd3c
paper_authors.name: Bo Tang
paper_authors._id: 66da5f19247166c36547fd3d
paper_authors.name: Feiyu Xiong
paper_authors._id: 66da5f19247166c36547fd3e
paper_authors.name: Zhiyu Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-06T09:08:20.895Z
paper_authors._id: 62a155e615eeab266b2f2243
paper_authors.avatarUrl: /avatars/e89ef156e73af028e3ce3664e6cb4e62.svg
paper_authors.fullname: Zhiyu Li
paper_authors.type: user
paper_authors.user: jimi888
paper_title: Attention Heads of Large Language Models: A Survey
paper_summary: Since the advent of ChatGPT, Large Language Models (LLMs) have excelled in
  various tasks but remain largely as black-box systems. Consequently, their
  development relies heavily on data-driven approaches, limiting performance
  enhancement through changes in internal architecture and reasoning pathways. As
  a result, many researchers have begun exploring the potential internal
  mechanisms of LLMs, aiming to identify the essence of their reasoning
  bottlenecks, with most studies focusing on attention heads. Our survey aims to
  shed light on the internal reasoning processes of LLMs by concentrating on the
  interpretability and underlying mechanisms of attention heads. We first distill
  the human thought process into a four-stage framework: Knowledge Recalling,
  In-Context Identification, Latent Reasoning, and Expression Preparation. Using
  this framework, we systematically review existing research to identify and
  categorize the functions of specific attention heads. Furthermore, we summarize
  the experimental methodologies used to discover these special heads, dividing
  them into two categories: Modeling-Free methods and Modeling-Required methods.
  Also, we outline relevant evaluation methods and benchmarks. Finally, we
  discuss the limitations of current research and propose several potential
  future directions. Our reference list is open-sourced at
  https://github.com/IAAR-Shanghai/Awesome-Attention-Heads.
paper_ai_keywords: Large Language Models (LLMs) attention heads Knowledge Recalling In-Context Identification Latent Reasoning Expression Preparation Modeling-Free methods Modeling-Required methods