title: Discovering Influential Neuron Path in Vision Transformers
summary: Vision Transformer models exhibit immense power yet remain opaque to human
  understanding, posing challenges and risks for practical applications. While
  prior research has attempted to demystify these models through input
  attribution and neuron role analysis, there's been a notable gap in considering
  layer-level information and the holistic path of information flow across
  layers. In this paper, we investigate the significance of influential neuron
  paths within vision Transformers, which is a path of neurons from the model
  input to output that impacts the model inference most significantly. We first
  propose a joint influence measure to assess the contribution of a set of
  neurons to the model outcome. And we further provide a layer-progressive neuron
  locating approach that efficiently selects the most influential neuron at each
  layer trying to discover the crucial neuron path from input to output within
  the target model. Our experiments demonstrate the superiority of our method
  finding the most influential neuron path along which the information flows,
  over the existing baseline solutions. Additionally, the neuron paths have
  illustrated that vision Transformers exhibit some specific inner working
  mechanism for processing the visual information within the same image category.
  We further analyze the key effects of these neurons on the image classification
  task, showcasing that the found neuron paths have already preserved the model
  capability on downstream tasks, which may also shed some lights on real-world
  applications like model pruning. The project website including implementation
  code is available at https://foundation-model-research.github.io/NeuronPath/.
paper_authors._id: 67d31012e19218b9780513e9
paper_authors.name: Yifan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:56:58.691Z
paper_authors._id: 67d30f6efcfb51018f9dd14c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/9JDoVovUqeiPRAjHlWqYF.png
paper_authors.fullname: Yifan Wang
paper_authors.type: user
paper_authors.user: Zc0in
paper_authors._id: 67d31012e19218b9780513ea
paper_authors.name: Yifei Liu
paper_authors._id: 67d31012e19218b9780513eb
paper_authors.name: Yingdong Shi
paper_authors._id: 67d31012e19218b9780513ec
paper_authors.name: Changming Li
paper_authors._id: 67d31012e19218b9780513ed
paper_authors.name: Anqi Pang
paper_authors._id: 67d31012e19218b9780513ee
paper_authors.name: Sibei Yang
paper_authors._id: 67d31012e19218b9780513ef
paper_authors.name: Jingyi Yu
paper_authors._id: 67d31012e19218b9780513f0
paper_authors.name: Kan Ren
paper_title: Discovering Influential Neuron Path in Vision Transformers
paper_summary: Vision Transformer models exhibit immense power yet remain opaque to human
  understanding, posing challenges and risks for practical applications. While
  prior research has attempted to demystify these models through input
  attribution and neuron role analysis, there's been a notable gap in considering
  layer-level information and the holistic path of information flow across
  layers. In this paper, we investigate the significance of influential neuron
  paths within vision Transformers, which is a path of neurons from the model
  input to output that impacts the model inference most significantly. We first
  propose a joint influence measure to assess the contribution of a set of
  neurons to the model outcome. And we further provide a layer-progressive neuron
  locating approach that efficiently selects the most influential neuron at each
  layer trying to discover the crucial neuron path from input to output within
  the target model. Our experiments demonstrate the superiority of our method
  finding the most influential neuron path along which the information flows,
  over the existing baseline solutions. Additionally, the neuron paths have
  illustrated that vision Transformers exhibit some specific inner working
  mechanism for processing the visual information within the same image category.
  We further analyze the key effects of these neurons on the image classification
  task, showcasing that the found neuron paths have already preserved the model
  capability on downstream tasks, which may also shed some lights on real-world
  applications like model pruning. The project website including implementation
  code is available at https://foundation-model-research.github.io/NeuronPath/.
paper_ai_keywords: Vision Transformer models input attribution neuron role analysis layer-level information information flow influential neuron paths joint influence measure layer-progressive neuron locating approach model inference neuron paths inner working mechanism image classification task model pruning