Discovering Influential Neuron Path in Vision Transformers
Vision Transformer models exhibit immense power yet remain opaque to human
understanding, posing challenges and risks for practical applications. While
prior research has attempted to demystify these models through input
attribution and neuron role analysis, there's been a notable gap in considering
layer-level information and the holistic path of information flow across
layers. In this paper, we investigate the significance of influential neuron
paths within vision Transformers, which is a path of neurons from the model
input to output that impacts the model inference most significantly. We first
propose a joint influence measure to assess the contribution of a set of
neurons to the model outcome. And we further provide a layer-progressive neuron
locating approach that efficiently selects the most influential neuron at each
layer trying to discover the crucial neuron path from input to output within
the target model. Our experiments demonstrate the superiority of our method
finding the most influential neuron path along which the information flows,
over the existing baseline solutions. Additionally, the neuron paths have
illustrated that vision Transformers exhibit some specific inner working
mechanism for processing the visual information within the same image category.
We further analyze the key effects of these neurons on the image classification
task, showcasing that the found neuron paths have already preserved the model
capability on downstream tasks, which may also shed some lights on real-world
applications like model pruning. The project website including implementation
code is available at https://foundation-model-research.github.io/NeuronPath/.
67d31012e19218b9780513e9
Yifan Wang
claimed_verified
2025-03-14T08:56:58.691Z
67d30f6efcfb51018f9dd14c
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/9JDoVovUqeiPRAjHlWqYF.png
Yifan Wang
user
Zc0in
67d31012e19218b9780513ea
Yifei Liu
67d31012e19218b9780513eb
Yingdong Shi
claimed_verified
2025-04-08T06:57:43.747Z
64794594ae0fa9dc23371eeb
/avatars/ce759abfa065735c15b48b65a0c029fd.svg
yingdong shi
user
jonathanShi
67d31012e19218b9780513ec
Changming Li
67d31012e19218b9780513ed
Anqi Pang
67d31012e19218b9780513ee
Sibei Yang
67d31012e19218b9780513ef
Jingyi Yu
67d31012e19218b9780513f0
Kan Ren
Discovering Influential Neuron Path in Vision Transformers
Vision Transformer models exhibit immense power yet remain opaque to human
understanding, posing challenges and risks for practical applications. While
prior research has attempted to demystify these models through input
attribution and neuron role analysis, there's been a notable gap in considering
layer-level information and the holistic path of information flow across
layers. In this paper, we investigate the significance of influential neuron
paths within vision Transformers, which is a path of neurons from the model
input to output that impacts the model inference most significantly. We first
propose a joint influence measure to assess the contribution of a set of
neurons to the model outcome. And we further provide a layer-progressive neuron
locating approach that efficiently selects the most influential neuron at each
layer trying to discover the crucial neuron path from input to output within
the target model. Our experiments demonstrate the superiority of our method
finding the most influential neuron path along which the information flows,
over the existing baseline solutions. Additionally, the neuron paths have
illustrated that vision Transformers exhibit some specific inner working
mechanism for processing the visual information within the same image category.
We further analyze the key effects of these neurons on the image classification
task, showcasing that the found neuron paths have already preserved the model
capability on downstream tasks, which may also shed some lights on real-world
applications like model pruning. The project website including implementation
code is available at https://foundation-model-research.github.io/NeuronPath/.
The research investigates influential neuron paths in Vision Transformer models to enhance understanding of their inner workings and preserve model capability in downstream tasks.
Vision Transformer input attribution neuron role analysis layer-level information neuron paths joint influence measure layer-progressive neuron locating information flow image classification model pruning