content.raw: 🔍 Today's pick in Interpretability & Analysis of LMs: Universal Neurons in GPT2 Language Models by 
content.type: text
content.value: 🔍 Today's pick in Interpretability & Analysis of LMs: Universal Neurons in GPT2 Language Models by 
content.raw: @wesg
content.type: mention
content.user: wesg
content.type: text
content.raw: @thorsley
content.type: mention
content.user: thorsley
content.type: text
content.raw: @luminolblue
content.type: mention
content.user: luminolblue
content.raw:  T.R. Kheirkhah Q. Sun, 
content.type: text
content.value:  T.R. Kheirkhah Q. Sun, 
content.raw: @willhath
content.type: mention
content.user: willhath
content.type: text
content.raw: @NeelNanda
content.type: mention
content.user: NeelNanda
content.raw:  D. Bertsimas
content.type: text
content.value:  D. Bertsimas
content.type: new_line
content.type: new_line
content.raw: This work investigates the universality of individual neurons across GPT2 models trained from different initial random seeds, starting from the assumption that such neurons are likely to exhibit interpretable patterns. The authors find 1-5% of neurons showing high correlation across five model seeds, i.e. consistently activating for the same inputs. In particular, those neurons can be grouped into families exhibiting similar functional roles, e.g. modulating the next token prediction entropy, controlling the output norm of an attention head, and promoting/suppressing vocabulary elements in the prediction. Finally, universal neurons are often observed to form antipodal pairs, conjectured to improve the robustness and calibration of model predictions via ensembling.
content.type: text
content.value: This work investigates the universality of individual neurons across GPT2 models trained from different initial random seeds, starting from the assumption that such neurons are likely to exhibit interpretable patterns. The authors find 1-5% of neurons showing high correlation across five model seeds, i.e. consistently activating for the same inputs. In particular, those neurons can be grouped into families exhibiting similar functional roles, e.g. modulating the next token prediction entropy, controlling the output norm of an attention head, and promoting/suppressing vocabulary elements in the prediction. Finally, universal neurons are often observed to form antipodal pairs, conjectured to improve the robustness and calibration of model predictions via ensembling.
content.type: new_line
content.type: new_line
content.raw: 📄 Paper: 
content.type: text
content.value: 📄 Paper: 
content.label: Universal Neurons in GPT2 Language Models (2401.12181)
content.raw: https://huggingface.co/papers/2401.12181
content.id: 2401.12181
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2401.12181
rawContent: 🔍 Today's pick in Interpretability & Analysis of LMs: Universal Neurons in GPT2 Language Models by @wesg @thorsley @luminolblue T.R. Kheirkhah Q. Sun, @willhath @NeelNanda D. Bertsimas
  
  This work investigates the universality of individual neurons across GPT2 models trained from different initial random seeds, starting from the assumption that such neurons are likely to exhibit interpretable patterns. The authors find 1-5% of neurons showing high correlation across five model seeds, i.e. consistently activating for the same inputs. In particular, those neurons can be grouped into families exhibiting similar functional roles, e.g. modulating the next token prediction entropy, controlling the output norm of an attention head, and promoting/suppressing vocabulary elements in the prediction. Finally, universal neurons are often observed to form antipodal pairs, conjectured to improve the robustness and calibration of model predictions via ensembling.
  
  📄 Paper: https://huggingface.co/papers/2401.12181
reactions.reaction: 👍
reactions.users: merve osanseviero wesg lunarflu abidlabs afrideva LiraMirui Aryanne sbrandeis
reactions.reaction: 🤗
reactions.users: s3nh merve osanseviero wesg clem abidlabs lunarflu