Birth of a Transformer: A Memory Viewpoint
Large language models based on transformers have achieved great empirical
successes. However, as they are deployed more widely, there is a growing need
to better understand their internal mechanisms in order to make them more
reliable. These models appear to store vast amounts of knowledge from their
training data, and to adapt quickly to new information provided in their
context or prompt. We study how transformers balance these two types of
knowledge by considering a synthetic setup where tokens are generated from
either global or context-specific bigram distributions. By a careful empirical
analysis of the training process on a simplified two-layer transformer, we
illustrate the fast learning of global bigrams and the slower development of an
"induction head" mechanism for the in-context bigrams. We highlight the role of
weight matrices as associative memories, provide theoretical insights on how
gradients enable their learning during training, and study the role of
data-distributional properties.
647972e8e504380e5c8988e8
Alberto Bietti
647972e8e504380e5c8988e9
Vivien Cabannes
647972e8e504380e5c8988ea
Diane Bouchacourt
647972e8e504380e5c8988eb
Herve Jegou
647972e8e504380e5c8988ec
Leon Bottou
Birth of a Transformer: A Memory Viewpoint
Large language models based on transformers have achieved great empirical
successes. However, as they are deployed more widely, there is a growing need
to better understand their internal mechanisms in order to make them more
reliable. These models appear to store vast amounts of knowledge from their
training data, and to adapt quickly to new information provided in their
context or prompt. We study how transformers balance these two types of
knowledge by considering a synthetic setup where tokens are generated from
either global or context-specific bigram distributions. By a careful empirical
analysis of the training process on a simplified two-layer transformer, we
illustrate the fast learning of global bigrams and the slower development of an
"induction head" mechanism for the in-context bigrams. We highlight the role of
weight matrices as associative memories, provide theoretical insights on how
gradients enable their learning during training, and study the role of
data-distributional properties.
Transformers exhibit fast learning of global information and slower adaptation to context-specific details through the induction head mechanism, with weight matrices functioning as associative memories.
transformers synthetic setup global bigrams context-specific bigrams induction head mechanism weight matrices associative memories gradients