LLM Circuit Analyses Are Consistent Across Training and Scale
Most currently deployed large language models (LLMs) undergo continuous
training or additional finetuning. By contrast, most research into LLMs'
internal mechanisms focuses on models at one snapshot in time (the end of
pre-training), raising the question of whether their results generalize to
real-world settings. Existing studies of mechanisms over time focus on
encoder-only or toy models, which differ significantly from most deployed
models. In this study, we track how model mechanisms, operationalized as
circuits, emerge and evolve across 300 billion tokens of training in
decoder-only LLMs, in models ranging from 70 million to 2.8 billion parameters.
We find that task abilities and the functional components that support them
emerge consistently at similar token counts across scale. Moreover, although
such components may be implemented by different attention heads over time, the
overarching algorithm that they implement remains. Surprisingly, both these
algorithms and the types of components involved therein can replicate across
model scale. These results suggest that circuit analyses conducted on small
models at the end of pre-training can provide insights that still apply after
additional pre-training and over model scale.
6696183c2dcf962073b57c65
Curt Tigges
6696183c2dcf962073b57c66
Michael Hanna
6696183c2dcf962073b57c67
Qinan Yu
6696183c2dcf962073b57c68
Stella Biderman
extracted_pending
2024-07-16T06:50:37.147Z
60347d3660e3dd96631c9093
https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
Stella Biderman
user
stellaathena
LLM Circuit Analyses Are Consistent Across Training and Scale
Most currently deployed large language models (LLMs) undergo continuous
training or additional finetuning. By contrast, most research into LLMs'
internal mechanisms focuses on models at one snapshot in time (the end of
pre-training), raising the question of whether their results generalize to
real-world settings. Existing studies of mechanisms over time focus on
encoder-only or toy models, which differ significantly from most deployed
models. In this study, we track how model mechanisms, operationalized as
circuits, emerge and evolve across 300 billion tokens of training in
decoder-only LLMs, in models ranging from 70 million to 2.8 billion parameters.
We find that task abilities and the functional components that support them
emerge consistently at similar token counts across scale. Moreover, although
such components may be implemented by different attention heads over time, the
overarching algorithm that they implement remains. Surprisingly, both these
algorithms and the types of components involved therein can replicate across
model scale. These results suggest that circuit analyses conducted on small
models at the end of pre-training can provide insights that still apply after
additional pre-training and over model scale.
Research examines the emergence and evolution of mechanisms or 'circuits' in decoder-only large language models during continuous training, finding consistent patterns across model scales and additional pre-training phases.
large language models LLMs continuous training finetuning encoder-only models decoder-only LLMs circuits token counts attention heads