title: LLM Circuit Analyses Are Consistent Across Training and Scale
summary: Most currently deployed large language models (LLMs) undergo continuous
  training or additional finetuning. By contrast, most research into LLMs'
  internal mechanisms focuses on models at one snapshot in time (the end of
  pre-training), raising the question of whether their results generalize to
  real-world settings. Existing studies of mechanisms over time focus on
  encoder-only or toy models, which differ significantly from most deployed
  models. In this study, we track how model mechanisms, operationalized as
  circuits, emerge and evolve across 300 billion tokens of training in
  decoder-only LLMs, in models ranging from 70 million to 2.8 billion parameters.
  We find that task abilities and the functional components that support them
  emerge consistently at similar token counts across scale. Moreover, although
  such components may be implemented by different attention heads over time, the
  overarching algorithm that they implement remains. Surprisingly, both these
  algorithms and the types of components involved therein can replicate across
  model scale. These results suggest that circuit analyses conducted on small
  models at the end of pre-training can provide insights that still apply after
  additional pre-training and over model scale.
paper_authors._id: 6696183c2dcf962073b57c65
paper_authors.name: Curt Tigges
paper_authors._id: 6696183c2dcf962073b57c66
paper_authors.name: Michael Hanna
paper_authors._id: 6696183c2dcf962073b57c67
paper_authors.name: Qinan Yu
paper_authors._id: 6696183c2dcf962073b57c68
paper_authors.name: Stella Biderman
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-16T06:50:37.147Z
paper_authors._id: 60347d3660e3dd96631c9093
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
paper_authors.fullname: Stella Biderman
paper_authors.type: user
paper_authors.user: stellaathena
paper_title: LLM Circuit Analyses Are Consistent Across Training and Scale
paper_summary: Most currently deployed large language models (LLMs) undergo continuous
  training or additional finetuning. By contrast, most research into LLMs'
  internal mechanisms focuses on models at one snapshot in time (the end of
  pre-training), raising the question of whether their results generalize to
  real-world settings. Existing studies of mechanisms over time focus on
  encoder-only or toy models, which differ significantly from most deployed
  models. In this study, we track how model mechanisms, operationalized as
  circuits, emerge and evolve across 300 billion tokens of training in
  decoder-only LLMs, in models ranging from 70 million to 2.8 billion parameters.
  We find that task abilities and the functional components that support them
  emerge consistently at similar token counts across scale. Moreover, although
  such components may be implemented by different attention heads over time, the
  overarching algorithm that they implement remains. Surprisingly, both these
  algorithms and the types of components involved therein can replicate across
  model scale. These results suggest that circuit analyses conducted on small
  models at the end of pre-training can provide insights that still apply after
  additional pre-training and over model scale.
paper_ai_summary: Research examines the emergence and evolution of mechanisms or 'circuits' in decoder-only large language models during continuous training, finding consistent patterns across model scales and additional pre-training phases.
paper_ai_keywords: large language models LLMs continuous training finetuning encoder-only models decoder-only LLMs circuits token counts attention heads