title: Emergence of Abstractions: Concept Encoding and Decoding Mechanism for
    In-Context Learning in Transformers
summary: Humans distill complex experiences into fundamental abstractions that enable
  rapid learning and adaptation. Similarly, autoregressive transformers exhibit
  adaptive learning through in-context learning (ICL), which begs the question of
  how. In this paper, we propose concept encoding-decoding mechanism to
  explain ICL by studying how transformers form and use internal abstractions in
  their representations. On synthetic ICL tasks, we analyze the training dynamics
  of a small transformer and report the coupled emergence of concept encoding and
  decoding. As the model learns to encode different latent concepts (e.g.,
  ``Finding the first noun in a sentence.") into distinct, separable
  representations, it concureently builds conditional decoding algorithms and
  improve its ICL performance. We validate the existence of this mechanism across
  pretrained models of varying scales (Gemma-2 2B/9B/27B, Llama-3.1 8B/70B).
  Further, through mechanistic interventions and controlled finetuning, we
  demonstrate that the quality of concept encoding is causally related and
  predictive of ICL performance. Our empirical insights shed light into better
  understanding the success and failure modes of large language models via their
  representations.
paper_authors._id: 67623311673b665026a73ee9
paper_authors.name: Seungwook Han
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T12:51:02.455Z
paper_authors._id: 6520d6db2a16045c092b3b36
paper_authors.avatarUrl: /avatars/dab34f141a1aef39d00c789ff85e729f.svg
paper_authors.fullname: Seungwook Han
paper_authors.type: user
paper_authors.user: hanseungwook
paper_authors._id: 67623311673b665026a73eea
paper_authors.name: Jinyeop Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T12:51:08.296Z
paper_authors._id: 65e37f6890a87c0f26d345da
paper_authors.avatarUrl: /avatars/884f645b2adde7ff3fe8689ecd5a2535.svg
paper_authors.fullname: Jinyeop Song
paper_authors.type: user
paper_authors.user: Jinyeop
paper_authors._id: 67623311673b665026a73eeb
paper_authors.name: Jeff Gore
paper_authors._id: 67623311673b665026a73eec
paper_authors.name: Pulkit Agrawal
paper_title: Emergence of Abstractions: Concept Encoding and Decoding Mechanism for
    In-Context Learning in Transformers
paper_summary: Humans distill complex experiences into fundamental abstractions that enable
  rapid learning and adaptation. Similarly, autoregressive transformers exhibit
  adaptive learning through in-context learning (ICL), which begs the question of
  how. In this paper, we propose concept encoding-decoding mechanism to
  explain ICL by studying how transformers form and use internal abstractions in
  their representations. On synthetic ICL tasks, we analyze the training dynamics
  of a small transformer and report the coupled emergence of concept encoding and
  decoding. As the model learns to encode different latent concepts (e.g.,
  ``Finding the first noun in a sentence.") into distinct, separable
  representations, it concureently builds conditional decoding algorithms and
  improve its ICL performance. We validate the existence of this mechanism across
  pretrained models of varying scales (Gemma-2 2B/9B/27B, Llama-3.1 8B/70B).
  Further, through mechanistic interventions and controlled finetuning, we
  demonstrate that the quality of concept encoding is causally related and
  predictive of ICL performance. Our empirical insights shed light into better
  understanding the success and failure modes of large language models via their
  representations.
paper_ai_summary: Transformers improve in-context learning through a concept encoding-decoding mechanism, forming and utilizing internal abstractions in their representations.
paper_ai_keywords: autoregressive transformers in-context learning concept encoding-decoding mechanism synthetic ICL tasks latent concepts conditional decoding pretrained models mechanistic interventions controlled finetuning large language models