Emergence of Abstractions: Concept Encoding and Decoding Mechanism for
  In-Context Learning in Transformers
Humans distill complex experiences into fundamental abstractions that enable
rapid learning and adaptation. Similarly, autoregressive transformers exhibit
adaptive learning through in-context learning (ICL), which begs the question of
how. In this paper, we propose concept encoding-decoding mechanism to
explain ICL by studying how transformers form and use internal abstractions in
their representations. On synthetic ICL tasks, we analyze the training dynamics
of a small transformer and report the coupled emergence of concept encoding and
decoding. As the model learns to encode different latent concepts (e.g.,
``Finding the first noun in a sentence.") into distinct, separable
representations, it concureently builds conditional decoding algorithms and
improve its ICL performance. We validate the existence of this mechanism across
pretrained models of varying scales (Gemma-2 2B/9B/27B, Llama-3.1 8B/70B).
Further, through mechanistic interventions and controlled finetuning, we
demonstrate that the quality of concept encoding is causally related and
predictive of ICL performance. Our empirical insights shed light into better
understanding the success and failure modes of large language models via their
representations.
67623311673b665026a73ee9
Seungwook Han
admin_assigned
2024-12-18T12:51:02.455Z
6520d6db2a16045c092b3b36
/avatars/dab34f141a1aef39d00c789ff85e729f.svg
Seungwook Han
user
hanseungwook
67623311673b665026a73eea
Jinyeop Song
admin_assigned
2024-12-18T12:51:08.296Z
65e37f6890a87c0f26d345da
/avatars/884f645b2adde7ff3fe8689ecd5a2535.svg
Jinyeop Song
user
Jinyeop
67623311673b665026a73eeb
Jeff Gore
67623311673b665026a73eec
Pulkit Agrawal
Emergence of Abstractions: Concept Encoding and Decoding Mechanism for
  In-Context Learning in Transformers
Humans distill complex experiences into fundamental abstractions that enable
rapid learning and adaptation. Similarly, autoregressive transformers exhibit
adaptive learning through in-context learning (ICL), which begs the question of
how. In this paper, we propose concept encoding-decoding mechanism to
explain ICL by studying how transformers form and use internal abstractions in
their representations. On synthetic ICL tasks, we analyze the training dynamics
of a small transformer and report the coupled emergence of concept encoding and
decoding. As the model learns to encode different latent concepts (e.g.,
``Finding the first noun in a sentence.") into distinct, separable
representations, it concureently builds conditional decoding algorithms and
improve its ICL performance. We validate the existence of this mechanism across
pretrained models of varying scales (Gemma-2 2B/9B/27B, Llama-3.1 8B/70B).
Further, through mechanistic interventions and controlled finetuning, we
demonstrate that the quality of concept encoding is causally related and
predictive of ICL performance. Our empirical insights shed light into better
understanding the success and failure modes of large language models via their
representations.
Transformers improve in-context learning through a concept encoding-decoding mechanism, forming and utilizing internal abstractions in their representations.
autoregressive transformers in-context learning concept encoding-decoding mechanism synthetic ICL tasks latent concepts conditional decoding pretrained models mechanistic interventions controlled finetuning large language models