title: The Coverage Principle: A Framework for Understanding Compositional
    Generalization
summary: Large language models excel at pattern matching, yet often fall short in
  systematic compositional generalization. We propose the coverage principle: a
  data-centric framework showing that models relying primarily on pattern
  matching for compositional tasks cannot reliably generalize beyond substituting
  fragments that yield identical results when used in the same contexts. We
  demonstrate that this framework has a strong predictive power for the
  generalization capabilities of Transformers. First, we derive and empirically
  confirm that the training data required for two-hop generalization grows at
  least quadratically with the token set size, and the training data efficiency
  does not improve with 20x parameter scaling. Second, for compositional tasks
  with path ambiguity where one variable affects the output through multiple
  computational paths, we show that Transformers learn context-dependent state
  representations that undermine both performance and interoperability. Third,
  Chain-of-Thought supervision improves training data efficiency for multi-hop
  tasks but still struggles with path ambiguity. Finally, we outline a
  mechanism-based taxonomy that distinguishes three ways neural networks
  can generalize: structure-based (bounded by coverage), property-based
  (leveraging algebraic invariances), and shared-operator (through function
  reuse). This conceptual lens contextualizes our results and highlights where
  new architectural ideas are needed to achieve systematic compositionally.
  Overall, the coverage principle provides a unified lens for understanding
  compositional reasoning, and underscores the need for fundamental architectural
  or training innovations to achieve truly systematic compositionality.
paper_authors._id: 68353261bc28496925a185c9
paper_authors.name: Hoyeon Chang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:58:28.992Z
paper_authors._id: 64d0d6684dfd5df70744b237
paper_authors.avatarUrl: /avatars/4ea57bfd407e8cb727c624f64af75478.svg
paper_authors.fullname: Chang
paper_authors.type: user
paper_authors.user: Hoyeon
paper_authors._id: 68353261bc28496925a185ca
paper_authors.name: Jinho Park
paper_authors._id: 68353261bc28496925a185cb
paper_authors.name: Hanseul Cho
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:59:21.074Z
paper_authors._id: 6486e2e3c4799f43eaf3602c
paper_authors.avatarUrl: /avatars/cfd3055e1f2604d07475b31f33d43d52.svg
paper_authors.fullname: Hanseul Cho
paper_authors.type: user
paper_authors.user: HanseulJo
paper_authors._id: 68353261bc28496925a185cc
paper_authors.name: Sohee Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:59:14.789Z
paper_authors._id: 606ae0adb579bb0e88515311
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1617617060380-noauth.jpeg
paper_authors.fullname: Sohee Yang
paper_authors.type: user
paper_authors.user: soheeyang
paper_authors._id: 68353261bc28496925a185cd
paper_authors.name: Miyoung Ko
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:59:07.932Z
paper_authors._id: 6485d617097c4b9f75d350e9
paper_authors.avatarUrl: /avatars/8dfc0fa03a7d5846fd38014beafb0381.svg
paper_authors.fullname: Miyoung Ko
paper_authors.type: user
paper_authors.user: miyoungko
paper_authors._id: 68353261bc28496925a185ce
paper_authors.name: Hyeonbin Hwang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:59:02.515Z
paper_authors._id: 647eaaf61a1fcad2fdc5d1ef
paper_authors.avatarUrl: /avatars/596d7a61d6e6227d38b661210a32fed0.svg
paper_authors.fullname: Hyeonbin Hwang 
paper_authors.type: user
paper_authors.user: hbin0701
paper_authors._id: 68353261bc28496925a185cf
paper_authors.name: Seungpil Won
paper_authors._id: 68353261bc28496925a185d0
paper_authors.name: Dohaeng Lee
paper_authors._id: 68353261bc28496925a185d1
paper_authors.name: Youbin Ahn
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:58:47.652Z
paper_authors._id: 65386bbe6d1c2659e102bbbb
paper_authors.avatarUrl: /avatars/6ed42e65f64cd72c7b3615c24beedb01.svg
paper_authors.fullname: Youbin Ahn
paper_authors.type: user
paper_authors.user: youbina
paper_authors._id: 68353261bc28496925a185d2
paper_authors.name: Minjoon Seo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:58:41.960Z
paper_authors._id: 621f05ba970615ad5861ceb1
paper_authors.avatarUrl: /avatars/7e1902aa71369a524afda9b0a9e88e22.svg
paper_authors.fullname: Minjoon Seo
paper_authors.type: user
paper_authors.user: minjoon
paper_title: The Coverage Principle: A Framework for Understanding Compositional
    Generalization
paper_summary: Large language models excel at pattern matching, yet often fall short in
  systematic compositional generalization. We propose the coverage principle: a
  data-centric framework showing that models relying primarily on pattern
  matching for compositional tasks cannot reliably generalize beyond substituting
  fragments that yield identical results when used in the same contexts. We
  demonstrate that this framework has a strong predictive power for the
  generalization capabilities of Transformers. First, we derive and empirically
  confirm that the training data required for two-hop generalization grows at
  least quadratically with the token set size, and the training data efficiency
  does not improve with 20x parameter scaling. Second, for compositional tasks
  with path ambiguity where one variable affects the output through multiple
  computational paths, we show that Transformers learn context-dependent state
  representations that undermine both performance and interoperability. Third,
  Chain-of-Thought supervision improves training data efficiency for multi-hop
  tasks but still struggles with path ambiguity. Finally, we outline a
  mechanism-based taxonomy that distinguishes three ways neural networks
  can generalize: structure-based (bounded by coverage), property-based
  (leveraging algebraic invariances), and shared-operator (through function
  reuse). This conceptual lens contextualizes our results and highlights where
  new architectural ideas are needed to achieve systematic compositionally.
  Overall, the coverage principle provides a unified lens for understanding
  compositional reasoning, and underscores the need for fundamental architectural
  or training innovations to achieve truly systematic compositionality.
paper_ai_summary: The coverage principle highlights limitations in Transformers' compositional generalization, emphasizing the need for new architectures or training methods to achieve systematic compositionality by distinguishing different mechanisms of generalization.
paper_ai_keywords: coverage principle data-centric framework sequential application pattern matching compositional generalization Transformers two-hop generalization token set size training data efficiency context-dependent state representations performance interoperability Chain-of-Thought supervision multi-hop tasks path ambiguity structure-based property-based shared-operator architectural innovations