Can Models Learn Skill Composition from Examples?
As large language models (LLMs) become increasingly advanced, their ability
to exhibit compositional generalization -- the capacity to combine learned
skills in novel ways not encountered during training -- has garnered
significant attention. This type of generalization, particularly in scenarios
beyond training data, is also of great interest in the study of AI safety and
alignment. A recent study introduced the SKILL-MIX evaluation, where models are
tasked with composing a short paragraph demonstrating the use of a specified
k-tuple of language skills. While small models struggled with composing even
with k=3, larger models like GPT-4 performed reasonably well with k=5 and
6.
  In this paper, we employ a setup akin to SKILL-MIX to evaluate the capacity
of smaller models to learn compositional generalization from examples.
Utilizing a diverse set of language skills -- including rhetorical, literary,
reasoning, theory of mind, and common sense -- GPT-4 was used to generate text
samples that exhibit random subsets of k skills. Subsequent fine-tuning of 7B
and 13B parameter models on these combined skill texts, for increasing values
of k, revealed the following findings: (1) Training on combinations of k=2
and 3 skills results in noticeable improvements in the ability to compose
texts with k=4 and 5 skills, despite models never having seen such examples
during training. (2) When skill categories are split into training and held-out
groups, models significantly improve at composing texts with held-out skills
during testing despite having only seen training skills during fine-tuning,
illustrating the efficacy of the training approach even with previously unseen
skills. This study also suggests that incorporating skill-rich (potentially
synthetic) text into training can substantially enhance the compositional
capabilities of models.
66fbaccbf49f515b65b48330
Haoyu Zhao
66fbaccbf49f515b65b48331
Simran Kaur
66fbaccbf49f515b65b48332
Dingli Yu
admin_assigned
2024-10-01T08:47:06.281Z
646bb050e96a751c5253955a
/avatars/8bf9c373e1b2b990998eb4643a71549b.svg
Dingli Yu
user
dingliyu
66fbaccbf49f515b65b48333
Anirudh Goyal
66fbaccbf49f515b65b48334
Sanjeev Arora
Can Models Learn Skill Composition from Examples?
As large language models (LLMs) become increasingly advanced, their ability
to exhibit compositional generalization -- the capacity to combine learned
skills in novel ways not encountered during training -- has garnered
significant attention. This type of generalization, particularly in scenarios
beyond training data, is also of great interest in the study of AI safety and
alignment. A recent study introduced the SKILL-MIX evaluation, where models are
tasked with composing a short paragraph demonstrating the use of a specified
k-tuple of language skills. While small models struggled with composing even
with k=3, larger models like GPT-4 performed reasonably well with k=5 and
6.
  In this paper, we employ a setup akin to SKILL-MIX to evaluate the capacity
of smaller models to learn compositional generalization from examples.
Utilizing a diverse set of language skills -- including rhetorical, literary,
reasoning, theory of mind, and common sense -- GPT-4 was used to generate text
samples that exhibit random subsets of k skills. Subsequent fine-tuning of 7B
and 13B parameter models on these combined skill texts, for increasing values
of k, revealed the following findings: (1) Training on combinations of k=2
and 3 skills results in noticeable improvements in the ability to compose
texts with k=4 and 5 skills, despite models never having seen such examples
during training. (2) When skill categories are split into training and held-out
groups, models significantly improve at composing texts with held-out skills
during testing despite having only seen training skills during fine-tuning,
illustrating the efficacy of the training approach even with previously unseen
skills. This study also suggests that incorporating skill-rich (potentially
synthetic) text into training can substantially enhance the compositional
capabilities of models.
Training smaller models on mixed language skills improves their ability to generalize to composing new combinations of skills not seen during training.
compositional generalization SKILL-MIX evaluation LLMs language skills rhetorical skills literary skills reasoning skills theory of mind common sense fine-tuning parameter-efficient fine-tuning