title: Skill Expansion and Composition in Parameter Space
summary: Humans excel at reusing prior knowledge to address new challenges and
  developing skills while solving problems. This paradigm becomes increasingly
  popular in the development of autonomous agents, as it develops systems that
  can self-evolve in response to new challenges like human beings. However,
  previous methods suffer from limited training efficiency when expanding new
  skills and fail to fully leverage prior knowledge to facilitate new task
  learning. In this paper, we propose Parametric Skill Expansion and Composition
  (PSEC), a new framework designed to iteratively evolve the agents' capabilities
  and efficiently address new challenges by maintaining a manageable skill
  library. This library can progressively integrate skill primitives as
  plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient
  finetuning, facilitating efficient and flexible skill expansion. This structure
  also enables the direct skill compositions in parameter space by merging LoRA
  modules that encode different skills, leveraging shared information across
  skills to effectively program new skills. Based on this, we propose a
  context-aware module to dynamically activate different skills to
  collaboratively handle new tasks. Empowering diverse applications including
  multi-objective composition, dynamics shift, and continual policy shift, the
  results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC
  exhibits superior capacity to leverage prior knowledge to efficiently tackle
  new challenges, as well as expand its skill libraries to evolve the
  capabilities. Project website: https://ltlhuuu.github.io/PSEC/.
paper_authors._id: 67ac4356401012b81050022a
paper_authors.name: Tenglong Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-12T09:15:09.627Z
paper_authors._id: 67ac430c4ab9207cc227d23f
paper_authors.avatarUrl: /avatars/59c499cc191e28a66ae917963c28ffb3.svg
paper_authors.fullname: Tenglong Liu
paper_authors.type: user
paper_authors.user: LTL07
paper_authors._id: 67ac4356401012b81050022b
paper_authors.name: Jianxiong Li
paper_authors._id: 67ac4356401012b81050022c
paper_authors.name: Yinan Zheng
paper_authors._id: 67ac4356401012b81050022d
paper_authors.name: Haoyi Niu
paper_authors._id: 67ac4356401012b81050022e
paper_authors.name: Yixing Lan
paper_authors._id: 67ac4356401012b81050022f
paper_authors.name: Xin Xu
paper_authors._id: 67ac4356401012b810500230
paper_authors.name: Xianyuan Zhan
paper_title: Skill Expansion and Composition in Parameter Space
paper_summary: Humans excel at reusing prior knowledge to address new challenges and
  developing skills while solving problems. This paradigm becomes increasingly
  popular in the development of autonomous agents, as it develops systems that
  can self-evolve in response to new challenges like human beings. However,
  previous methods suffer from limited training efficiency when expanding new
  skills and fail to fully leverage prior knowledge to facilitate new task
  learning. In this paper, we propose Parametric Skill Expansion and Composition
  (PSEC), a new framework designed to iteratively evolve the agents' capabilities
  and efficiently address new challenges by maintaining a manageable skill
  library. This library can progressively integrate skill primitives as
  plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient
  finetuning, facilitating efficient and flexible skill expansion. This structure
  also enables the direct skill compositions in parameter space by merging LoRA
  modules that encode different skills, leveraging shared information across
  skills to effectively program new skills. Based on this, we propose a
  context-aware module to dynamically activate different skills to
  collaboratively handle new tasks. Empowering diverse applications including
  multi-objective composition, dynamics shift, and continual policy shift, the
  results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC
  exhibits superior capacity to leverage prior knowledge to efficiently tackle
  new challenges, as well as expand its skill libraries to evolve the
  capabilities. Project website: https://ltlhuuu.github.io/PSEC/.
paper_ai_keywords: Parametric Skill Expansion and Composition (PSEC) LoRA (Low-Rank Adaptation) parameter-efficient fine-tuning skill primitives skill library parameter space context-aware module D4RL benchmark DSRL benchmarks DeepMind Control Suite