Skill Expansion and Composition in Parameter Space
Humans excel at reusing prior knowledge to address new challenges and
developing skills while solving problems. This paradigm becomes increasingly
popular in the development of autonomous agents, as it develops systems that
can self-evolve in response to new challenges like human beings. However,
previous methods suffer from limited training efficiency when expanding new
skills and fail to fully leverage prior knowledge to facilitate new task
learning. In this paper, we propose Parametric Skill Expansion and Composition
(PSEC), a new framework designed to iteratively evolve the agents' capabilities
and efficiently address new challenges by maintaining a manageable skill
library. This library can progressively integrate skill primitives as
plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient
finetuning, facilitating efficient and flexible skill expansion. This structure
also enables the direct skill compositions in parameter space by merging LoRA
modules that encode different skills, leveraging shared information across
skills to effectively program new skills. Based on this, we propose a
context-aware module to dynamically activate different skills to
collaboratively handle new tasks. Empowering diverse applications including
multi-objective composition, dynamics shift, and continual policy shift, the
results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC
exhibits superior capacity to leverage prior knowledge to efficiently tackle
new challenges, as well as expand its skill libraries to evolve the
capabilities. Project website: https://ltlhuuu.github.io/PSEC/.
67ac4356401012b81050022a
Tenglong Liu
claimed_verified
2025-02-12T09:15:09.627Z
67ac430c4ab9207cc227d23f
/avatars/59c499cc191e28a66ae917963c28ffb3.svg
Tenglong Liu
user
LTL07
67ac4356401012b81050022b
Jianxiong Li
67ac4356401012b81050022c
Yinan Zheng
67ac4356401012b81050022d
Haoyi Niu
67ac4356401012b81050022e
Yixing Lan
67ac4356401012b81050022f
Xin Xu
67ac4356401012b810500230
Xianyuan Zhan
Skill Expansion and Composition in Parameter Space
Humans excel at reusing prior knowledge to address new challenges and
developing skills while solving problems. This paradigm becomes increasingly
popular in the development of autonomous agents, as it develops systems that
can self-evolve in response to new challenges like human beings. However,
previous methods suffer from limited training efficiency when expanding new
skills and fail to fully leverage prior knowledge to facilitate new task
learning. In this paper, we propose Parametric Skill Expansion and Composition
(PSEC), a new framework designed to iteratively evolve the agents' capabilities
and efficiently address new challenges by maintaining a manageable skill
library. This library can progressively integrate skill primitives as
plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient
finetuning, facilitating efficient and flexible skill expansion. This structure
also enables the direct skill compositions in parameter space by merging LoRA
modules that encode different skills, leveraging shared information across
skills to effectively program new skills. Based on this, we propose a
context-aware module to dynamically activate different skills to
collaboratively handle new tasks. Empowering diverse applications including
multi-objective composition, dynamics shift, and continual policy shift, the
results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC
exhibits superior capacity to leverage prior knowledge to efficiently tackle
new challenges, as well as expand its skill libraries to evolve the
capabilities. Project website: https://ltlhuuu.github.io/PSEC/.
PSEC facilitates autonomous agents' skill expansion and task handling through a parametric approach using Low-Rank Adaptation and skill composition, demonstrating superior performance in leveraging prior knowledge.
Parametric Skill Expansion and Composition PSEC Low-Rank Adaptation LoRA parameter-efficient finetuning skill composition skill library context-aware module D4RL DSRL benchmarks DeepMind Control Suite