Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts
<|reference_start|>Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts: Reinforcement learning (RL) is a powerful approach for acquiring a good-performing policy. However, learning diverse skills is challenging in RL due to the commonly used Gaussian policy parameterization. We propose \textbf{Di}verse \textbf{Skil}l \textbf{L}earning (Di-SkilL\footnote{Videos and code are available on the project webpage: \url{https://alrhub.github.io/di-skill-website/}}), an RL method for learning diverse skills using Mixture of Experts, where each expert formalizes a skill as a contextual motion primitive. Di-SkilL optimizes each expert and its associate context distribution to a maximum entropy objective that incentivizes learning diverse skills in similar contexts. The per-expert context distribution enables automatic curricula learning, allowing each expert to focus on its best-performing sub-region of the context space. To overcome hard discontinuities and multi-modalities without any prior knowledge of the environment's unknown context probability space, we leverage energy-based models to represent the per-expert context distributions and demonstrate how we can efficiently train them using the standard policy gradient objective. We show on challenging robot simulation tasks that Di-SkilL can learn diverse and performant skills.<|reference_end|>
@article{celik2024acquiring,
    title={Acquiring Diverse Skills using Curriculum Reinforcement Learning with
  Mixture of Experts},
    author={Onur Celik, Aleksandar Taranovic, Gerhard Neumann},
    journal={arXiv preprint arXiv:2403.06966},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.06966},
    primaryClass={cs.LG cs.RO}
}