Safety-Prioritizing Curricula for Constrained Reinforcement Learning
Cevahir Koprulu Thiago D. Simão Nils Jansen ufuk topcu
Curriculum learning aims to accelerate reinforcement learning (RL) by generating curricula, i.e., sequences of tasks of increasing difficulty. 
Although existing curriculum generation approaches provide benefits in sample efficiency, they overlook safety-critical settings where an RL agent must adhere to safety constraints.
Thus, these approaches may generate tasks that cause RL agents to violate safety constraints during training and behave suboptimally after. 
We develop a safe curriculum generation approach (SCG) that aligns the objectives of constrained RL and curriculum learning: improving safety during training and boosting sample efficiency.
SCG generates sequences of tasks where the RL agent can be safe and performant by initially generating tasks with minimum safety violations over high-reward ones.
We empirically show that compared to the state-of-the-art curriculum learning approaches and their naively modified safe versions, SCG achieves optimal performance and the lowest amount of constraint violations during training.
curriculum learning, constrained reinforcement learning
We propose a safe curriculum generation method that reduces safety constraint violations during training while boosting the learning speed of constrained RL agents.