Efficient Multi-Task Reinforcement Learning via Task-Specific Action Correction
<|reference_start|>Efficient Multi-Task Reinforcement Learning via Task-Specific Action Correction: Multi-task reinforcement learning (MTRL) demonstrate potential for enhancing the generalization of a robot, enabling it to perform multiple tasks concurrently. However, the performance of MTRL may still be susceptible to conflicts between tasks and negative interference. To facilitate efficient MTRL, we propose Task-Specific Action Correction (TSAC), a general and complementary approach designed for simultaneous learning of multiple tasks. TSAC decomposes policy learning into two separate policies: a shared policy (SP) and an action correction policy (ACP). To alleviate conflicts resulting from excessive focus on specific tasks' details in SP, ACP incorporates goal-oriented sparse rewards, enabling an agent to adopt a long-term perspective and achieve generalization across tasks. Additional rewards transform the original problem into a multi-objective MTRL problem. Furthermore, to convert the multi-objective MTRL into a single-objective formulation, TSAC assigns a virtual expected budget to the sparse rewards and employs Lagrangian method to transform a constrained single-objective optimization into an unconstrained one. Experimental evaluations conducted on Meta-World's MT10 and MT50 benchmarks demonstrate that TSAC outperforms existing state-of-the-art methods, achieving significant improvements in both sample efficiency and effective action execution.<|reference_end|>
@article{feng2024efficient,
    title={Efficient Multi-Task Reinforcement Learning via Task-Specific Action
  Correction},
    author={Jinyuan Feng, Min Chen, Zhiqiang Pu, Tenghai Qiu, Jianqiang Yi},
    journal={arXiv preprint arXiv:2404.05950},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.05950},
    primaryClass={cs.LG cs.AI cs.RO}
}