Multi-Objective Deep Reinforcement Learning for Optimisation in Autonomous Systems
<|reference_start|>Multi-Objective Deep Reinforcement Learning for Optimisation in Autonomous Systems: Reinforcement Learning (RL) is used extensively in Autonomous Systems (AS) as it enables learning at runtime without the need for a model of the environment or predefined actions. However, most applications of RL in AS, such as those based on Q-learning, can only optimize one objective, making it necessary in multi-objective systems to combine multiple objectives in a single objective function with predefined weights. A number of Multi-Objective Reinforcement Learning (MORL) techniques exist but they have mostly been applied in RL benchmarks rather than real-world AS systems. In this work, we use a MORL technique called Deep W-Learning (DWN) and apply it to the Emergent Web Servers exemplar, a self-adaptive server, to find the optimal configuration for runtime performance optimization. We compare DWN to two single-objective optimization implementations: {\epsilon}-greedy algorithm and Deep Q-Networks. Our initial evaluation shows that DWN optimizes multiple objectives simultaneously with similar results than DQN and {\epsilon}-greedy approaches, having a better performance for some metrics, and avoids issues associated with combining multiple objectives into a single utility function.<|reference_end|>
@article{rosero2024multi-objective,
    title={Multi-Objective Deep Reinforcement Learning for Optimisation in
  Autonomous Systems},
    author={Juan C. Rosero, Ivana Dusparic, Nicol'as Cardozo},
    journal={arXiv preprint arXiv:2408.01188},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.01188},
    primaryClass={cs.AI}
}