Deep Reinforcement Learning Behavioral Mode Switching Using Optimal Control Based on a Latent Space Objective
<|reference_start|>Deep Reinforcement Learning Behavioral Mode Switching Using Optimal Control Based on a Latent Space Objective: In this work, we use optimal control to change the behavior of a deep reinforcement learning policy by optimizing directly in the policy's latent space. We hypothesize that distinct behavioral patterns, termed behavioral modes, can be identified within certain regions of a deep reinforcement learning policy's latent space, meaning that specific actions or strategies are preferred within these regions. We identify these behavioral modes using latent space dimension-reduction with \ac*{pacmap}. Using the actions generated by the optimal control procedure, we move the system from one behavioral mode to another. We subsequently utilize these actions as a filter for interpreting the neural network policy. The results show that this approach can impose desired behavioral modes in the policy, demonstrated by showing how a failed episode can be made successful and vice versa using the lunar lander reinforcement learning environment.<|reference_end|>
@article{remman2024deep,
    title={Deep Reinforcement Learning Behavioral Mode Switching Using Optimal
  Control Based on a Latent Space Objective},
    author={Sindre Benjamin Remman, Bj{o}rn Andreas Kristiansen and Anastasios M.
  Lekkas},
    journal={arXiv preprint arXiv:2406.01178},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.01178},
    primaryClass={cs.LG cs.SY eess.SY}
}