Variable Time Step Reinforcement Learning for Robotic Applications
<|reference_start|>Variable Time Step Reinforcement Learning for Robotic Applications: Traditional reinforcement learning (RL) generates discrete control policies,   assigning one action per cycle. These policies are usually implemented as in a   fixed-frequency control loop. This rigidity presents challenges as optimal   control frequency is task-dependent; suboptimal frequencies increase   computational demands and reduce exploration efficiency. Variable Time Step   Reinforcement Learning (VTS-RL) addresses these issues with adaptive control   frequencies, executing actions only when necessary, thus reducing   computational load and extending the action space to include action durations.   In this paper we introduce the Multi-Objective Soft Elastic Actor-Critic   (MOSEAC) method to perform VTS-RL, validating it through theoretical analysis   and experimentation in simulation and on real robots. Results show faster   convergence, better training results, and reduced energy consumption with   respect to other variable- or fixed-frequency approaches.<|reference_end|>
@article{wang2024variable,
    title={Variable Time Step Reinforcement Learning for Robotic Applications},
    author={Dong Wang and Giovanni Beltrame},
    journal={arXiv preprint arXiv:2407.00290},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.00290},
    primaryClass={cs.RO}
}