Enhancing Policy Gradient with the Polyak Step-Size Adaption
<|reference_start|>Enhancing Policy Gradient with the Polyak Step-Size Adaption: Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other RL algorithms, its practical application is often hindered by sensitivity to hyper-parameters, particularly the step-size. In this paper, we introduce the integration of the Polyak step-size in RL, which automatically adjusts the step-size without prior knowledge. To adapt this method to RL settings, we address several issues, including unknown f* in the Polyak step-size. Additionally, we showcase the performance of the Polyak step-size in RL through experiments, demonstrating faster convergence and the attainment of more stable policies.<|reference_end|>
@article{li2024enhancing,
    title={Enhancing Policy Gradient with the Polyak Step-Size Adaption},
    author={Yunxiang Li, Rui Yuan, Chen Fan, Mark Schmidt, Samuel Horv'ath,
  Robert M. Gower, Martin Tak'av{c}},
    journal={arXiv preprint arXiv:2404.07525},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.07525},
    primaryClass={cs.LG}
}