An Accelerated Multi-level Monte Carlo Approach for Average Reward Reinforcement Learning with General Policy Parametrization
<|reference_start|>An Accelerated Multi-level Monte Carlo Approach for Average Reward Reinforcement Learning with General Policy Parametrization: In our study, we delve into average-reward reinforcement learning with general policy parametrization. Within this domain, current guarantees either fall short with suboptimal guarantees or demand prior knowledge of mixing time. To address these issues, we introduce Randomized Accelerated Natural Actor Critic, a method that integrates Multi-level Monte-Carlo and Natural Actor Critic. Our approach is the first to achieve global convergence rate of $\tilde{\mathcal{O}}(1/\sqrt{T})$ without requiring knowledge of mixing time, significantly surpassing the state-of-the-art bound of $\tilde{\mathcal{O}}(1/T^{1/4})$.<|reference_end|>
@article{ganesh2024order-optimal,
    title={Order-Optimal Global Convergence for Average Reward Reinforcement
  Learning via Actor-Critic Approach},
    author={Swetha Ganesh, Washim Uddin Mondal and Vaneet Aggarwal},
    journal={arXiv preprint arXiv:2407.18878},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.18878},
    primaryClass={cs.LG}
}