Recurrent Natural Policy Gradient for POMDPs
<|reference_start|>Recurrent Natural Policy Gradient for POMDPs: In this paper, we study a natural policy gradient method based on recurrent neural networks (RNNs) for partially-observable Markov decision processes, whereby RNNs are used for policy parameterization and policy evaluation to address curse of dimensionality in non-Markovian reinforcement learning. We present finite-time and finite-width analyses for both the critic (recurrent temporal difference learning), and correspondingly-operated recurrent natural policy gradient method in the near-initialization regime. Our analysis demonstrates the efficiency of RNNs for problems with short-term memory with explicit bounds on the required network widths and sample complexity, and points out the challenges in the case of long-term dependencies.<|reference_end|>
@article{cayci2024recurrent,
    title={Recurrent Natural Policy Gradient for POMDPs},
    author={Semih Cayci, Atilla Eryilmaz},
    journal={arXiv preprint arXiv:2405.18221},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.18221},
    primaryClass={math.OC cs.LG stat.ML}
}