Reinforcement Leaning for Infinite-Dimensional Systems
<|reference_start|>Reinforcement Leaning for Infinite-Dimensional Systems: Interest in reinforcement learning (RL) for massive-scale systems consisting of large populations of intelligent agents interacting with heterogeneous environments has witnessed a significant surge in recent years across diverse scientific domains. However, due to the large-scale nature of the system, the majority of state-of-the-art RL techniques either encounter high computational cost or exhibit compromised performance. To mitigate these challenges, we propose a novel RL architecture along with the derivation of effective algorithms to learn optimal policies for any arbitrarily large system of agents. Specifically, we model such a system as a parameterized control system defined on an infinite-dimensional function space. We then develop a moment kernel transform to map the parameterized system and the value function of an RL problem into a reproducing kernel Hilbert space. This transformation subsequently generates a finite-dimensional moment representation for this RL problem. Leveraging this representation, we develop a hierarchical algorithm for learning optimal policies for the infinite-dimensional parameterized system. We further enhance efficiency of the algorithm by exploiting early stopping at each hierarchy, by which we show the fast convergence property of the algorithm through constructing a convergent spectral sequence. The performance and efficiency of the proposed algorithm are validated using practical examples.<|reference_end|>
@article{zhang2024reinforcement,
    title={Reinforcement Leaning for Infinite-Dimensional Systems},
    author={Wei Zhang, Jr-Shin Li},
    journal={arXiv preprint arXiv:2409.15737},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.15737},
    primaryClass={eess.SY cs.SY math.OC}
}