Unified continuous-time q-learning for mean-field game and mean-field control problems
<|reference_start|>Unified continuous-time q-learning for mean-field game and mean-field control problems: This paper studies the continuous-time q-learning in the mean-field jump-diffusion models from the representative agent's perspective. To overcome the challenge when the population distribution may not be directly observable, we introduce the integrated q-function in decoupled form (decoupled Iq-function) and establish its martingale characterization together with the value function, which provides a unified policy evaluation rule for both mean-field game (MFG) and mean-field control (MFC) problems. Moreover, depending on the task to solve the MFG or MFC problem, we can employ the decoupled Iq-function by different means to learn the mean-field equilibrium policy or the mean-field optimal policy respectively. As a result, we devise a unified q-learning algorithm for both MFG and MFC problems by utilizing all test policies stemming from the mean-field interactions. For several examples in the jump-diffusion setting, within and beyond the LQ framework, we can obtain the exact parameterization of the decoupled Iq-functions and the value functions, and illustrate our algorithm from the representative agent's perspective with satisfactory performance.<|reference_end|>
@article{wei2024unified,
    title={Unified continuous-time q-learning for mean-field game and mean-field
  control problems},
    author={Xiaoli Wei, Xiang Yu, Fengyi Yuan},
    journal={arXiv preprint arXiv:2407.04521},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.04521},
    primaryClass={math.OC cs.LG q-fin.CP}
}