Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell Massive MIMO Systems
<|reference_start|>Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell Massive MIMO Systems: We develop a multi-agent reinforcement learning (MARL) algorithm to minimize the total energy consumption of multiple massive MIMO (multiple-input multiple-output) base stations (BSs) in a multi-cell network while preserving the overall quality-of-service (QoS) by making decisions on the multi-level advanced sleep modes (ASMs) and antenna switching of these BSs. The problem is modeled as a decentralized partially observable Markov decision process (DEC-POMDP) to enable collaboration between individual BSs, which is necessary to tackle inter-cell interference. A multi-agent proximal policy optimization (MAPPO) algorithm is designed to learn a collaborative BS control policy. To enhance its scalability, a modified version called MAPPO-neighbor policy is further proposed. Simulation results demonstrate that the trained MAPPO agent achieves better performance compared to baseline policies. Specifically, compared to the auto sleep mode 1 (symbol-level sleeping) algorithm, the MAPPO-neighbor policy reduces power consumption by approximately 8.7% during low-traffic hours and improves energy efficiency by approximately 19% during high-traffic hours, respectively.<|reference_end|>
@article{cai2024multi-agent,
    title={Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell
  Massive MIMO Systems},
    author={Tianzhang Cai, Qichen Wang, Shuai Zhang, "Ozlem Tuu{g}fe Demir,
  Cicek Cavdar},
    journal={arXiv preprint arXiv:2402.03204},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.03204},
    primaryClass={cs.IT cs.AI cs.LG math.IT}
}