Structured Reinforcement Learning for Media Streaming at the Wireless Edge
<|reference_start|>Structured Reinforcement Learning for Media Streaming at the Wireless Edge: Media streaming is the dominant application over wireless edge (access) networks. The increasing softwarization of such networks has led to efforts at intelligent control, wherein application-specific actions may be dynamically taken to enhance the user experience. The goal of this work is to develop and demonstrate learning-based policies for optimal decision making to determine which clients to dynamically prioritize in a video streaming setting. We formulate the policy design question as a constrained Markov decision problem (CMDP), and observe that by using a Lagrangian relaxation we can decompose it into single-client problems. Further, the optimal policy takes a threshold form in the video buffer length, which enables us to design an efficient constrained reinforcement learning (CRL) algorithm to learn it. Specifically, we show that a natural policy gradient (NPG) based algorithm that is derived using the structure of our problem converges to the globally optimal policy. We then develop a simulation environment for training, and a real-world intelligent controller attached to a WiFi access point for evaluation. We empirically show that the structured learning approach enables fast learning. Furthermore, such a structured policy can be easily deployed due to low computational complexity, leading to policy execution taking only about 15$\mu$s. Using YouTube streaming experiments in a resource constrained scenario, we demonstrate that the CRL approach can increase quality of experience (QOE) by over 30\%.<|reference_end|>
@article{bura2024structured,
    title={Structured Reinforcement Learning for Media Streaming at the Wireless
  Edge},
    author={Archana Bura, Sarat Chandra Bobbili, Shreyas Rameshkumar, Desik
  Rengarajan, Dileep Kalathil, Srinivas Shakkottai},
    journal={arXiv preprint arXiv:2404.07315},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.07315},
    primaryClass={eess.SY cs.AI cs.LG cs.SY}
}