Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling
<|reference_start|>Online Learning of Weakly Coupled MDP Policies for Load Balancing and Auto Scaling: Load balancing and auto scaling are at the core of scalable, contemporary systems, addressing dynamic resource allocation and service rate adjustments in response to workload changes. This paper introduces a novel model and algorithms for tuning load balancers coupled with auto scalers, considering bursty traffic arriving at finite queues. We begin by presenting the problem as a weakly coupled Markov Decision Processes (MDP), solvable via a linear program (LP). However, as the number of control variables of such LP grows combinatorially, we introduce a more tractable relaxed LP formulation, and extend it to tackle the problem of online parameter learning and policy optimization using a two-timescale algorithm based on the LP Lagrangian.<|reference_end|>
@article{eshwar2024online,
    title={Online Learning of Weakly Coupled MDP Policies for Load Balancing and
  Auto Scaling},
    author={S.R. Eshwar, Lucas Lopes Felipe, Alexandre Reiffers-Masson, Daniel
  Sadoc Menasch'e, Gugan Thoppe},
    journal={arXiv preprint arXiv:2406.14141},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.14141},
    primaryClass={eess.SY cs.AI cs.NI cs.SY}
}