  We present a framework that lets a service provider achieve end-to-end
management objectives under varying load. Dynamic control actions are performed
by a reinforcement learning (RL) agent. Our work includes experimentation and
evaluation on a laboratory testbed where we have implemented basic information
services on a service mesh supported by the Istio and Kubernetes platforms. We
investigate different management objectives that include end-to-end delay
bounds on service requests, throughput objectives, and service differentiation.
These objectives are mapped onto reward functions that an RL agent learns to
optimize, by executing control actions, namely, request routing and request
blocking. We compute the control policies not on the testbed, but in a
simulator, which speeds up the learning process by orders of magnitude. In our
approach, the system model is learned on the testbed; it is then used to
instantiate the simulator, which produces near-optimal control policies for
various management objectives. The learned policies are then evaluated on the
testbed using unseen load patterns.

Machine Learning