  The unit commitment (UC) problem, which determines operating schedules of
generation units to meet demand, is a fundamental task in power systems
operation. Existing UC methods using mixed-integer programming are not
well-suited to highly stochastic systems. Approaches which more rigorously
account for uncertainty could yield large reductions in operating costs by
reducing spinning reserve requirements; operating power stations at higher
efficiencies; and integrating greater volumes of variable renewables. A
promising approach to solving the UC problem is reinforcement learning (RL), a
methodology for optimal decision-making which has been used to conquer
long-standing grand challenges in artificial intelligence. This thesis explores
the application of RL to the UC problem and addresses challenges including
robustness under uncertainty; generalisability across multiple problem
instances; and scaling to larger power systems than previously studied. To
tackle these issues, we develop guided tree search, a novel methodology
combining model-free RL and model-based planning. The UC problem is formalised
as a Markov decision process and we develop an open-source environment based on
real data from Great Britain's power system to train RL agents. In problems of
up to 100 generators, guided tree search is shown to be competitive with
deterministic UC methods, reducing operating costs by up to 1.4\%. An advantage
of RL is that the framework can be easily extended to incorporate
considerations important to power systems operators such as robustness to
generator failure, wind curtailment or carbon prices. When generator outages
are considered, guided tree search saves over 2\% in operating costs as
compared with methods using conventional $N-x$ reserve criteria.

Systems and Control;Machine Learning;Systems and Control