Policy Optimization for PDE Control with a Warm Start
<|reference_start|>Policy Optimization for PDE Control with a Warm Start: Dimensionality reduction is crucial for controlling nonlinear partial differential equations (PDE) through a "reduce-then-design" strategy, which identifies a reduced-order model and then implements model-based control solutions. However, inaccuracies in the reduced-order modeling can substantially degrade controller performance, especially in PDEs with chaotic behavior. To address this issue, we augment the reduce-then-design procedure with a policy optimization (PO) step. The PO step fine-tunes the model-based controller to compensate for the modeling error from dimensionality reduction. This augmentation shifts the overall strategy into reduce-then-design-then-adapt, where the model-based controller serves as a warm start for PO. Specifically, we study the state-feedback tracking control of PDEs that aims to align the PDE state with a specific constant target subject to a linear-quadratic cost. Through extensive experiments, we show that a few iterations of PO can significantly improve the model-based controller performance. Our approach offers a cost-effective alternative to PDE control using end-to-end reinforcement learning.<|reference_end|>
@article{zhang2024policy,
    title={Policy Optimization for PDE Control with a Warm Start},
    author={Xiangyuan Zhang, Saviz Mowlavi, Mouhacine Benosman, Tamer Bac{s}ar},
    journal={arXiv preprint arXiv:2403.01005},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.01005},
    primaryClass={eess.SY cs.AI cs.SY math.OC}
}