General boundary value problems of the Korteweg-de Vries equation on a bounded domain
MCRF Home
Necessary conditions for infinite horizon optimal control problems with state constraints
September & December 2018, 8(3&4): 557-582. doi: 10.3934/mcrf.2018023
Numerical study of polynomial feedback laws for a bilinear control problem
Tobias Breiten 1, , Karl Kunisch 1,2,, and Laurent Pfeiffer 1,
Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria
RICAM Institute, Austrian Academy of Sciences, Altenbergerstrasse 69, 4040 Linz, Austria
* Corresponding author: Karl Kunisch
Received September 2017 Revised July 2018 Published September 2018
Figure(13) / Table(5)
An infinite-dimensional bilinear optimal control problem with infinite-time horizon is considered. The associated value function can be expanded in a Taylor series around the equilibrium, the Taylor series involving multilinear forms which are uniquely characterized by generalized Lyapunov equations. A numerical method for solving these equations is proposed. It is based on a generalization of the balanced truncation model reduction method and some techniques of tensor calculus, in order to attenuate the curse of dimensionality. Polynomial feedback laws are derived from the Taylor expansion and are numerically investigated for a control problem of the Fokker-Planck equation. Their efficiency is demonstrated for initial values which are sufficiently close to the equilibrium.
Keywords: Value function, Hamilton Jacobi Bellman equation, bilinear control systems, Riccati equation, generalized Lyapunov equations, Fokker-Planck equation, balanced truncation, tensor calculus.
Mathematics Subject Classification: Primary: 49J20, 49N35, 93B40, 93D15.
Citation: Tobias Breiten, Karl Kunisch, Laurent Pfeiffer. Numerical study of polynomial feedback laws for a bilinear control problem. Mathematical Control & Related Fields, 2018, 8 (3&4) : 557-582. doi: 10.3934/mcrf.2018023
C. Aguilar and A. Krener, Numerical solutions to the Bellman equation of optimal control, Journal of Optimization Theory and Applications, 160 (2014), 527-552. doi: 10.1007/s10957-013-0403-8. Google Scholar
A. Alla and M. Falcone, An adaptive pod approximation method for the control of advectiondiffusion equations, in Control and Optimization with PDE Constraints (eds. K. Bredies, C. Clason, K. Kunisch and G. von Winckel), Springer Basel, Basel, 2013, 1-17. doi: 10.1007/978-3-0348-0631-2_1. Google Scholar
F. Ancona and A. Bressan, Flow stability of patchy vector fields and robust feedback stabilization, SIAM Journal on Control and Optimization, 41 (2002), 1455-1476. doi: 10.1137/S0363012901391676. Google Scholar
M. Badra, Lyapunov function and local feedback boundary stabilization of the Navier-Stokes equations, SIAM Journal on Control and Optimization, 48 (2009), 1797-1830. doi: 10.1137/070682630. Google Scholar
P. Benner, T. Breiten, C. Hartmann and B. Schmidt, Model Reduction of Controlled Fokker-Planck and Liouville-von Neumann Equations, Technical report, 2017, Available from https://arXiv.org/abs/1706.09882. Google Scholar
P. Benner and T. Damm, Lyapunov equations, energy functionals, and model order reduction of bilinear and stochastic systems, SIAM Journal on Control and Optimization, 49 (2011), 686-711. doi: 10.1137/09075041X. Google Scholar
A. Bensoussan, G. Da Prato, M. Delfour and S. Mitter, Representation and Control of Infinite Dimensional Systems, Birkhäuser Boston Basel Berlin, 2007. doi: 10.1007/978-0-8176-4581-6. Google Scholar
T. Breiten, K. Kunisch and L. Pfeiffer, Control strategies for the Fokker-Planck equation, ESAIM: Control, Optimisation and Calculus of Variations, 24 (2018), 741-763. doi: 10.1051/cocv/2017046. Google Scholar
T. Breiten, K. Kunisch and L. Pfeiffer, Taylor Expansions for the HJB Equation Associated with a Bilinear Control Problem, Technical report, SFB-Report 2017-006, 2017, Available from http://imsc.uni-graz.at/mobis/publications/SFB-Report-2017-006_v2.pdf. Google Scholar
E. Carlini and F.J. Silva, On the discretization of some nonlinear Fokker-Planck-Kolmogorov equations and applications, SIAM J. Numer. Anal., 56 (2018), 2148-2177. doi: 10.1137/17M1143022. Google Scholar
J. Chang and G. Cooper, A practical scheme for Fokker--Planck equations, Journal of Computational Physics, 6 (1970), 1-16. Google Scholar
T. Damm, Direct methods and ADI-preconditioned Krylov subspace methods for generalized Lyapunov equations, Numerical Linear Algebra with Applications, 15 (2008), 853-871. doi: 10.1002/nla.603. Google Scholar
L. Grasedyck, Existence and computation of low Kronecker-rank approximations for large linear systems of tensor product structure, Computing, 72 (2004), 247-265. doi: 10.1007/s00607-003-0037-z. Google Scholar
K. Ito and S. Ravindran, A reduced-order method for simulation and control of fluid flows, Journal of Computational Physics, 143 (1998), 403-425, URL http://www.sciencedirect.com/science/article/pii/S0021999198959435. doi: 10.1006/jcph.1998.5943. Google Scholar
D. Kalise and K. Kunisch, Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs, SIAM J. Sci. Comput., 40 (2018), A629-A652. doi: 10.1137/17M1116635. Google Scholar
A. Krener, C. Aguilar and T. Hunt, Mathematical system theory - festschrift in honor of uwe helmke on the occasion of his sixtieth birthday, CreateSpace, Chapter Series Solutions of HJB Equations, (2013), 247-260. Google Scholar
K. Kunisch, S. Volkwein and L. Xie, HJB-POD-based feedback design for the optimal control of evolution problems, SIAM Journal on Applied Dynamical Systems, 3 (2004), 701-722. doi: 10.1137/030600485. Google Scholar
B.J. Matkowsky and Z. Schuss, Eigenvalues of the Fokker-Planck operator and the approach to equilibrium for diffusions in potential fields, SIAM Journal on Applied Mathematics, 40 (1981), 242-254. doi: 10.1137/0140020. Google Scholar
M. Opmeer, Decay of Hankel singular values of analytic control systems, Systems & Control Letters, 59 (2010), 635-638. doi: 10.1016/j.sysconle.2010.07.009. Google Scholar
J.-P. Raymond, Feedback boundary stabilization of the two-dimensional Navier--Stokes equations, SIAM Journal on Control and Optimization, 45 (2006), 790-828. doi: 10.1137/050628726. Google Scholar
Figure 1. 1D Fokker-Planck equation
Figure 2. Comparison of the original and reduced models, for $n = 100$, $r = 25$, and $\beta = 10^{-4}$
Figure 3">Figure 4. Comparison of the reduced models with $r = 21$ and $r = 9$, derived from a finer discretization (with $n = 1000$), with the setup of Figure 3
Figure 5. Convergence of the control laws for $\beta = 10^{-4}, n = 1000$ and $r = 9.$
Figure 7. Initial condition and controls for the test case 2
Figure 10. Control shape functions α1 and α2
Figure 11. Singular value decay for n = 2500
Figure 12. Initial condition and controls for the test case 4
Table 1. Convergence results for the test case 1
$\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$
1e$^{-3}$ 0.038 0.038 0.038 0.038 0.038 0.038
(A) Cost of the controls $u_p$.
$\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$
$p=2$ $p=3$ $p=4$ $p=5$ $p=6$
1e$^{-3}$ 0.228 0.026 0.024 0.024 0.024
1e$^{-4}$ 4.26 1.19 0.82 0.61 0.61
1e$^{-5}$ 29.8 10.3 7.91 4.70 4.05
(B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
(A) Cost of the controls up.
1e$^{-4}$ 18.50 7.02 3.16 4.01 1.52
5e$^{-5}$ 46.87 13.18 8.40 8.17 2.65
1e$^{-5}$ 90.5 78.0 39.0 42.6 34.3
(B) L2-distance between the controls up and the optimal control uopt.
1e$^{-4}$ 0.381 0.368 2.689 $\infty$ $\infty$ 0.246
5e$^{-5}$ 0.381 0.432 $\infty$ $\infty$ $\infty$ 0.206
1e$^{-5}$ 0.365 $\infty$ $\infty$ $\infty$ $\infty$ 0.147
5e$^{-4}$ 11.26 5.03 7.11 11.89 11.99
1e$^{-4}$ 46.34 35.36 57.08 $\infty$ $\infty$
5e$^{-5}$ 74.79 60.86 $\infty$ $\infty$ $\infty$
1e$^{-5}$ 172.3 $\infty$ $\infty$ $\infty$ $\infty$
$\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$
1e$^{-5}$ 0.524 0.182 20.696 0.164 0.158
$p=2$ $p=3$ $p=4$ $p=5$
1e$^{-3}$ 3.53 0.80 0.19 0.14
1e$^{-4}$ 27.40 5.78 1.83 1.24
5e$^{-5}$ 52.50 11.06 3.69 2.40
1e$^{-5}$ 257.01 63.97 84.31 10.61
1e$^{-3}$ 2.45 2.41 $\infty$ $\infty$ 0.93
5e$^{-3}$ 21.59 19.80 9.66 20.06
1e$^{-3}$ 47.34 55.69 $\infty$ $\infty$
Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369
Bian-Xia Yang, Shanshan Gu, Guowei Dai. Existence and multiplicity for Hamilton-Jacobi-Bellman equation. Communications on Pure & Applied Analysis, 2021, 20 (11) : 3767-3793. doi: 10.3934/cpaa.2021130
Sylvain De Moor, Luis Miguel Rodrigues, Julien Vovelle. Invariant measures for a stochastic Fokker-Planck equation. Kinetic & Related Models, 2018, 11 (2) : 357-395. doi: 10.3934/krm.2018017
Marco Torregrossa, Giuseppe Toscani. On a Fokker-Planck equation for wealth distribution. Kinetic & Related Models, 2018, 11 (2) : 337-355. doi: 10.3934/krm.2018016
Michael Herty, Christian Jörres, Albert N. Sandjo. Optimization of a model Fokker-Planck equation. Kinetic & Related Models, 2012, 5 (3) : 485-503. doi: 10.3934/krm.2012.5.485
José Antonio Alcántara, Simone Calogero. On a relativistic Fokker-Planck equation in kinetic theory. Kinetic & Related Models, 2011, 4 (2) : 401-426. doi: 10.3934/krm.2011.4.401
Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161
Zhen-Zhen Tao, Bing Sun. A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation. Electronic Research Archive, 2021, 29 (5) : 3429-3447. doi: 10.3934/era.2021046
Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251
Fabio Camilli, Serikbolsyn Duisembay, Qing Tang. Approximation of an optimal control problem for the time-fractional Fokker-Planck equation. Journal of Dynamics & Games, 2021, 8 (4) : 381-402. doi: 10.3934/jdg.2021013
Helge Dietert, Josephine Evans, Thomas Holding. Contraction in the Wasserstein metric for the kinetic Fokker-Planck equation on the torus. Kinetic & Related Models, 2018, 11 (6) : 1427-1441. doi: 10.3934/krm.2018056
Andreas Denner, Oliver Junge, Daniel Matthes. Computing coherent sets using the Fokker-Planck equation. Journal of Computational Dynamics, 2016, 3 (2) : 163-177. doi: 10.3934/jcd.2016008
Ioannis Markou. Hydrodynamic limit for a Fokker-Planck equation with coefficients in Sobolev spaces. Networks & Heterogeneous Media, 2017, 12 (4) : 683-705. doi: 10.3934/nhm.2017028
Manh Hong Duong, Yulong Lu. An operator splitting scheme for the fractional kinetic Fokker-Planck equation. Discrete & Continuous Dynamical Systems, 2019, 39 (10) : 5707-5727. doi: 10.3934/dcds.2019250
Giuseppe Toscani. A Rosenau-type approach to the approximation of the linear Fokker-Planck equation. Kinetic & Related Models, 2018, 11 (4) : 697-714. doi: 10.3934/krm.2018028
Larry M. Bates, Francesco Fassò, Nicola Sansonetto. The Hamilton-Jacobi equation, integrability, and nonholonomic systems. Journal of Geometric Mechanics, 2014, 6 (4) : 441-449. doi: 10.3934/jgm.2014.6.441
Xuhui Wang, Lei Hu. A new method to solve the Hamilton-Jacobi-Bellman equation for a stochastic portfolio optimization model with boundary memory. Journal of Industrial & Management Optimization, 2021 doi: 10.3934/jimo.2021137
Ludovic Dan Lemle. $L^1(R^d,dx)$-uniqueness of weak solutions for the Fokker-Planck equation associated with a class of Dirichlet operators. Electronic Research Announcements, 2008, 15: 65-70. doi: 10.3934/era.2008.15.65
Joseph G. Conlon, André Schlichting. A non-local problem for the Fokker-Planck equation related to the Becker-Döring model. Discrete & Continuous Dynamical Systems, 2019, 39 (4) : 1821-1889. doi: 10.3934/dcds.2019079
Simon Plazotta. A BDF2-approach for the non-linear Fokker-Planck equation. Discrete & Continuous Dynamical Systems, 2019, 39 (5) : 2893-2913. doi: 10.3934/dcds.2019120
Tobias Breiten Karl Kunisch Laurent Pfeiffer