Decision and control are two of the core functionalities of high-level automated vehicles. Current mainstream methods, such as functionality decomposition or end-to-end reinforcement learning (RL), either suffer high time complexity or poor interpretability and limited safety performance in real-world complex autonomous driving tasks. In this paper, we present an interpretable and efficient decision and control framework for automated vehicles, which decomposes the driving task into multi-path planning and optimal tracking that are structured hierarchically. First, the multi-path planning is to generate several paths only considering static constraints . Then, the  optimal tracking is designed to track the optimal path while considering the dynamic obstacles. To that end, in theory, we formulate a constrained optimal control problem (OCP) for each candidate path, optimize them separately and choose the one with the best tracking performance to follow. More importantly , we propose a model-based reinforcement learning (RL) algorithm , which is served as an approximate constrained OCP solver , to unload the heavy computation by the paradigm of offline training and online application . Specifically, the OCPs for all paths are considered together to construct a multi-task RL problem and then solved offline by our algorithm into value and policy networks, for real-time online path selecting and tracking respectively. We verify our framework in both simulation and the real world. Results show that our method has better online computing efficiency and driving performance including traffic efficiency and safety compared with baseline methods . In addition, it yields great interpretability and adaptability among different driving tasks. The real road test also suggests that it is applicable in complicated traffic scenarios without even tuning .
Decision and control are core functionalities of high-level automated vehicles. Current mainstream methods, such as functionality decomposition and end-to-end reinforcement learning (RL), either suffer high time complexity or poor interpretability and adaptability on real-world autonomous driving tasks. In this paper, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically. First, the static path planning generates several candidate paths only considering static traffic elements . Then, the dynamic optimal tracking is designed to track the optimal path while considering the dynamic obstacles. To that end, we formulate a constrained optimal control problem (OCP) for each candidate path, optimize them separately and follow the one with the best tracking performance . To unload the heavy online computation , we propose a model-based reinforcement learning (RL) algorithm that can be served as an approximate constrained OCP solver . Specifically, the OCPs for all paths are considered together to construct a single complete RL problem and then solved offline in the form of value and policy networks, for real-time online path selecting and tracking respectively. We verify our framework in both simulations and the real world. Results show that compared with baseline methods IDC has an order of magnitude higher online computing efficiency , as well as better driving performance including traffic efficiency and safety . In addition, it yields great interpretability and adaptability among different driving tasks. The effectiveness of the proposed method is also demonstrated in real road tests with complicated traffic conditions .
D
two of the
R
or
and
R
limited safety performance in
adaptability on
D
complex
R
efficient
computationally efficient framework called integrated
R
framework
(IDC)
R
multi-path planning and
static path planning and dynamic
R
multi-path planning is to generate several
static path planning generates several candidate
R
constraints
traffic elements
A
dynamic
D
in theory,
R
choose
follow
R
to follow. More importantly
. To unload the heavy online computation
R
, which is
that can be
D
, to unload the heavy computation by the paradigm of offline training and online application
R
multi-task
single complete
R
by our algorithm into
in the form of
R
simulation
simulations
R
our method has better
compared with baseline methods IDC has an order of magnitude higher
R
and
, as well as better
D
compared with baseline methods
R
real road test also suggests that it is applicable in complicated traffic scenarios without even tuning
effectiveness of the proposed method is also demonstrated in real road tests with complicated traffic conditions