  Advanced vehicle control is a fundamental building block in the development
of autonomous driving systems. Reinforcement learning (RL) promises to achieve
control performance superior to classical approaches while keeping
computational demands low during deployment. However, standard RL approaches
like soft-actor critic (SAC) require extensive amounts of training data to be
collected and are thus impractical for real-world application. To address this
issue, we apply recently developed data-efficient deep RL methods to vehicle
trajectory control. Our investigation focuses on three methods, so far
unexplored for vehicle control: randomized ensemble double Q-learning (REDQ),
probabilistic ensembles with trajectory sampling and model predictive path
integral optimizer (PETS-MPPI), and model-based policy optimization (MBPO). We
find that in the case of trajectory control, the standard model-based RL
formulation used in approaches like PETS-MPPI and MBPO is not suitable. We,
therefore, propose a new formulation that splits dynamics prediction and
vehicle localization. Our benchmark study on the CARLA simulator reveals that
the three identified data-efficient deep RL approaches learn control strategies
on a par with or better than SAC, yet reduce the required number of environment
interactions by more than one order of magnitude.

Machine Learning;Robotics