Our goal is to train a policy for autonomous driving via imitation learning that is robust enough to drive a real vehicle. We find that standard behavior cloning is insufficient for handling complex driving scenarios, even when we leverage a perception system for preprocessing the input and a controller for executing the output on the car: 30 million examples are still not enough. We propose exposing the learner to synthesized data in the form of perturbations to the expert's driving, which creates interesting situations such as collisions and/or going off the road. Rather than purely imitating all data, we augment the imitation loss with additional losses that penalize undesirable events and encourage progress -the perturbations then provide an important signal for these losses and lead to robustness of the learned model. We show that the model can handle complex situations in simulation, and present ablation experiments that emphasize the importance of each of our proposed changes and show that the model is responding to the appropriate causal factors. Finally, we demonstrate the model driving a car in the real world.
The paper describes a framework for training a self-driving policy by augmenting imitation loss with additional loss terms that penalize undesired behaviors and that encourage progress. The policy takes as input a parsed representation of the scene (rather than raw images) and outputs pose trajectories for a down-stream controller. The method is trained on simulated data that includes perturbations to improve generalizability. The framework is evaluated in simulation through a series of ablations to better understand the contribution of the different loss terms.
STRENGTHS
+ Paper acknowledges the difficulty of end-to-end (pixels-to-torque) learning for autonomous driving and instead reasons over pre-processed inputs in the form of lower-dimensional images (and image sequences) that capture obstacles as bounding boxes and simple lines for routes, grayscale intensities, etc. Similarly, the output is a trajectory that is then fed to a controller responsible for tracking this trajectory.
WEAKNESSES
- The insufficiency of behavioral cloning is not surprising, as noted, given the covariate shift. It would be interesting to consider a  no-regret formulation analogous to Ross et al., 2011, even though it would require interaction with a human.
- The limitation of producing paths in this way is that the network does not explicitly reason over the feasibility of the path, which is important for non-holonomic vehicles. Instead, the network must learn the kinematic and dynamic constraints.
- Perturbations of the simulated trajectories are used to expose the model to collisions and other rare events, but is not clear that simple trajectory perturbations such as those used here provide a sufficient exposure to these rare events.
- The fact that the 2D image that expresses the vehicle's position is absolute limits the environment in which the network is valid. The experiments are conducted on images corresponding to an 80m x 80m environment, which is trivially small.
- The proposed framework is highly specific to self-driving and the extent to which it provides insights for other domains is not clear.
- The ablation experiments are not very compelling. In the case of the nudging experiment, all models result in collisions with M4 being the best model with a 10% collision rate. The trajectory perturbation results are better. In the case of the slowing experiment, M3 is the only version to not result in collision, whereas M4 collides 5% of the time. It isn't clear than which model is preferable since, while M3 never collides in the case of the slowing down experiment, it collides 45% of the time in the nudging experiment, almost as frequently as the M0 baseline.
- The paper claims that the model was run on a real robot, but there is no experimental evaluation of the results, only a reference to videos. The results of these experiments should be quantified and discussed or the reference to running on a real vehicle should be toned down, if not removed.
- Equation 3 requires knowledge of the ground-truth distribution. How is this determined?