We address the problem of imitating multi-modal expert demonstrations in sequential decision making problems. In many practical applications, for example video games, behavioural demonstrations are readily available that contain multi-modal structure not captured by typical existing imitation learning approaches. For example, differences in the observed players' behaviours may be representative of different underlying playstyles. In this paper, we use a generative model to capture different emergent playstyles in an unsupervised manner, enabling the imitation of a diverse range of distinct behaviours. We utilise a variational autoencoder to learn an embedding of the different types of expert demonstrations on the trajectory level, and jointly learn a latent representation with a policy. In experiments on a range of 2D continuous control problems representative of Minecraft environments, we empirically demonstrate that our model can capture a multi-modal structured latent space from the demonstrated behavioural trajectories.
This paper presents an approach to multi-modal imitation learning by using a variational auto-encoder to embed demonstrated trajectories into a structured latent space that captures the multi-modal structure. This is done through a stochastic neural network with a bi-directional LSTM and mean pooling architecture that predicts the mean and log-variance of the latent state. This is followed by a state and action/policy decoder (both LSTMs) that recursively generate trajectories from latent space samples. The entire model is trained by optimising the ELBO on a set of pre-specified expert demonstrations. At test time, samples are generated from the latent space and recursively decoded to generate state and action trajectories. The method is tested on three low-dimensional continuous control tasks and is able to learn structured latent spaces capturing the modes in the training data as well as generating good trajectory reconstructions.
Learning from multi-modal demonstration data is an important sub-area in imitation learning. As the paper pointed out, there has been a lot of recent work in this area. A lot of the ideas in this paper are similar to those proposed in prior work -- the network for embedding the trajectory is similar to the ones from Wang et al & Co-Reyes et al with the major difference being in the structure of the action decoder (and what inputs to encoder). Also, prior work has dealt with problems that are high-dimensional (Wang et al) and has shown results when operating directly on visual data (InfoGAIL). Comparatively, the results in this paper are on toy problems. 
As there is no direct comparison to prior work provided in the paper, it is hard to quantify how much better the proposed approach is in comparison to prior work. For example, the "2D Circle Example" was taken from the InfoGAIL paper. It would have been good to use that as a baseline example to compare those two methods and highlight the advantages of the proposed approach -- did it require less data? fewer environment interactions? etc. 
The results on the Zombie Attack Scenario seem poor. Specifically, in the avoid scenario, the approach seems to fail almost half the time. It would be good if the authors spend more time on this -- again, a comparison to prior work would establish some baselines and give us a good idea of the expected performance on this scenario. The videos show a single representative example for the "Attack" and "Avoid" scenarios. More examples including failures need to be included so that the distribution of results can be captured. 
There is little in terms of generalisation or ablation studies in the paper. For example, in the Zombie Attack Scenario one could generate data with different zombie behaviours and measure performance on held out behaviours. Similarly, as an ablation, the authors could look at directly predicting actions instead of states & actions (states could be generated through a pre-trained dynamics model).
Figure 6. is hard to parse and could be explained better. No details are provided on the network architecture (number/size of the LSTM/fully connected layers), number of demonstrations used, training algorithm, hyper-parameters etc. 
Few typos in the paper: 
  Page 6 - between the animation links 'avoiding' 'region'
  Fig 7 caption - the zombie but are not in attacking range -> but the zombies are not in the attacking range,
Relevant citations that can be added:
1) Hausman, K., Chebotar, Y., Schaal, S., Sukhatme, G., & Lim, J. J. (2017). Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets. In Advances in Neural Information Processing Systems (pp. 1235-1245).
2) Tamar, A., Rohanimanesh, K., Chow, Y., Vigorito, C., Goodrich, B., Kahane, M., & Pridmore, D. (2018). Imitation Learning from Visual Data with Multiple Intentions.
Overall, I find the paper to be incremental and lacking good experimental results and comparisons. The strengths of the paper are not clear and need to be explained and evaluated well. Substantial work is needed to significantly improve the paper before it can be accepted.