Published as a conference paper at ICLR 2018 IMITATION LEARNING FROM VISUAL DATA WITH MULTIPLE INTENTIONS
Recent advances in learning from demonstrations (LfD) with deep neural networks have enabled learning complex robot skills that involve high dimensional perception such as raw image inputs. LfD algorithms generally assume learning from single task demonstrations. In practice, however, it is more efficient for a teacher to demonstrate a multitude of tasks without careful task set up, labeling, and engineering. Unfortunately in such cases, traditional imitation learning techniques fail to represent the multi-modal nature of the data, and often result in sub-optimal behavior. In this paper we present an LfD approach for learning multiple modes of behavior from visual data. Our approach is based on a stochastic deep neural network (SNN), which represents the underlying intention in the demonstration as a stochastic activation in the network. We present an efficient algorithm for training SNNs, and for learning with vision inputs, we also propose an architecture that associates the intention with a stochastic attention module. Furthermore, we demonstrate our method on real robot visual object reaching tasks, and show that it can reliably learn the multiple behavior modes in the demonstration data. Video results are available at https://vimeo.com/240212286/fd401241b9.
The authors provide a method for learning from demonstrations where several modalities of the same task are given. The authors argue that in the case where several demonstrations exists and a deterministic (i.e., regular network) is given, the network learns some average policy from the demonstrations.
The paper begins with the authors stating the motivation and problem of how to program robots to do a task based only on demonstrations rather on explicit modeling or programming. They put the this specific work in the right context of imitation learning and IRL. Afterward, the authors argue that deterministic network cannot adequately several modalities. The authors cover in Section 2 related topics, and indeed the relevant literature includes behavioral cloning, IRL , Imitation learning, GAIL, and VAEs. I find that recent paper by Tamar et al 2016. on Value Iteration Networks is highly relevant to this work: the authors there learn similar tasks (i.e., similar modalities) using the same network. Even the control task is very similar to the current proposed task in this paper.
The authors argue that their contribution is 3-fold: (1) does not require robot  rollouts, (2) does not require label for a task, (3) work within raw image inputs. Again, Tamar et al. 2016 deals with this 3 points.
I went over the math. It seems right and valid. Indeed, SNN is a good choice for adding (Bayesian) context to a task. Also, I see the advantage of referring only to the "good" quantiles when needed. It is indeed a good method for dealing with the variance. 
I must say that I was impressed with the authors making the robot succeed in the tasks in hand (although reaching to an object is fairly simple task). 
My concerns are as follows:
1) Seems like that the given trajectories are naturally divided with different tasks, i.e., a single trajectory consists only a single task. For me, this is not the pain point in this tasks. the pain point is knowing when tasks are begin and end. 
2) I'm not sure, and I haven't seen evidence in the paper (or other references) that SNN is the only (optimal?) method for this context. Why not adding (non Bayesian) context (not label) to the task will not work as well? 
3) the robot task is impressive. but proving the point, and for the ease of comparing to different tasks, and since we want to show the validity of the work on more than 200 trials, isn't showing the task on some simulation is better for understanding the different regimes that this method has advantage? I know how hard is to make robotic tasks work...   
4) I’m not sure that the comparison of the suggested architecture to one without any underlying additional variable Z or context (i.e., non-Bayesian setup) is fair. "Vanilla" NN indeed may fail miserably . So, the comparison should be to any other work that can deal with "similar environment but different details".
To summarize, I like the work and I can see clearly the motivation. But I think some more work is needed in this work: comparing to the right current state of the art, and show that in principal (by demonstrating on other simpler simulations domains) that this method is better than other methods.