In this work we propose a novel end-to-end imitation learning approach which combines natural language, vision, and motion information to produce an abstract representation of a task, which is in turn used to synthesize specific motion controllers at run-time. This multimodal approach enables generalization to a wide variety of environmental conditions and allows an end-user to direct a robot policy through verbal communication. We empirically validate our approach with an extensive set of simulations and show that it achieves a high task success rate over a variety of conditions while remaining amenable to probabilistic interpretability.
The paper addresses the problem of using multiple modalities for learning from demonstration. Approaches that take in task or joint space data to learn a policy for replicating that task are numerous. Doing the same with multiple modalities involved, in particular vision, language and motion, has only been recently considered, so this is a timely paper. 
The core contribution is pretty well summarised by the architecture in figure 1, which involves a combination of encodings of the words and sentences, images and parameters of a DMP in order to generate movement commands from a high level instruction. 
Unless I have missed something in the experimental setup, all of the considered task variations are movement commands of the form <Move> to <Object>. The network setup allows for synonyms of two kinds, so <Move> can be replaced by numerous verbal synonyms such as advance and go, and the object can be specified in terms of shapes, colors and so on, but otherwise this is the only specification of the task. This has been addressed in the recent literature using neural network architectures similar to the one being proposed here, e.g., see the following papers. These papers already solve the proposed problem and provide similar explanations. It would be helpful to see comparative discussion with respect to those methods and a clear statement of novelty with respect to such prior work:
[R1] M. Burke, S. Penkov, S. Ramamoorthy, From explanation to synthesis: Compositional program induction for learning from demonstration, Robotics: Science and Systems (R:SS), 2019.
[R2] Y. Hristov, D. Angelov, A.Lascarides, M. Burke, S. Ramamoorthy, Disentangled Relational Representations for Explaining and Learning from Demonstration, Conference on Robot Learning (CoRL), 2019. 
An interesting feature in R2 that the authors do not explicitly address here is the issue of relational specifications in the language, e.g., in addition to saying "move to the red bowl", we may also wish to say "place on top of red block". In the way that MPN is currently set up to map from the language input directly to hyperparameters of the DMP, and considering the embedding structure, it is not clear if MPN is capable of handling such specifications. If so, the claim of generalisation on the language input should be stated more clearly.
The ablation study is setup somewhat differently than what I would have expected. The authors consider the effect of changing the training set size and if the language input includes synonyms or not. Those two aspects seem to produce the expected results. It would also be interesting to see an ablation study in the sense of replacing or removing aspects of the architecture to see its relative effect on the overall model performance. So, for instance, if one did not have a DMP with the hyperparameters being estimated by a network and instead had a more straightforward encoding of where to move to - does it make a difference and how much? Likewise, how much performance benefit, if any, is being derived from an uninterpreted image I being combined as described in the embedding as opposed to an alternative that detects an objects and combines that position differently. The paper would have been stronger if such architectural choices were better justified and also demonstrated in the experiments.