In this paper, we present an approach to learn recomposable motor primitives across large-scale and diverse manipulation demonstrations. Current approaches to decomposing demonstrations into primitives often assume manually defined primitives and bypass the difficulty of discovering these primitives. On the other hand, approaches in primitive discovery put restrictive assumptions on the complexity of a primitive, which limit applicability to narrow tasks. Our approach attempts to circumvent these challenges by jointly learning both the underlying motor primitives and recomposing these primitives to form the original demonstration. Through constraints on both the parsimony of primitive decomposition and the simplicity of a given primitive, we are able to learn a diverse set of motor primitives, as well as a coherent latent representation for these primitives. We demonstrate, both qualitatively and quantitatively, that our learned primitives capture semantically meaningful aspects of a demonstration. This allows us to compose these primitives in a hierarchical reinforcement learning setup to efficiently solve robotic manipulation tasks like reaching and pushing.
Paper Summary:
The paper proposes a method for learning a set of primitives for robotic movements from a dataset of demonstrations, showing a diverse set of tasks, in an unsupervised fashion. The central underlying idea is that robotic tasks can be solved by combining fundamental building blocks, the so-called "motor programs", in the right way. The described algorithm takes a demonstration and uses a transformer network to embed the trajectory into a sequence of latent variables. Then each individual latent is transformed to a 10 step trajectory for the joint space of the robot via an LSTM network. Finally the individual trajectories are concatenated and the reconstruction is compared to the original demonstration through dynamic time warping. In this structure the latent variables represent a query to a specific learned primitive, which can be accessed using the LSTM. 
In the experimental section, the paper gives mostly qualitative insight into the learned representation. The paper visualizes different movements and their corresponding latent variables projected onto two dimensions. Further, it is shown that the segmentation of demonstrations roughly compares to how a human expert would manually segment the given tasks. Finally, the paper shows that a hierarchical RL algorithm trained in the learned latent space outperforms one which works directly in the low-level control space of the robot in the sense that it learns to solve the given task much faster. 
Evaluation:
The problem of discovering primitives is approached by the authors in a novel and interesting way, however in my opinion the paper should be rejected because:
    (a) the experimental section is not convincing enough to support the claim that the method captures the shared motions across different skills. Especially, the paper misses to adequately show how these motions can be recombined and used to solve robotic tasks. 
    (b) the paper is imprecise and missing important details in both the description of the method and the experimental verification 
    (c) the paper misses important related work, which tackles the same problem.
The two main claims in the paper are:
1. The presented method learns a latent space which represents common shared motions among diverse tasks encountered in robotics
2. Robotic tasks can be solved by recombining primitives from the aformentioned space 
Although it is impossible to verify the first claim based on the paper alone, the provided webpage, which shows an animated version of Figure 3 nicely visualizes the learned latent space and shows that the representation is somewhat smooth with similar movements clustered together.
a1) Figure 5 is supposed to show that the method manages to segment given demonstrations in a meaningful way, but even after zooming into the pdf, it is impossible to see what is actually going on. A visualization in a video would be preferable.
a2) 4.1.1 and Figure 4 show that individual primitives can be executed on a real robot, however, the paper fails to show the execution of a combination of primitives. 
a3) Given that the method seems to loose the connection of movements to time it would be interesting to see whether a combination actually results in smooth, natural movement of the robot.
a4) The main quantitative assessment of the usefulness of the learned motor program network is given by the RL experiments in section 4.3. However, the baseline method seems to output one single velocity control action per evaluation of the policy (?), whereas the presented method essentially outputs an action sequence of 50 actions per policy evaluation. State-of-the-art methods commonly use frame-skipping and repeat the same action for multiple timesteps, because it makes the resulting optimization problem easier and speeds up the learning. See for example (Mnih 2013) (Mnih 2015) (Lillycrap 2015) (Hafner 2018). It would be interesting to compare against a baseline which also incorporates some form of frame-skipping and validate the speedup is not simply due to chunking of action sequences.
a5) Finally, the method outperforms the plain PPO baseline when it comes to speeding up the learning process, but the solutions found do not look like natural robot movements. You can clearly spot different primitives and transitions between individual segments look unnatural and jerky. How does the baseline solution compare in this regard?
a6) Given that the presented movements look unnatural and the fact that the paper only shows the execution of individual primitives in the rest of the paper, I simply cannot support the second claim.
Details I am missing from the paper:
b1) How is the "continuation probability" computed with the transformer network?
b2) Are the biases in section 3.2 necessary to make the method work at all?
b3) In section 4.2, how does the sequence alignment work?
b4) In section 4.2, what is the training set for the labelling task?
b5) What are the exact task parameters given to the policy in the RL task?
b6) What do you mean with, "these motor programs are executed without environment feedback"? Does the policy determine the complete sequence of programs in one step?
c1) Finally, to my knowledge the problem of learning meaningful primitives and showing that they can be combined in a different way to solve novel, unseen tasks has already been investigated in (Lioutikov 2017). Although this paper approaches the problem very differently and the dimensionality of the primitives is lower, I still consider this paper very much related to what is presented by the authors. I would suggest adding it to the related work. The paper shows that previous methods for discovering primitives from a set of different tasks exist.
In my opinion the paper in its current is probably not yet ready for publication. However, I strongly encourage the authors to address the above mentioned problems.
References:
Mnih, Volodymyr, et al. "Playing atari with deep reinforcement learning." arXiv preprint arXiv:1312.5602 (2013).
Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529.
Lillicrap, Timothy P., et al. "Continuous control with deep reinforcement learning." arXiv preprint arXiv:1509.02971 (2015).
Hafner, Danijar, et al. "Learning latent dynamics for planning from pixels." arXiv preprint arXiv:1811.04551 (2018).
Lioutikov, Rudolf, et al. "Learning movement primitive libraries through probabilistic segmentation." The International Journal of Robotics Research 36.8 (2017): 879-894.