People are incredibly skilled at imitating others by simply observing them. They achieve this even in the presence of significant morphological differences and capabilities. Further, people are able to do this from raw perceptions of the actions of others, without direct access to the abstracted demonstration actions and with only partial state information. People therefore solve a difficult problem of understanding the salient features of both observations of others and the relationship to their own state when learning to imitate specific tasks. We can attempt to reproduce a similar demonstration via trial and error and through this gain more understanding of the task space. To reproduce this ability an agent would need to both learn how to recognize the differences between itself and some demonstration and at the same time learn to minimize the distance between its own performance and that of the demonstration. In this paper we propose an approach using only visual information to learn a distance metric between agent behaviour and a given video demonstration. We train a Recurrent Neural Network (RNN)-based Siamese model to compute distances in space and time between motion clips while training an Rienforcement Learning (RL) policy to minimize this distance. Furthermore, we examine a particularly challenging form of this problem where the agent must learn an imitation based task given a single demonstration. We demonstrate our approach in the setting of deep learning based control for physical simulation of humanoid walking in both 2D with 10 degrees of freedom (DoF) and 3D with 38 DoF.
Brief summary: This work proposes a way to perform imitation learning from raw videos of behaviors, without the need for any special time-alignment or actions present. They are able to do this by using a recurrent siamese network architecture to learn a distance function, which can be used to provide rewards for learning behaviors, without the need for any explicit pose estimation. They demonstrate effectiveness on 2 different locomotion domains. 
Overall impression:
Overall, my impression from this paper is that the idea is to use a recurrent siamese network to learn distances which make sense in latent space and provide rewards for RL. This is able to learn interesting behaviors for 2 tasks. But I think the writing needs significant work for clarity and completeness, and there needs to be many more baseline comparisons. 
Abstract comments:
trail and error -> trial and error
Introduction comments:
Alternative reasons why pose estimation won’t work is because for any manipulation tasks, you can’t just detect pose of the agent, you also have to detect pose of the objects which may be novel/different
Few use image based inputs and none consider the importance of learning a distance function in time as well as space -> missed a few citations (eg imitation from observation (Liu, Gupta, et al))
Therefore we learned an RNN-based distance function that can give reward for out of sync but similar behaviour -> could be good to emphasize difference from imitation from observation (Liu, Gupta, et al) and TCN (Semanet et al), since they both assume some sort of time alignment
Missing related work section. There is a lot of related work at this point and it is crucial to add this in. Some things that come to mind beyond those already covered are:
1. Model-based Imitation Learning from State Trajectories
2. Reward Estimation via State Prediction
3. infoGAIL
4. Imitation from observation 
5. SFV: Reinforcement Learning of Physical Skills from Videos
6. Universal planning networks
7. https://arxiv.org/abs/1808.00928
8. This might also be related to VICE (Fu, Singh et al), in that they also hope to learn distances but for goal images only.
It seems like there is some discussion of this in Section 3.1, but it should be it’s own separate section.
Section 3 comments:
a new model can be learned to match this trajectory using some distance metric between the expert trajectories and trajectories produced by the policy π -> what does this mean. Can this be clarified?
 The first part of Section 3 belongs in preliminaries. It is not a part of the approach. 
Section 3.2
Equations 9 and 10 are a bit unnecessary, take away from the main point
What does distance from desired behaviour mean? This is not common terminology and should be clarified explicitly.
Equation 11 is very confusing. The loss function is double defined.  what exactly Is the margin \rho (is it learned?) The exact rationale behind this objective, the relationship to standard siamese networks/triplet losses like TCN should be discussed carefully. This is potentially the most important part of the paper, it should be discussed in detail.Also is there a typo, should it be || f(si) - f(sn)|| if we want it to be distances? Also the role of trajectories is completely not discussed in equation 11.
Section 3.3 
The recurrent siamese architecture makes sense, but what the positive and negative examples are, what exactly the loss function is, needs to be defined clearly. Also if there are multiple demonstrations of a task, which distance do we use then?
The RL simulation environment is it made in-house, based on bullet or something else?
Data augmentation - how necessary is this for method success? Can an ablation be done to show the necessity of this?
Algorithm 1 has some typos 
- > is missing in line 3
- Describe where reward r is coming from in line 10
Section 4.1
Walking gate -> walking gait
There are no comparisons with any of the prior methods for performing this kind of thing. For example, using the pose estimation baseline etc. Using the non-recurrent version. Using TCN type of things. It’s not hard to run these and might help a lot, because right now there are no baseline comparisons