Reinforcement learning (RL) is a metaheuristic aiming at teaching an agent to interact with an environment and maximizing the reward in a complex task. RL algorithms often encounter the difficulty in defining a reward function in a sparse solution space. Imitation learning (IL) deals with this issue by providing a few expert demonstrations, and then either mimicking the expert's behavior (behavioral cloning, BC) or recovering the reward function by assuming the optimality of the expert (inverse reinforcement learning, IRL). Conventional IL approaches formulate the agent policy by mapping one single state to a distribution over actions, which did not consider sequential information. This strategy can be less accurate especially in IL, a weakly supervised learning environment, especially when the number of expert demonstrations is limited. This paper presents an effective approach named Sequential IMItation LEarning (SIMILE). The core idea is to introduce sequential information, so that an agent can refer to both the current state and past state-action pairs to make a decision. We formulate our approach into a recurrent model, and instantiate it using LSTM so as to fuse both long-term and short-term information. SIMILE is a generalized IL framework which is easily applied to BL and IRL, two major types of IL algorithms. Experiments are performed on several robot controlling tasks in OpenAI Gym. SIMILE not only achieves performance gain over the baseline approaches, but also enjoys the benefit of faster convergence and better stability of testing performance. These advantages verify a higher learning efficiency of SIM-ILE, and implies its potential applications in real-world scenarios, i.e., when the agent-environment interaction is more difficult and/or expensive.
The paper proposes to integrate sequential information into imitation learning techniques.  The assumption is that mostly all the IL techniques are learning a policy which depends on state at time t, while the information contained in this state may be not sufficient to choose the right action (actually, this is the POMDP setting, the notion of POMDP not appearing in the paper....). The authors thus propose to use a recurrent neural network to encode the state by aggregating past information, instead of just using the features of the state at time t. They thus instantiate this idea on different methods and show that, on some problems, this approach can increase the quality of the final policy.
Actually, the contribution of the paper is a simple extension of existing methods: using a RNN instead of a simple NN in imitation learning models. First of all, when dealing with classical environments such as Atari, many papers propose to use the last N frames as a state encoding (instead of the last frame), following the same intuition. The studied setting thus corresponds to the PO-MDP case and using a RNN in POMDP is for example what is done in  [Merel etal. 2017]. Moreover, the problem of imitation learning (and particularly inverse RL) in POMDP has been of the interest of many papers like [Choi et al. 2008] for instance and many more, and it is unclear what is the positioning of this paper w.r.t existing works. Since the paper proposes just to encode history with a RNN, the proposed solution lacks of originality, and the contribution of the paper in term of model is quite low.  But the authors explain how this can be instantiated in three different settings (IRL, GAIL and BC) -- note that the section concerning the use of Adaboost is not clear and could be better described -- which can be of the interest of the community. 
Concerning the experiments, I don't understand what is the split between training and testing data. Is it pairs of state-action coming from the experts ? or trajectories ? Moreover, I don't understand why these environments correspond to POMDP cases and the authors have to give details on that. For instance, mountain-car is clearly not a POMDP problem in its classical shape, nor Acrobot. As if, it makes the experiments very difficult to reproduce. The interest of using the RNN to encode history does not seem clear for each of the cases since it often degrades the final performance, so I don't know exactly what insights I can extract from the paper.
Pro:
* The approach is proposed for IRL, GAIL and BC
Cons:
* Lack of positionning w.r.t POMDP litterature
* Lack of details in the experiments, and lack of good experimental results
* Low contribution in term of model
[Merel et al. 2017]  Learning human behaviors from motion capture
by adversarial imitation
[Choi et al.] Inverse Reinforcement Learning in Partially Observable
Environments