SAMPLE EFFICIENT IMITATION LEARNING FOR CON- TINUOUS CONTROL
The goal of imitation learning (IL) is to enable a learner to imitate expert behavior given expert demonstrations. Recently, generative adversarial imitation learning (GAIL) has shown significant progress on IL for complex continuous tasks. However, GAIL and its extensions require a large number of environment interactions during training. In real-world environments, the more an IL method requires the learner to interact with the environment for better imitation, the more training time it requires, and the more damage it causes to the environments and the learner itself. We believe that IL algorithms could be more applicable to real-world problems if the number of interactions could be reduced. In this paper, we propose a model-free IL algorithm for continuous control. Our algorithm is made up mainly three changes to the existing adversarial imitation learning (AIL) methods -(a) adopting off-policy actor-critic (Off-PAC) algorithm to optimize the learner policy, (b) estimating the state-action value using off-policy samples without learning reward functions, and (c) representing the stochastic policy function so that its outputs are bounded. Experimental results show that our algorithm achieves competitive results with GAIL while significantly reducing the environment interactions.
The paper proposes a method for imitation learning via inverse reinforcement learning based on a specific modeling of the reward. It is modeled as the log probability of a state action pair to belong to the expert policy. It models this distribution as a Bernoulli one and thus it reduces the IRL problem to a classification task. The global method also uses an off-policy algorithm to learn the value function of the current agent policy to improve sample efficiency. The method is tested on a set of continuous control tasks such as walker, hopper or humanoid. 
I think the paper has several flaws. First, I found the paper not very well written and organized. It is hard to read. It uses some terminology in a way that is different from the rest of the littérature (such as Q-learning as learning the Q-function of the expert policy instead of using the  optimal Bellman operator (even if the expert is supposed to be optimal)). I also think that the related work section is missing a lot of important refs because it really focuses on recent papers while imitation learning has a long history. 
Yet, my main concern is that the proposed method seems to reduce to a classification problem to me and is likely to suffer from the same issues than the supervised learning method (AKA behavior cloning). It probably overfits a lot and there is nothing in the experiments that shows how robust is the method to perturbations. In a discrete world, this method would ideally place a reward of 1 in every state visited by the expert and 0 elsewhere which is very likely to overfit and result in unstable behaviors in the presence of noise etc. I would like to see experiments showing robustness. 
The experiments are also a bit strange since the learning is stopped early for the proposed method. Is it because the learning is unstable ?