Meta-reinforcement learning aims to learn fast reinforcement learning (RL) procedures that can be applied to new tasks or environments. While learning fast RL procedures holds promise for allowing agents to autonomously learn a diverse range of skills, existing methods for learning efficient RL are impractical for real world settings, as they rely on slow reinforcement learning algorithms for metatraining, even when the learned procedures are fast. In this paper, we propose to learn a fast reinforcement learning procedure through supervised imitation of an expert, such that, after meta-learning, an agent can quickly learn new tasks through trial-and-error. Through our proposed method, we show that it is possible to learn fast RL using demonstrations, rather than relying on slow RL, where expert agents can be trained quickly by using privileged information or off-policy RL methods. Our experimental evaluation on a number of complex simulated robotic domains demonstrates that our method can effectively learn to learn from spare rewards and is significantly more efficient than prior meta reinforcement learning algorithms.
This work addresses the problem of learning a policy-learning-procedure, through meta-learning, that can adapt quickly to new tasks. This work uses MAML for meta-learning, and with this choice, the problem can be broken down into two loops: 
1) inner loop: adapting a policy \pi_phi based on unseen rollouts, where initial parameters phi were provided by the meta-trainer in the outer loop 
2) outer loop: the meta-trainer tries to learn parameters phi on batches of tasks that provide good initial parameters 
In prior work on meta-reinforcement learning via MAML, both the outer as well as inner objective attempt to minimize a RL objective, leading to an algorithm that has very high sample-complexity. This work uses imitation learning for the outer loop procedure, to significantly decrease sample-complexity.
Technical Contribution:
-----------------------
The idea of using imitation learning for reinforcement learning is well explored in the literature, and so using this idea in itself is not real contribution. There are several issues with the presentation of this work, that make it incredibly difficult to identify a technical contribution:
1. overreaching statements without details to backup: you are writing the paper as if you are learning a "RL algorithm" that can be used to quickly learn new tasks. your manuscript does not really provide a description for this "algorithm". After re-reading several other papers I concluded that what you mean is that you learn an initial set of policy parameters that can quickly adapt to new related tasks and an update rule with which you update these parameters. However, standard MAML uses SGD as an update rule so there is really nothing to be learned here. Unfortunately, your paper provides zero detail on these claims of learning a "RL procedure", so for now I have to assume that you are simply learning a good initial set of policy parameters through meta-learning. If that is the case, then using imitation learning in this setting is really not novel, this has been done by a lot of other people before (you're just using MAML to learn "better" initial parameters).
2. you're technical section (section 4) provides some details on the technical challenges of using demonstrations to perform the outer loop optimization step. Unfortunately, you are not putting your work in the context of existing work ([1], [2]), that discuss and address the importance/issue of sampling in meta-rl with MAML. So it's impossible to know whether there is any new insight here
Experimental Evaluation:
-------------------------
The experimental evaluation is very "thin", other than the original MAML-RL and pure imitation learning no other more recent baselines ([1], [2]) have been compared to. And only 2 relatively simple simulation settings are tested. 
Summary:
-----------
Very minor contribution, a manuscript that is lacking important details and does not relate it's technical section to existing work, with very thin evaluation. 
[1] The Importance of Sampling in Meta-Reinforcement Learning, NIPS 2018
[2] CONTINUOUS ADAPTATION VIA META-LEARNING IN NONSTATIONARY AND COMPETITIVE ENVIRONMENTS, ICLR 2018