A common strategy in modern learning systems is to learn a representation which is useful for many tasks, a.k.a. representation learning. We study this strategy in the imitation learning setting for Markov decision processes (MDPs) where multiple experts' trajectories are available. We formulate representation learning as a bi-level optimization problem where the "outer" optimization tries to learn the joint representation and the "inner" optimization encodes the imitation learning setup and tries to learn task-specific parameters. We instantiate this framework for the imitation learning settings of behavior cloning and observation-alone. Theoretically, we provably show using our framework that representation learning can reduce the sample complexity of imitation learning in both settings. We also provide proof-of-concept experiments to verify our theoretical findings.
This paper theoretically explores reinforcement/imitation learning via representation learning. The key theoretical question being investigated is the relationship between representation learning in a multi-task/meta learning setup and its dependence to the sample/task complexity. The paper sets up the problem in bilevel optimization framework, where the inner optimization learns/optimizes task specific losses, while the outer optimization learns the representation used in the inner level tasks. The main takeaway from the two theorems (which are the core contributions of this paper) are that when the number of tasks is higher than the number of samples, representation learning can reduce the sample complexity. The paper explores two scenarios in imitation learning, namely behavioral cloning and when only the states of the experts are available (and not their actions). Some experiments are provided to empirically validate the theory.
Pros: 
1. The paper presents a theoretical investigation into multi-task/meta learning for RL via learning representations. While, the main theoretical contributions are perhaps marginal with regard to prior work, the problem setting (RL) seems novel and the two theorems in this context are interesting.
2. The paper is well-written and appears to be very rigorous. I did not check for the correctness of all technical parts. There are several "abuse of notations" in the main text, which sometimes impact the otherwise smooth read of the technical parts. 
3. A good and concise review of RL concepts is provided.
Cons:
1. The paper lacks a good literature review to place this work in the right context. For example, while the paper refers to the works of Maurer 2016 and Sun et al. 2019 at several places, it is not formally and clearly mentioned anywhere what are the similarities to these prior works and what are the new contributions. For example, Maurer 2016 proposes a multi-task learning setup using representation learning, and most of Theorem 4.1 in this paper is taken from the results in that paper. While, the current paper uses bilevel optimization setting in an RL context, it is not clear to me if this (bilevel + RL) setting has any significant bearing against the theoretical results furnished by Maurer 2016. For example, the theorems in this paper (as far as I see) show that the bounds are scaled by a constant defined by H^2, the trajectory length? If there is something beyond this, then the paper needs to explicitly point it out. The same comment goes with the results against Sun et al. 2019 in Theorem 5.1.
2. Given that the main goal of this paper is to connect sample complexity with representation learning, it is important the paper provide a theorem stating this precisely. Theorems 4.1 and 5.1 provide a general bound, and the sample complexity is being described in the explanations of this theorem, which is very informal. Also, against what is claimed in the abstract, it appears that representation learning helps reduce sample complexity only when the number of tasks are larger, which perhaps needs to be explicitly mentioned. Also, note that there is a bearing of the bound on the trajectory length H (Theorem 4.1, and 5.1). Shouldn't this factor be also accounted for when explaining the sample complexity? 
3.  There has been several recent works on model-agnostic meta-learning (that also uses bilevel optimization and implicit gradients), however, older works on meta-learning (only for imitation learning) have been cited. The paper should include more recent works in this area and contrast against their theoretical findings. 
Apart from these, below are some minor comments that could help improve the reading of this paper:
a. Theorem 3.1 is not really a theorem, since it is very informal. Also, fix the Theorem numbers. 
b. Bullet 1. after Theorem 3.1, \ell^x(\pi) concentrates to \ell^x(\pu*). Also, the mention about sample complexity here should be backed with some reference/citation.
c. Assumption 4.2: The notation \pi_\mu(s)_{\pi*(\mu(s)) is unclear, shouldn't the subscript contain an argmax over the actions for \pi*? 
d. Theorem 4.1 and 5.1, what does it mean by "probability 1-\delta over the choice of the dataset X" ? Also, \mu^n seems undefined.
e. The first two terms in Theorem 4.1 are claimed standard, provide the citations?
f. Theorem 4.1, perhaps use some other notation for c, which is defined as the cost/reward in the RL setting.
Overall, the paper has some interesting theoretical results, and is mathematically rigorous, however lacks a clear distinction from prior and more recent works in this area.