BABYAI: A PLATFORM TO STUDY THE SAMPLE EFFI- CIENCY OF GROUNDED LANGUAGE LEARNING
Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific reasons. Though, given the lack of sample efficiency in current learning methods, reaching this goal may require substantial research efforts. We introduce the BabyAI research platform, with the goal of supporting investigations towards including humans in the loop for grounded language learning. The BabyAI platform comprises an extensible suite of 19 levels of increasing difficulty. Each level gradually leads the agent towards acquiring a combinatorially rich synthetic language, which is a proper subset of English. The platform also provides a hand-crafted bot agent, which simulates a human teacher. We report estimated amount of supervision required for training neural reinforcement and behavioral-cloning agents on some BabyAI levels. We put forward strong evidence that current deep learning methods are not yet sufficiently sample-efficient in the context of learning a language with compositional properties.
This paper focuses on grounded language learning with a human in the loop, in the sense where the language is synthetic, the environment is a 2D grid world, and the human is a simulated human teacher implemented using heuristics. This setup is dubbed the BabyAI platform, and includes curriculum learning over 19 levels of increasing difficulty.  
Overall, the BabyAI platform is conceptually similar to numerous previous works that seek to learn grounded language in simulation environments. These efforts differ along various axes, for example visual realism, 2D vs 3D, partially vs. fully observed, different tasks, world-state manipulation or not, etc. The main original aspect of the BabyAI platform is the simulated human-teacher. 
Strengths
- Learning with a human in the loop is an extremely important problem to study, although currently efforts are hampered by cost, lack or reproducibility, and the sample inefficiency of existing learning methods. This paper addresses all three of these issues, albeit by removing the human and natural language. This is simultaneously the greatest weakness of this approach. The contribution of this paper therefore rests on the quality/interestingness/utility of the provided synthetic language and the synthetic teacher.
- Fortunately, the synthetic language does exhibit interesting compositional properties, it is readily extensible, it has the appealing property that it can be readily interpreted as a subset of english, and it is accompanied by a verifier to check if the specified actions were completed. 
Weaknesses
- If the ultimate goal is learning with a human in the loop, the usefulness of the synthetic teacher is not clear, particularly as it is apparently easier to imitate from an RL trained agent than the teacher. The explanation 'This can explained by the fact that the RL expert has the same neural network architecture as the learner' does no seem obvious to me. 
- Regarding the human in the loop, since this is aspirational and not an aspect of the paper, the title of the paper does not seem reflective of its content (even with the 'First steps' qualifier). 
- If the main unique aspect is the simulated human-teacher, it is not clear why it is necessary to create a new environment, rather than re-using an existing environment. The effect of this is to limit comparisons with recent work and an increasing fragmentation of research across tasks that are related but can’t be compared.
Summary:
This paper represents an important direction, in that it provides a testbed for studying the sample efficiency of grounded language learning in a simplified (yet still challenging and compositional) environment. I believe the environment and the provided synthetic language and verifier will prove useful to the community, and despite some reservations about the title and the simulated human-teacher, I recommend acceptance.