Workshop track -ICLR 2018 BUILDING GENERALIZABLE AGENTS WITH A REALIS- TIC AND RICH 3D ENVIRONMENT
Teaching an agent to navigate in an unseen 3D environment is a challenging task, even in the event of simulated environments. To generalize to unseen environments, an agent needs to be robust to low-level variations (e.g. color, texture, object changes), and also high-level variations (e.g. layout changes of the environment). To improve overall generalization, all types of variations in the environment have to be taken under consideration via different level of data augmentation steps. To this end, we propose House3D, a rich, extensible and efficient environment that contains 45,622 human-designed 3D scenes of visually realistic houses, ranging from single-room studios to multi-storied houses, equipped with a diverse set of fully labeled 3D objects, textures and scene layouts, based on the SUNCG dataset . The diversity in House3D opens the door towards scene-level augmentation, while the label-rich nature of House3D enables us to inject pixel-& task-level augmentations such as domain randomization (Tobin et al., 2017) and multi-task training. Using a subset of houses in House3D, we show that reinforcement learning agents trained with an enhancement of different levels of augmentations perform much better in unseen environments than our baselines with raw RGB input by over 8% in terms of navigation success rate. House3D is publicly available at
Building rich 3D environments where to run simulations is a very interesting area of research. 
Strengths:
1.	The authors propose a virtual environment of indoor scenes having a much larger scale compared to similar interactive environments and access to multiple visual modalities. They also show how the number of available scenes greatly impacts generalization in navigation based tasks. 
2.	The authors provide a thorough analysis on the contribution of different feature types (Mask, Depth, RGB) towards the success rate of the goal task. The improvements and generalization brought by the segmentation and depth masks give interesting insights towards building new navigation paradigms for real-world robotics. 
Weaknesses:
1.	The authors claim that the proposed environment allows for multiple applications and interactions, however from the description in section 3, the capacities of the simulator beyond navigation are unclear.
The dataset proposed, Home3D, adds a number of functionalities over the SUNCG dataset. The SUNCG dataset provides a large number of 3D scanned houses.  The most important contributions with respect to SUNCG are:
- An efficient renderer: an important aspect.
- Introducing physics: this is very interesting, unfortunately the contribution here is very small. Although I am sure the authors are planing to move beyond the current state of their implementation, the only physical constraint currently implemented is an occupancy rule and collision detection. This is not technically challenging. 
Therefore, the added novelty with respect to SUNCG is very limited.
2.	The paper presents the proposed task as navigation from high level task description, but given that the instructions are fixed for a given target, there are only 5 possible instructions which are encoded as one-hot vectors. Given this setting, it is unclear the need for a gated attention mechanism. While this limited setting allows for a clear generalization analysis, it would have been good to study a setting with more complex instructions, allowing to evaluate instructions not seen during training.
3.	While the authors make a good point showing generalization towards unseen scenes, it would have been good to also show generalization towards real scenarios, demonstrating the realistic nature of House3D and the advantages of using non-RGB features.
4.	It would have been good to report an analysis on the number of steps performed by the agent before reaching its goal on the success cases. It seems to me that the continuous policy would be justified in this setting. 
Comments
-	It is unclear to me how the reward shaping addition helps generalize to unseen houses at test time, as suggested by the authors.
-	I miss a reference to (https://arxiv.org/pdf/1609.05143.pdf) beyond the AI-THOR environment, given that they also approach target driven navigation using an actor-critic approach.
The paper proposes a new realistic indoor virtual environment, having a much larger number of scenes than similar environments. From the experiments shown, it seems that the scale increase, together with the availability of features such as Segmentation and Depth improve generalization in navigation tasks, which makes it a promising framework for future work on this direction. However, the task proposed seems too simple considering the power of this environment, and the models used to solve the task don’t seem to bring  relevant novelties from previous approaches. (https://arxiv.org/pdf/1706.07230.pdf)