While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is challenging and often requires substantial interactions with the environment. Further, a wide variety of domains have dynamics that share common foundations like the laws of physics, which are rarely exploited by these algorithms. Humans often acquire such physics priors that allow us to easily adapt to the dynamics of any environment. In this work, we propose an approach to learn such physics priors and incorporate them into an RL agent. Our method involves pre-training a frame predictor on raw videos and then using it to initialize the dynamics prediction model on a target task. Our prediction model, SpatialNet, is designed to implicitly capture localized physical phenomena and interactions. We show the value of incorporating this prior through empirical experiments on two different domainsa newly created PhysWorld and games from the Atari benchmark, outperforming competitive approaches and demonstrating effective transfer learning.
Quality: The paper proposed a new method to learn some physics prior in an environment along with a new SpatialNetwork Architecture. Instead of learning a specific dynamics model, they propose to learn a dynamics model that is action-free, purely learning the extrinsic dynamics.  They formulate this problem as a video prediction problem. A series of experiments are conducted on PhysWorld (a new physics based simulator) and a subset of Atari games.
Clarity: The writing is good.
Originality: This work is original as most of the model-based RL works are focusing on learning one environment instead of common rules of physics.
significance of this work: This work propose an interesting direction to pursue.
cons:
1. In Figure 4, the authors show that a pretrained model can learn faster than random initialization. However, it is hard to ablate the factor that causes this effect.  Does the dynamics predictor learn the physics priors or is it just because it learn the visual prior of the shape of the objects, etc? 
2. The baseline for atari games is quite limited. First of all, 3 out of 5  atari games  in the original PPO paper show that ACER performs better than PPO. (asteroid, breakout, DemonAttack). I think it is better to make some improvement upon state-of-the-art methods.
3. All the experiments are shown with only 3 random seeds, without error bar in the main paper. Although the reward plots are shown in Figure 11. 
4. 5 out of 10 atari games are similar to PPO (according to Figure 11). It's hard to be conclusive when half of the experiments are positive and the rest are not. 
5. Lack of discussion about ego-dynamics. There are physics priors for both the environment and the controller. Usually the controller/agent  requires an action to predict its dynamics. Then why should we omit the ego-dynamics and only model the outer world. 
6. Physics prior usually happen in physical environment. The proposed method works well in the physworld environments. But is there some task that are more realistic than atari games that can leverage the power of physics priors more? It's good that this method works in some atari games. But isn't learning the dynamics of atari games a bit off the topic? 
7. The transfer learning experiments should contain a baseline -- maml/reptile. Since you are learning physics prior, it is fair to add meta-learning baselines for comparison.
I think the direction is interesting and the effort is made well. But the experiments are less convincing than the abstract/introduction.