Interaction Networks for Learning about Objects, Relations and Physics
Reasoning about objects, relations, and physics is central to human intelligence, and 1 a key goal of artificial intelligence. Here we introduce the interaction network, a 2 model which can reason about how objects in complex systems interact, supporting 3 dynamical predictions, as well as inferences about the abstract properties of the 4 system. Our model takes graphs as input, performs object-and relation-centric 5 reasoning in a way that is analogous to a simulation, and is implemented using 6 deep neural networks. We evaluate its ability to reason about several challenging 7 physical domains: n-body problems, rigid-body collision, and non-rigid dynamics.
The authors aim to develop a learning-based physics engine. The engine developed in the paper take as input an interaction graph. This graph has nodes corresponding to objects and edges corresponding to relations between the objects. At each step of the simulation, a trained MLP is used to predict the effects of inter-object interactions. Then, a separate trained MLP is used to predict the state of the object at the next time-step taking into account both the previous state of the object and the effects of its relationship with other objects. The authors train and test the proposed model on three different types of scenarios: n-body systems; balls bouncing in a box; and strings composed of springs that collide with rigid objects. They demonstrate accurate predictions and generalization ability of the model one time-step into the future.
                  A. Experiments         * It seems only one timestep into a future is explored (lines 252-253). The authors should have tried to evaluate the model over a longer timescale. Especially in the case of balls bouncing in a box, it seems much harder to correctly predict the future positions as errors in positions tend to accumulate over the tilmestep.         * The qualitative results (Fig. 2) indicates the model inaccurately predicts over a longer timescale - a property rather important for a  physics engine.                  B. Model         * The presented model works under quite a strong assumptions where an 'interaction graph' is given. Works such as Fragkiadaki et al. [5] (ICLR'16) demonstrate similar abilities but under weaker modelling assumption (from a raw visual input).         * It also seems the model requires quite a lot of training data (1 million). Hence it is hard to tell if the model has any possibility to generalize outside of the 'simulation' domain.          C. Motivation         * It is unclear for me what is the main motivation/purpose behind the submission. If simulation and 'interaction graph' are available then arguably we can use 'laws of physics' without a need to learn anything. Although other works (e.g. [5]) still use simulation, the interactions are learnt from the data only, and hence such a line of research ([5]) may have an impact for real-world scenarios. Likewise, in real-world scenarios extracting an 'interaction graph' may become a quite difficult problem -- and such graph is likely to be 'corrupted by noise'. Such issues are not addressed in the submission.    ---  I liked the rebuttal, and modified my score a bit. However, there are things that still bother me.  Regarding the experiments:  The submission lacks of quantitative results on long-term predictions (either with MSE or by proposing an alternative measure). The MSE results given in Table 1 (and majority results in the supplementary material) already exhibit small errors. Hypothetical future work that builds upon the submission would arguable explore longer-term predictions, and need a comparison. Moreover, qualitative results may be good in the 'intuitive physics' domain, but to build 'physics engine' or to allow reproducibility/comparisons automatic measures showing deviations from the ground truth are also important.   I also think the Table from supp. should be moved to the main paper -- it shows quite important results.  It would also be great if the authors make the dataset / code publicly available.   Regarding the motivation/impact:  I agree with the authors that predicting functional forms in the interaction networks might be useful in many scenarios ranging from intuitive physics, learning engines of systems with unknown simulation, or social networks, but none of the aforementioned tasks have been explored in the submission. For instance, in the 'intuitive physics from images' domain, the authors have to extract a noisy graph representation of an image. Is IN competitive with the work Fragkiadaki et al. in this scenario? I see the experiments shown in the submission as valuable additional experiments done in a more controlled environment but they miss real applications. Therefore, I feel the work is somehow incomplete, but I also agree it has a good potential, and is a nice initial study in a controlled environment under the assumption that the graph is given.