to exploit an input parse tree, the “neural programmer-interpreter” [ 22] is a composable network that
mimics the execution trace of a program, and the “spatial transformer” [ 11] learns to dynamically
modify network connectivity to capture certain types of interactions. Others have explored deep
learning of logical and arithmetic relations [ 27], and relations suitable for visual question-answering
[1].
The behavior of our model is similar in spirit to a physical simulation engine [ 2], which generates
sequences of states by repeatedly applying rules that approximate the effects of physical interactions
and dynamics on objects over time. The interaction rules are relation-centric, operating on two or
more objects that are interacting, and the dynamics rules are object-centric, operating on individual
objects and the aggregated effects of the interactions they participate in. A similar approach has been
developed independently by Chang et al. [4].
Previous AI work on physical reasoning explored commonsense knowledge, qualitative representations, and simulation techniques for approximating physical prediction and inference [ 29,9] . The
“NeuroAnimator” [ 8] was perhaps the ﬁrst quantitative approach to learning physical dynamics, by
training neural networks to predict and control the state of articulated bodies. Ladický et al. [ 14]
Reasoning about objects, relations, and physics is central to human
intelligence, and a key goal of artificial intelligence. Here we introduce the
interaction network, a model which can reason about how objects in complex
systems interact, supporting dynamical predictions, as well as inferences about
the abstract properties of the system. Our model takes graphs as input,
performs object- and relation-centric reasoning in a way that is analogous to a
simulation, and is implemented using deep neural networks. We evaluate its
ability to reason about several challenging physical domains: n-body problems,
rigid-body collision, and non-rigid dynamics. Our results show it can be
trained to accurately simulate the physical trajectories of dozens of objects
over thousands of time steps, estimate abstract quantities such as energy, and
generalize automatically to systems with different numbers and configurations
of objects and relations. Our interaction network implementation is the first
general-purpose, learnable physics engine, and a powerful general framework for
reasoning about object and relations in a wide variety of complex real-world
domains.
Peter W. Battaglia Razvan Pascanu Matthew Lai Danilo Rezende Koray Kavukcuoglu