LEARNING PARTICLE DYNAMICS FOR MANIPULATING RIGID BODIES, DEFORMABLE OBJECTS, AND FLUIDS
Real-life control tasks involve matters of various substances-rigid or soft bodies, liquid, gas-each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on approximation techniques, their simulation often deviates from real-world physics, especially in the long term. In this paper, we propose to learn a particle-based simulator for complex control tasks. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. We demonstrate robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam, with experiments both in simulation and in the real world. Our study helps lay the foundation for robot learning of dynamic scenes with particle-based representations.
The authors present an algorithm for learning the dynamics prediction of deformable and fluid bodies by modeling them as (potentially hierarchical) systems of many interacting particles.  This model applies a shared encoder to the particle states (positions and velocities), a shared relation network to nearby pairs, and a shared propagator network to the summed relation network outputs.  In some cases this process is applied in a multi-scale hierarchical fashion.  The authors demonstrate accurate rollouts of system dynamics and usefulness for manipulative control of deformable objects.
I find the motivation in the introduction persuasive and the algorithmic approach sound.  I also like the application to RL.  However, I do have some concerns, as follows:
1)  The novelty of the method is questionable.  Specifically, the hierarchical interaction network proposed here seems extremely similar to the prior (and cited) paper (Mrowca et al., 2018), which the authors do not directly compare against.  If there is a non-negligible difference between the two algorithms, then the authors should explicitly discuss the difference and empirically compare the two, in order to benefit others in the community who otherwise would not know which to use.
2)  The paper would benefit a lot from a diagram of the model.  Specifically, it would be good to have a diagram of the hierarchical interaction network demonstrating the multiscale propagation.  This could go in Figure 1, perhaps replacing elements (b) and (d) of the current Figure 1, which in my opinion are unnecessary and can be removed.
3)  The paper uses domain-specific hyperparameters, yet does not discuss or analyze the effects of these hyperparameters much.  Specifically, for this method to be useful to others, we would like to know how to choose (i) the propagation step L, (ii) the number of roots, and (iii) the neighborhood distance d.  In the paper, these numbers are chosen differently for the different environments without explanation.  Graphs showing performance on each task over a range of values of these parameters would be good (perhaps in the supplementary material).  Also, using the same hyperparameters for all environments (or at least a common generating function) would help support the generality of this model.
4)  The treatment of rigid bodies seems a bit hand-held.  Specifically, to determine the dynamics of rigid bodies, there is a ground-truth calculation which calculates computes the velocity and angular velocity of the body from the model predictions for its constituent particles.  Furthermore, if I understand correctly, there is a different motion predictor network for those particles in a rigid body than those in the surrounding fluid --- is this correct?  If so, this raises the questions:  (i) What happens if the same motion-predictor network is used for all particles, and (ii) What happens if the ground-truth rigid dynamics calculation is not done, so the model has to do all the work?  It would be interesting to have these as baselines.
5)  It would be nice to see more generalization results.  There is only one generalization experiment, testing for generalization over particle number in FluidShake.  However, the FluidShake model is not hierarchical.  The hierarchical models are a big emphasis in the paper, so showing generalization on BoxBath or RiceGrip would be much more meaningful.
6)  No confidence intervals for the quantitative results.  Confidence intervals would be good to see in the table in Figure 3-a.  Also, the bar graph in Figure 5 really would benefit from errorbars --- it is difficult to determine if the results are significant.
7)  While the text is generally clear and definitely understandable, I have a couple of comments about it:  (i) The last three paragraphs of the introduction are repetitive and I think they can be removed, or at least shortened a lot.  There are also quite a number of grammatical errors throughout the paper, though it is still comprehensible.
EDIT:  In their revision the authors addressed these concerns well and the paper is much more convincing (see longer comment below).  In light of this I have changed my rating from a 5 to an 8.