We present DiffSim, a new differentiable programming language tailored for building high-performance differentiable physical simulators. Based on an imperative programming language, DiffSim generates gradients of simulation steps using source code transformations that preserve arithmetic intensity and parallelism. A light-weight tape is used to record the whole simulation program structure and replay the gradient kernels in a reversed order, for end-to-end backpropagation. We demonstrate the performance and productivity of our language in gradientbased learning and optimization tasks on 10 different physical simulators. For example, a differentiable elastic object simulator written in our language is 4.2× shorter than the hand-engineered CUDA version yet runs as fast, and is 188× faster than the TensorFlow implementation. Using our differentiable programs, neural network controllers are typically optimized within only tens of iterations. Figure 1: Left: Our language allows us to seamlessly integrate a neural network (NN) controller and a physical simulation module, and update the weights of the controller or the initial state parameterization (blue). Our simulations typically have 512 ∼ 2048 time steps, and each time step has up to one thousand parallel operations. Right: 10 differentiable simulators built with DiffSim.
This paper introduces DiffSim, a programming language for high-performance differentiable physics simulations. The paper demonstrates 10 different simulations with controller optimization. It shows that the proposed language is easier to use and faster than the other alternatives, such as CUDA and TensorFlow. At the end, the paper provides insightful discussions why the gradient of the simulation could be wrong.
Differentiable physics simulation is an important research area, especially for optimal control and reinforcement learning. While I am impressed by the large variety of examples demonstrated in the paper, I am leaning towards rejecting the paper because of its poor presentation. The paper only gives a simple and high-level example of the language (optimizing the rest length of springs that form a triangle), very brief descriptions of 10 examples and some discussions about the difficulty of computing useful gradients, but without any in-depth discussion how everything is implemented. This is not enough for an ICLR paper. For example, the paper does not answer some of the fundamental problems of differentiable physics. For example, collision and contact are inherently non-differentiable. How does the paper handle it in the examples of locomotion and billiards (Figure 4)? In addition, how does the paper back-propagate the gradient through the incompressibility conditions (Poisson solve) of fluid simulation? 
Here is my suggestions how to improve the writing. There are several ways to write the paper, with different emphasis. If this paper is more about introducing a new programming language, Appendix B Compiler Design and Implementation would be important and should be moved to main text. If the paper want to emphasize how to handle the non-differentiable cases of the simulation, then detailed derivations of contact, collision, and linear/nonlinear solving (due to incompressibility conditions or implicit integrators) should be presented. If the paper would like to demonstrate how differentiable physics simulation can help with controller optimization, then two to three examples, such as the locomotion control for soft bodies or rigid bodies, should be analyzed in far more details, and compared with traditional method without differentiable simulation. It is good to focus on one of the above points, based on the venue that this paper is submitted to. Currently, the paper is trying to touch all three. But due to the page limit, it is not thorough, or detailed in any one of them.
-------------------------Update after rebuttal------------------------------
Thank you for the revision of the paper and the additional comparisons with Jax. The revised version reads much better. The response and the revision addressed most of my concerns. Thus, I raised my rating to weak accept.