Learning to Poke by Poking: Experiential Learning of Intuitive Physics
We investigate an experiential learning paradigm for acquiring an internal model of intuitive physics. Our model is evaluated on a real-world robotic manipulation task that requires displacing objects to target locations by poking. The robot gathered over 400 hours of experience by executing more than 100K pokes on different objects. We propose a novel approach based on deep neural networks for modeling the dynamics of robot's interactions directly from images, by jointly estimating forward and inverse models of dynamics. The inverse model objective provides supervision to construct informative visual features, which the forward model can then predict and in turn regularize the feature space for the inverse model. The interplay between these two objectives creates useful, accurate models that can then be used for multi-step decision making. This formulation has the additional benefit that it is possible to learn forward models in an abstract feature space and thus alleviate the need of predicting pixels. Our experiments show that this joint modeling approach outperforms alternative methods.
This paper presents an approach on how to learn to arange four objects by poking them with a robotic arm from visual input alone. The authors's model-based method jointly learns the forward and inverse dynamics. Instead of reconstructing images, the forward model predicts the feature vector of a layer in the deep neural network, that is supposedly more abstract due to the regularizing nature of the inverse model. They evaluate the policies defined by the trained inverse models on a simulation and on a lot of real-world data by measuring relative location error obtained through template matching. The paper concludes with a short section about active data collection instead of random sampling.
Pros: + A good idea to move away from generative models when they're not necesary + Real-world experiment with lots of data provides credible proof of concept  Cons: - 'Strong' baseline is undefined. It's hard to judge the performance especially given the variances.  - Unclear evaluation metric, because location and position terms seem to be mixed (l. 211, 212, 215 and Fig. 5 caption) and the influence of rotation is not clear. How exactly are the distances between object locations defined (d_ft and d_it) in terms of position and angle? - Using the median might indicate outliers that are not being discussed.  Questions / Suggestions: - Separated evaluation in terms of position and angle might be useful. The shown trajectories seem to get the position right, but the angle is in some cases quite wrong. It would also be interesting to see if the angle is present in the representation x_t by comparing the activations while rotating the objects for the same position. - The claim in line 114 that, given x and u, a forward model is unique, only holds for single-agent Markov environments. What happens if the robot's arm is occluding the object?  Minor issues: - The 'intuitive' part of intuitive physics is not clearly explained in the introduction. - Lots of language errors. Random or additional 'the' in many sentences (132, 208, 226, Fig4 caption). - The three streams in Figure 3 are not easily identifiable. - The oracle evaluation is explained in line 264, after everything has already been discussed. It might make sense to do this beforehand. - Reference [10] is Sascha Lange, as in [9].