Published as a conference paper at ICLR 2018 UNDERSTANDING IMAGE MOTION WITH GROUP REPRESENTATIONS
Motion is an important signal for agents in dynamic environments, but learning to represent motion from unlabeled video is a difficult and underconstrained problem. We propose a model of motion based on elementary group properties of transformations and use it to train a representation of image motion. While most methods of estimating motion are based on pixel-level constraints, we use these group properties to constrain the abstract representation of motion itself. We demonstrate that a deep neural network trained using this method captures motion in both synthetic 2D sequences and real-world sequences of vehicle motion, without requiring any labels. Networks trained to respect these constraints implicitly identify the image characteristic of motion in different sequence types. In the context of vehicle motion, this method extracts information useful for localization, tracking, and odometry. Our results demonstrate that this representation is useful for learning motion in the general setting where explicit labels are difficult to obtain.
The paper presents a method that given a sequence of frames, estimates a corresponding motion embedding to be the hidden state of an RNN (over convolutional features) at the last frame of the sequence. The parameters of the motion embedding are trained to preserve properties of associativity  and invertibility of motion, where the frame sequences have been recomposed (from video frames) in various way to create pairs of frame sequences with those -automatically obtained- properties. This means, the motion embedding is essentially trained without any human annotations.
Experimentally, the paper shows that in synthetic moving MNIST frame sequences motion embedding discovers different patterns of motion, while it ignores image appearance (i.e., the digit label). The paper also shows that linear regressor trained in KITTI on top of the unsupervised motion embedding to estimate camera motion performs better than chance. 
Q to the authors: what labelled data were used to train the linear regressor in the KITTI experiment? 
Empirically, it appears that supervision by preserving group transformations may not  be immensely valuable for learning motion representations. 
Pros
1)The neural architecture for motion embedding computation appears reasonable
2)The paper tackles an interesting problem
Cons
1)For a big part of the introduction the paper refers to the problem of `` ````"learning motion” or `''understanding motion”  without being specific what it means by that. 
2)The empirical results are not convincing of the strength of imposing group transformations for self-supervised learning of motion embeddings.
3)The KITTI experiment is not well explained as it is not clear how this regressor was trained to predict egomotion out of the motion embedding.