  Self-supervised representation learning (SSRL) methods have shown great
success in computer vision. In recent studies, augmentation-based contrastive
learning methods have been proposed for learning representations that are
invariant or equivariant to pre-defined data augmentation operations. However,
invariant or equivariant features favor only specific downstream tasks
depending on the augmentations chosen. They may result in poor performance when
the learned representation does not match task requirements. Here, we consider
an active observer that can manipulate views of an object and has knowledge of
the action(s) that generated each view. We introduce Contrastive Invariant and
Predictive Equivariant Representation learning (CIPER). CIPER comprises both
invariant and equivariant learning objectives using one shared encoder and two
different output heads on top of the encoder. One output head is a projection
head with a state-of-the-art contrastive objective to encourage invariance to
augmentations. The other is a prediction head estimating the augmentation
parameters, capturing equivariant features. Both heads are discarded after
training and only the encoder is used for downstream tasks. We evaluate our
method on static image tasks and time-augmented image datasets. Our results
show that CIPER outperforms a baseline contrastive method on various tasks.
Interestingly, CIPER encourages the formation of hierarchically structured
representations where different views of an object become systematically
organized in the latent representation space.

Computer Vision and Pattern Recognition;Artificial Intelligence