2. Using temporal relationships in video: While the focus of this review is on image
(and not video) processing, a range of specialized methods have been developed for learning
single-image representations by pre-training on videos. Note that information restoration
methods are particularly useful for videos, which contain multiple modalities of information
that can be masked. Wang and Gupta [2015] pre-train a model using a triplet loss that
promotes similarities between representations of an object in two diﬀerent frames. The
resulting model performed well for object detection. Pathak et al. [2017] trains a model to
predict the motion of objects in a single frame, and adapts the resulting features to solve
single-frame detection problems. Agrawal et al. [2015] predicts the ego-motion of a camera
given multiple frames. Owens et al. [2016] propose to remove the audio track from a video,
and then predict the missing sound. For specialized applications like depth mapping,
A Cookbook of Self-Supervised Learning