We introduce a general approach, called test-time training, for improving the performance of predictive models when training and test data come from different distributions. Test-time training turns a single unlabeled test instance into a selfsupervised learning problem, on which we update the model parameters before making a prediction on this instance. We show that this simple idea leads to surprising improvements on diverse image classification benchmarks aimed at evaluating robustness to distribution shifts. Theoretical investigations on a convex model reveal helpful intuitions for when we can expect our approach to help.
The motivation is to increase accuracy of CNNs with unseen (unknown) distribution shifts. To this end, instead of the recent approach of training-time self-supervision, they adopted it for test-time [limited novelty]. More precisely, for a classification task, they considered two headed neural network with one head for main classification task and another for an auxiliary task (e.g. predicting rotation degree of rotated images). The feature extractor (up to k layers) is shared between these tasks thus updated using the two tasks, while two heads are updated according to each task. In test-time, the samples (drawn from shifted data distributions) are being used for updating the shared feature extractor through the auxiliary task. They have investigated their test-time approach in a series of experiments in online and offline settings on synthetic shift in data distribution of image classification tasks. The distributional shifts are synthesized adding some non-adversarial perturbation to clean images. The authors proof that their test-time approach can lead to lower error rate on given test-time samples when the underlying learning model is a linear regression model. 
The authors incorrectly interchange OOD with domain shifts in their manuscript. The common usage of OOD set is to capture novel samples that are not properly following the training set distribution, e.g. from different concepts than the set of classes given in the training set. With an OOD set, a robust model dealing with it should be able to detect whether an instance is in-distribution or OOD, making a special decision for the latter case (e.g., rejecting the instance).
In domain shift, we look at the scenario where test objects are the same as used for training the model (i.e., correspond to one of the classes the model is processing), but might be perturbed or coming having a different distribution (e.g. SVHN for MNIST or vice versa). The approaches for domain shift concern to improve robustness of CNNs to such shifts in data distribution. Accordingly, the title of the paper inaccurately reflects of the claim of the paper and is misleading, this paper is not on learning with out-of-distribution instances.
The other important point is about catastrophic forgetting phenomena in online setting of their approach, which was not addressed thoroughly in the paper. How not to forget what the model has previously learnt a test-time training? I see this somewhat has been empirically shown in Fig 2 with accuracy on the original data, but what is the mechanism not to forget what have being learned so far?
Besides generalization enhancement, the advantage of test-time self-supervision over training-time joint self-supervised is not clear for the readers, particularly considering the problem of the pitfall of catastrophic forgetting phenomena in test-time training. This pitfall does not exist for training-time joint self-supervised approach. What are the advantages of this approach?
The claims (about synthetic shift in distribution shift) are well supported in a series of experiments, where the distribution shifts are synthesized using adding some non-adversarial perturbation to clean images. However, the other common experiments on real shifts in distribution (e.g. SVHN for MNIST and vice versa, and those performed in Volpi et al  2018) are missing, which can help the paper to being better supported and justified for its practicality.
[Volpi et al  2018]: Generalizing to Unseen Domains via Adversarial Data Augmentation, NIPS 2018
I found the paper rather difficult to follow and not very coherent in its organization. The main idea is fundamentally simple, but it is still difficult to get it from the text. It needed me 2-3 readings before really getting the point of the paper.
** Update ** I read other reviews and comments. The answer of the authors to my comments are somehow satisfactory, especially the point of changing from "out-of-distribution" to "domain shift", which avoid some confusion. I upgraded my rating to a "weak accept".