  The literature on robustness towards common corruptions shows no consensus on
whether adversarial training can improve the performance in this setting.
First, we show that, when used with an appropriately selected perturbation
radius, $\ell_p$ adversarial training can serve as a strong baseline against
common corruptions improving both accuracy and calibration. Then we explain why
adversarial training performs better than data augmentation with simple
Gaussian noise which has been observed to be a meaningful baseline on common
corruptions. Related to this, we identify the $\sigma$-overfitting phenomenon
when Gaussian augmentation overfits to a particular standard deviation used for
training which has a significant detrimental effect on common corruption
accuracy. We discuss how to alleviate this problem and then how to further
enhance $\ell_p$ adversarial training by introducing an efficient relaxation of
adversarial training with learned perceptual image patch similarity as the
distance metric. Through experiments on CIFAR-10 and ImageNet-100, we show that
our approach does not only improve the $\ell_p$ adversarial training baseline
but also has cumulative gains with data augmentation methods such as AugMix,
DeepAugment, ANT, and SIN, leading to state-of-the-art performance on common
corruptions.
  The code of our experiments is publicly available at
https://github.com/tml-epfl/adv-training-corruptions.

Machine Learning;Artificial Intelligence;Computer Vision and Pattern Recognition;Machine Learning