title: Testing Robustness Against Unforeseen Adversaries
abstract: Most existing adversarial defenses only measure robustness to $L_p$ adversarial attacks. Not only are adversaries unlikely to exclusively create small $L_p$ perturbations, adversaries are unlikely to remain fixed. Adversaries adapt and evolve their attacks; hence adversarial defenses must be robust to a broad range of unforeseen attacks. We address this discrepancy between research and reality by proposing a new evaluation framework called ImageNet-UA. Our framework enables the research community to test ImageNet model robustness against attacks not encountered during training. To create ImageNet-UA's diverse attack suite, we introduce a total of four novel adversarial attacks. We also demonstrate that, in comparison to ImageNet-UA, prevailing $L_\infty$ robustness assessments give a narrow account of adversarial robustness. By evaluating current defenses with ImageNet-UA, we find they provide little robustness to unforeseen attacks. We hope the greater variety and realism of ImageNet-UA enables development of more robust defenses which can generalize beyond attacks seen during training.
authors: Daniel Kang Yi Sun Dan Hendrycks Tom B Brown Jacob Steinhardt
hIndex: Jacob Steinhardt 31