Adversarial Examples that Fool both Computer Vision and Time-Limited Humans
Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans are prone to similar mistakes. Here, we address this question by leveraging recent techniques that transfer adversarial examples from computer vision models with known parameters and architecture to other models with unknown parameters and architecture, and by matching the initial processing of the human visual system. We find that adversarial examples that strongly transfer across computer vision models influence the classifications made by time-limited human observers.
This paper introduces an interesting experiment that tries to show that adversarial examples that transfer across recently proposed deep learning models can influence the visual classification produced by time-limited humans (at most 2.2 to 2.5 seconds for looking at the image and making the classification).  The idea is to take a set of 38 observers to classify images into 2 classes from within each of three separate groups (pets - cat or dog, hazard - snake or spider, vegetables - broccoli or cabbage).  The images are presented in different ways: 1) original image, 2) adversarial image manipulated to be classified by an ensemble classification method (of 10 recently proposed deep learning models), 3) flip, which is the original image manipulated by a vertically flipped adversarial perturbation (this is a control case to show that low signal-to-noise ratio alone does not explain poor human classification), and 4) false, which is an image outside the three groups above, manipulated by an adversarial perturbation to be classified as one of the classes within one of the three groups.  In general, results show that 1) adversarial perturbations in the false class successfully biased human classification towards the target class; and 2) adversarial perturbations cause observers to select the incorrect class even when the correct class is available (this is the 2nd manipulation described above).  I found this paper quite interesting to read and I support its publication.  I have a couple of comments that may influence the final writing of the paper.  Can there be an error due to mistakes in pressing the buttons given that the assignment of classes to buttons was randomized?  Was there some calibration phase before each experiment to rule that issue out?  Another experiment would be to measure how humans perform as time limit increases or decreases?