It is widely known that well-designed perturbations can cause state-of-the-art machine learning classifiers to mis-label an image, with sufficiently small perturbations that are imperceptible to the human eyes. However, by detecting the inconsistency between the image and wrong label, the human observer would be alerted of the attack. In this paper, we aim to design attacks that not only make classifiers generate wrong labels, but also make the wrong labels imperceptible to human observers. To achieve this, we propose an algorithm called LabelFool which identifies a target label similar to the ground truth label and finds a perturbation of the image for this target label. We first find the target label for an input image by a probability model, then move the input in the feature space towards the target label. Subjective studies on ImageNet show that in the label space, our attack is much less recognizable by human observers, while objective experimental results on ImageNet show that we maintain similar performance in the image space as well as attack rates to state-of-the-art attack algorithms.
This paper proposes a method for constructing adversarial attacks that are less detectable by humans, by changing the target class to be a class similar to the original class of the image. The resulting attack methodology is then studied in terms of its imperceptibility in label space, and shown to be less perceptible in label space to human observers, while not coming at a cost in image space.
The paper presents compelling evaluation of the method and does seem to succeed in proving that their proposed attack satisfies the stated goal. However, it appears as though this goal is somewhat counter to the main point of adversarial examples---indeed, if the label is reasonable to a human, then what makes the adversarial example adversarial? The main threat in adversarial examples research seems to be that it is possible to induce predictions that are arbitrarily different from humans' on natural-looking in puts. Thus, changing the label to something that a human actually agrees with would actually reduce the impact of the adversarial attack.
In order to improve the paper, I would suggest applying the same (or similar) methodologies to other areas of ML security where imperceptibility in label space is commonly desired---for example, in data poisoning attacks or backdoor attacks. In general, such attacks are much more likely to be "inspected" by humans, and so imperceptibility in both label and image space is very desirable. However, I suspect that this would require significant effort and changes to the paper, and so for now I recommend rejection.