Deep learning models are known to be vulnerable to adversarial examples. A practical adversarial attack should require as little as possible knowledge of attacked models. Current substitute attacks need pre-trained models to generate adversarial examples and their attack success rates heavily rely on the transferability of adversarial examples. Current score-based and decision-based attacks require lots of queries for the attacked models. In this study, we propose a novel adversarial imitation attack. First, it produces a replica of the attacked model by a two-player game like the generative adversarial networks (GANs). The objective of the generative model is to generate examples which lead the imitation model returning different outputs with the attacked model. The objective of the imitation model is to output the same labels with the attacked model under the same inputs. Then, the adversarial examples generated by the imitation model are utilized to fool the attacked model. Compared with the current substitute attacks, imitation attack can use less training data to produce a replica of the attacked model and improve the transferability of adversarial examples. Experiments demonstrate that our imitation attack requires less training data than the black-box substitute attacks, but achieves an attack success rate close to the white-box attack on unseen data with no query.
The authors propose to use a generative adversarial network to train a substitute that replicates (imitates) a learned model under attack. They then show that the adversarial examples for the substitute can be effectively used to attack the learned model. The idea is straightforward. The proposed approach leads to better success rates of attacking than other substitute-training approaches that require more training examples. Promising experimental results against decision and score-based attach schemes also demonstrate the effectiveness of the proposed approach.
My main concern is that the comparison to other approaches seem unfair, because
(1) Substitute models typically use all training data and query the learned model once per training example; the proposed approach uses fewer training data but needs 1800 queries per example. So it is not surprising that the proposed approach can be better than substitute models. As the authors focus on "practical" scenarios, the number of queries should be fixed as a constraint for all approaches to be fair.
(2) Having said (1), there is not enough detail in this paper to understand the similarity and difference between the proposed approach and decision-based ones (which is claimed to have similar query complexity during training). The authors mix the results with score-based (requiring more information), making the results more confusing to the readers. The authors are encouraged to present more detailed discussions on the comparison with decision-based competitors.
Also,
(3) One thing that is missing from the experiments is how well the replica clones the original model. All the information in the experiments are somewhat "indirect" (success rate, test accuracy, etc.) to answer this question, but there is no direct evidence. Is a well-cloned replica really a better substitute to construct adversarial examples? For instance, for replica with different "cloning accuracy" (rather than test accuracy), is a better replica also a better substitute? The paper fails to answer this question that best matches its design motivation.
The above are my main concerns. A minor one is
(4) The abstract that directly uses notations like T, G and D is horribly hard to read.