Deep learning has become the state of the art approach in many machine learning problems such as classification. It has recently been shown that deep learning is highly vulnerable to adversarial perturbations. Taking the camera systems of self-driving cars as an example, small adversarial perturbations can cause the system to make errors in important tasks, such as classifying traffic signs or detecting pedestrians. Hence, in order to use deep learning without safety concerns a proper defense strategy is required. We propose to use ensemble methods as a defense strategy against adversarial perturbations. We find that an attack leading one model to misclassify does not imply the same for other networks performing the same task. This makes ensemble methods an attractive defense strategy against adversarial attacks. We empirically show for the MNIST and the CIFAR-10 data sets that ensemble methods not only improve the accuracy of neural networks on test data but also increase their robustness against adversarial perturbations.
In this manuscript, the authors empirically investigated the robustness of some different deep neural networks ensembles to two types of attacks, namely FGSM and BIM, on two popular datasets, MNIST and CIFAR10. The authors concluded that the ensembles are more accurate on both clean and adversaries samples than a single deep neural network. Therefore, the ensembles are more robust in terms of the ability to correctly classify the adversary attacks.
As the authors stated, an attack that is designed to fool one network does not necessarily fool the other networks in the same way. This is likely why ensembles appear more robust than single deep learners. However, robustness of ensembles to the white-box attacks that are generated from the ensemble is still low for FGS. Generally speaking, although FGS attacks generated from one network can fool less the whole ensembles, generating FGS adversaries from a given ensemble is still able to effectively fool it. Therefore, if the attacker has access to the ensemble or even know the classification system based on that ensemble, then the ensemble-based system is still vulnerable to the attacks generated specifically from it. Simple ensemble methods are not likely to confer significant robustness gains against adversaries.
In contrast to FGS results, surprisingly BIM-Grad1 is able to fool more the ensemble than BIM-Grad2. Therefore, it seems that if the attacker makes BIM adversaries from only a single classifier, then she can simply and yet effectively mislead the whole ensemble. In comparison to BIM-Grad2, BIM-Grad1 results show that BIM attacks from one network (BIM-Grad1) can more successfully fool the other different networks in the ensembles in a similar way! BIM-Grad2 is not that much able to fool the ensemble-based system even this attack generated from the ensemble (white-box attacks). In order to confirm the robustness of the ensembles to BIM attacks, the authors can do more experiments by generating BIM-Grad2 attacks with higher number of iterations.
Indeed, the low number of iterations might cause the lower rate of success for generating adversaries by BIM-Grad2. In fact, BIM adversaries from the ensembles might require more number of iterations to effectively fool the majority of the members in the ensembles. Therefore, increasing the number of iterations can increase the successful rate of generating BIM-Average Grad2 adversaries. Note that in this case, it is recommended to compare the amount of distortion (perturbation) with different number of iterations in order to indicate the effectiveness of the ensembles to white-box BIM attacks.
Despite to averaging the output probabilities to compute the ensemble final prediction, the authors generated the adversaries from the ensemble by computing the sum of the gradients of the classifiers loss. A proper approach would have been to average of these gradients. The fact the sum is not divided by the number of members (i.e., sum of gradients instead of average of gradients) is increasing the step size of the adversarial method proportionally to the ensemble size, raising questions on the validity of the comparison with the single-model adversarial generation.
Overall, I found the paper as having several methodological flaws in the experimental part, and rather light in terms of novel ideas. As noticed in the introduction, the idea of using ensemble for enhancing robustness as already been proposed. Making a paper only to restate it, is too light for acceptation. Moreover, experimental setup using a lot of space for comparing results on standard datasets (i.e., MNIST and CIFAR10), even with long presentation of these datasets. Several issues are raised in the current experiments and require adjustments. Experiments should also be more elaborated to make the case stronger, following at least some of indications provided.