Published as a conference paper at ICLR 2018 CHARACTERIZING ADVERSARIAL SUBSPACES USING LOCAL INTRINSIC DIMENSIONALITY
Deep Neural Networks (DNNs) have recently been shown to be vulnerable against adversarial examples, which are carefully crafted instances that can mislead DNNs to make errors during prediction. To better understand such attacks, a characterization is needed of the properties of regions (the so-called 'adversarial subspaces') in which adversarial examples lie. We tackle this challenge by characterizing the dimensional properties of adversarial regions, via the use of Local Intrinsic Dimensionality (LID). LID assesses the space-filling capability of the region surrounding a reference example, based on the distance distribution of the example to its neighbors. We first provide explanations about how adversarial perturbation can affect the LID characteristic of adversarial regions, and then show empirically that LID characteristics can facilitate the distinction of adversarial examples generated using state-of-the-art attacks. As a proof-of-concept, we show that a potential application of LID is to distinguish adversarial examples, and the preliminary results show that it can outperform several state-of-the-art detection measures by large margins for five attack strategies considered in this paper across three benchmark datasets . Our analysis of the LID characteristic for adversarial regions not only motivates new directions of effective adversarial defense, but also opens up more challenges for developing new attacks to better understand the vulnerabilities of DNNs.
The paper considers a problem of adversarial examples applied to the deep neural networks. The authors conjecture that the intrinsic dimensionality of the local neighbourhood of adversarial examples significantly differs from the one of normal (or noisy) examples. More precisely, the adversarial examples are expected to have intrinsic dimensionality much higher than the normal points (see Section 4).  Based on this observation they propose to use the intrinsic dimensionality as a way to separate adversarial examples from the normal (and noisy) ones during the test time. In other words, the paper proposes a particular approach for the adversarial defence.
It turns out that there is a well-studied concept in the literature capturing the desired intrinsic dimensionality: it is called the local intrinsic dimensionality (LID, Definition 1) . Moreover, there is a known empirical estimator of LID, based on the k-nearest neighbours. The authors propose to use this estimator in computing the intrinsic dimensionalities for the test time examples. For every test-time example X the resulting Algorithm 1 computes LID estimates of X activations computed for all intermediate layer of DNN. These values are finally used as features in classifying adversarial examples from normal and noisy ones. 
The authors empirically evaluate the proposed technique across multiple state-of-the art adversarial attacks, 3 datasets (MNIST, CIFAR10, and SVHN) and compare their novel adversarial detection technique to 2 other ones recently reported in the literature. The experiments support the conjecture mentioned above and show that the proposed technique *significantly* improves the detection accuracy compared to 2 other methods across all attacks and datasets (see Table 1).
Interestingly, the authors also test whether adversarial attacks can bypass LID-based detection methods by incorporating LID in their design. Preliminary results show that even in this case the proposed method manages to detect adversarial examples most of the time. In other words, the proposed technique is rather stable and can not be easily exploited.
I really enjoyed reading this paper. All the statements are very clear, the structure is transparent and easy to follow. The writing is excellent. I found only one typo (page 8, "We also NOTE that..."), otherwise I don't actually have any comments on the text.
Unfortunately, I am not an expert in the particular field of adversarial examples, and can not properly assess the conceptual novelty of the proposed method. However, it seems that it is indeed novel and given rather convincing empirical justifications, I would recommend to accept the paper.