Laplacian Networks: Bounding Indicator Function Smoothness for Neural Networks Robustness
For the past few years, Deep Neural Network (DNN) robustness has become a question of paramount importance. As a matter of fact, in sensitive settings misclassification can lead to dramatic consequences. Such misclassifications are likely to occur when facing adversarial attacks, hardware failures or limitations, and imperfect signal acquisition. To address this question, authors have proposed different approaches aiming at increasing the robustness of DNNs, such as adding regularizers or training using noisy examples. In this paper we propose a new regularizer built upon the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DNN architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. Since it is agnostic to the type of deformations that are expected when predicting with the DNN, the proposed regularizer can be combined with existing ad-hoc methods. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness of DNNs on classical supervised learning vision datasets.
 This paper proposes the interesting addition of a graph-based regularisers, in NNs architectures, for improving their robustness to different perturbations or noise. The regularisation enforces smoothness on a graph built on the different features at different layers of the NN system. The proposed ideas are quite interesting, and integrates nicely into NN architectures. A few paths for improvements: - the *optimal* choice of the power of the Laplacian, in 3.5, is eluded - the figures are not presented ideally, nor in a very readable form - for example, their are 90-degree rotated compared to classical presentations, and the plots are hardly readable - the might exist a tradeoff between robustness, and performance (accuracy), that seem to be explaining the proposed results (see Fawzi - Machine Learning 2018, for example) - in 4.2, what is a mean case of adversarial noise? Also, it would be good to see the effect of the regularizer of both the *original* network, and on the network trained with data augmentation. It is not clear which one is considered here, but it would be interesting to study both, actually. - the second paragraph of the conclusion (transfer of perturbations) opens interesting perspective, but the problem might not be as trivial as the authors seem to hint in the text. Overall, very interesting and nice work, which might be better positioned (especially in terms of experiments) wrt to other recent methods that propose to improve robustness in NNs.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Laplacian Networks: Bounding Indicator Function Smoothness for Neural Networks Robustness
For the past few years, Deep Neural Network (DNN) robustness has become a question of paramount importance. As a matter of fact, in sensitive settings misclassification can lead to dramatic consequences. Such misclassifications are likely to occur when facing adversarial attacks, hardware failures or limitations, and imperfect signal acquisition. To address this question, authors have proposed different approaches aiming at increasing the robustness of DNNs, such as adding regularizers or training using noisy examples. In this paper we propose a new regularizer built upon the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DNN architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. Since it is agnostic to the type of deformations that are expected when predicting with the DNN, the proposed regularizer can be combined with existing ad-hoc methods. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness of DNNs on classical supervised learning vision datasets.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper proposes the interesting addition of a graph-based regularisers, in NNs architectures, for improving their robustness to different perturbations or noise. The regularisation enforces smoothness on a graph built on the different features at different layers of the NN system. The proposed ideas are quite interesting, and integrates nicely into NN architectures. A few paths for improvements: - the *optimal* choice of the power of the Laplacian, in 3.5, is eluded - the figures are not presented ideally, nor in a very readable form - for example, their are 90-degree rotated compared to classical presentations, and the plots are hardly readable - the might exist a tradeoff between robustness, and performance (accuracy), that seem to be explaining the proposed results (see Fawzi - Machine Learning 2018, for example) - in 4.2, what is a mean case of adversarial noise? Also, it would be good to see the effect of the regularizer of both the *original* network, and on the network trained with data augmentation. It is not clear which one is considered here, but it would be interesting to study both, actually. - the second paragraph of the conclusion (transfer of perturbations) opens interesting perspective, but the problem might not be as trivial as the authors seem to hint in the text. Overall, very interesting and nice work, which might be better positioned (especially in terms of experiments) wrt to other recent methods that propose to improve robustness in NNs.
assistant
 This paper proposes the interesting addition of a graph-based regularisers, in NNs architectures, for improving their robustness to different perturbations or noise. The regularisation enforces smoothness on a graph built on the different features at different layers of the NN system. The proposed ideas are quite interesting, and integrates nicely into NN architectures. A few paths for improvements: - the *optimal* choice of the power of the Laplacian, in 3.5, is eluded - the figures are not presented ideally, nor in a very readable form - for example, their are 90-degree rotated compared to classical presentations, and the plots are hardly readable - the might exist a tradeoff between robustness, and performance (accuracy), that seem to be explaining the proposed results (see Fawzi - Machine Learning 2018, for example) - in 4.2, what is a mean case of adversarial noise? Also, it would be good to see the effect of the regularizer of both the *original* network, and on the network trained with data augmentation. It is not clear which one is considered here, but it would be interesting to study both, actually. - the second paragraph of the conclusion (transfer of perturbations) opens interesting perspective, but the problem might not be as trivial as the authors seem to hint in the text. Overall, very interesting and nice work, which might be better positioned (especially in terms of experiments) wrt to other recent methods that propose to improve robustness in NNs.