Cost-Sensitive Robustness against Adversarial Examples
Several recent works have developed methods for training classifiers that are certifiably robust against norm-bounded adversarial perturbations. These methods assume that all the adversarial transformations are equally important, which is seldom the case in real-world applications. We advocate for cost-sensitive robustness as the criteria for measuring the classifier*s performance for tasks where some adversarial transformation are more important than others. We encode the potential harm of each adversarial transformation in a cost matrix, and propose a general objective function to adapt the robust training method of Wong & Kolter (2018) to optimize for cost-sensitive robustness. Our experiments on simple MNIST and CIFAR10 models with a variety of cost matrices show that the proposed approach can produce models with substantially reduced cost-sensitive robust error, while maintaining classification accuracy.
 The paper introduces a new concept of certified cost-sensitive robustness against adversarial attacks. A cost-sensitive robust optimization formulation is then proposed for deep adversarial learning. Experimental results on two benchmark datasets (MNIST, CIFAR-10) are reported to show the superiority of the proposed method to overall robustness method, both with binary and real-value cost matrices. The idea of cost-sensitive adversarial deep learning is well motivated. The proposed method is clearly presented and the results are easy to access. My main concern is about the novelty of the approach which looks mostly incremental as a rather direct extension of the robust model (Wong & Kolter 2018) to cost-sensitive setting. Particularly, the duality lower-bound based loss function and its related training procedure are almost identical to those from (Wong & Kolter 2018), up to certain trivial modification to respect the pre-specified misclassification costs. The numerical results show some promise. However, as a practical paper, the current empirical study appears limited in data scale: I believe additional evaluation on more challenging data sets can be useful to better support the importance of approach. Pros: - The concept of certified cost-sensitive robustness is well motivated and clearly presented. Cons: - The novelty of method is mostly incremental given the prior work of (Wong & Kolter 2018). - Numerical results show some promise of cost-sensitive adversarial learning in the considered settings, but still not supportive enough to the importance of approach.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Cost-Sensitive Robustness against Adversarial Examples
Several recent works have developed methods for training classifiers that are certifiably robust against norm-bounded adversarial perturbations. These methods assume that all the adversarial transformations are equally important, which is seldom the case in real-world applications. We advocate for cost-sensitive robustness as the criteria for measuring the classifier*s performance for tasks where some adversarial transformation are more important than others. We encode the potential harm of each adversarial transformation in a cost matrix, and propose a general objective function to adapt the robust training method of Wong & Kolter (2018) to optimize for cost-sensitive robustness. Our experiments on simple MNIST and CIFAR10 models with a variety of cost matrices show that the proposed approach can produce models with substantially reduced cost-sensitive robust error, while maintaining classification accuracy.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper introduces a new concept of certified cost-sensitive robustness against adversarial attacks. A cost-sensitive robust optimization formulation is then proposed for deep adversarial learning. Experimental results on two benchmark datasets (MNIST, CIFAR-10) are reported to show the superiority of the proposed method to overall robustness method, both with binary and real-value cost matrices. The idea of cost-sensitive adversarial deep learning is well motivated. The proposed method is clearly presented and the results are easy to access. My main concern is about the novelty of the approach which looks mostly incremental as a rather direct extension of the robust model (Wong & Kolter 2018) to cost-sensitive setting. Particularly, the duality lower-bound based loss function and its related training procedure are almost identical to those from (Wong & Kolter 2018), up to certain trivial modification to respect the pre-specified misclassification costs. The numerical results show some promise. However, as a practical paper, the current empirical study appears limited in data scale: I believe additional evaluation on more challenging data sets can be useful to better support the importance of approach. Pros: - The concept of certified cost-sensitive robustness is well motivated and clearly presented. Cons: - The novelty of method is mostly incremental given the prior work of (Wong & Kolter 2018). - Numerical results show some promise of cost-sensitive adversarial learning in the considered settings, but still not supportive enough to the importance of approach.
assistant
 The paper introduces a new concept of certified cost-sensitive robustness against adversarial attacks. A cost-sensitive robust optimization formulation is then proposed for deep adversarial learning. Experimental results on two benchmark datasets (MNIST, CIFAR-10) are reported to show the superiority of the proposed method to overall robustness method, both with binary and real-value cost matrices. The idea of cost-sensitive adversarial deep learning is well motivated. The proposed method is clearly presented and the results are easy to access. My main concern is about the novelty of the approach which looks mostly incremental as a rather direct extension of the robust model (Wong & Kolter 2018) to cost-sensitive setting. Particularly, the duality lower-bound based loss function and its related training procedure are almost identical to those from (Wong & Kolter 2018), up to certain trivial modification to respect the pre-specified misclassification costs. The numerical results show some promise. However, as a practical paper, the current empirical study appears limited in data scale: I believe additional evaluation on more challenging data sets can be useful to better support the importance of approach. Pros: - The concept of certified cost-sensitive robustness is well motivated and clearly presented. Cons: - The novelty of method is mostly incremental given the prior work of (Wong & Kolter 2018). - Numerical results show some promise of cost-sensitive adversarial learning in the considered settings, but still not supportive enough to the importance of approach.