Learning with Protection: Rejection of Suspicious Samples under Adversarial Environment
We propose a novel framework for avoiding the misclassification of data by using a framework of learning with rejection and adversarial examples. Recent developments in machine learning have opened new opportunities for industrial innovations such as self-driving cars. However, many machine learning models are vulnerable to adversarial attacks and industrial practitioners are concerned about accidents arising from misclassification. To avoid critical misclassifications, we define a sample that is likely to be mislabeled as a suspicious sample. Our main idea is to apply a framework of learning with rejection and adversarial examples to assist in the decision making for such suspicious samples. We propose two frameworks, learning with rejection under adversarial attacks and learning with protection. Learning with rejection under adversarial attacks is a naive extension of the learning with rejection framework for handling adversarial examples. Learning with protection is a practical application of learning with rejection under adversarial attacks. This algorithm transforms the original multi-class classification problem into a binary classification for a specific class, and we reject suspicious samples to protect a specific label. We demonstrate the effectiveness of the proposed method in experiments.
There is still no universal method to deal with adversarial examples, and introducing a reject option to flag potential attacks seems a sensitive choice for many applications. The considered problem is well-motivated and introduced, and I’m unaware of prior work studying classification with reject option in the context of adversarial examples. However, I think there are different dimensions along which the paper could be improved: - My understanding of classification with a reject option is that the rejection cost c(x) is a design choice that can depend on the specific application. While c(x) is introduced as part of the framework it is then derived in a very specific way, removing the design aspect or at least not explaining very clearly how one would design it. Also, Algorithm 1 doesn’t have corresponding parameters. - The rejection function r(x) relies on z^* which essentially amounts to computing an adversarial perturbation for a given testing point, see (2). The authors state that they use a 30-step of the PGD algorithm to find z^*. The attacks on which the method is tested only uses 10 PGD steps. What if the attacker is stronger in the sense that it runs significantly more PGD iterations than used to compute the rejection function? How sensitive is the rejection function to different initializations? - Similarly, what happens if the attacker and the rejection function rely on different norms to compute the attack and the rejection score, respectively? I think it is important to investigate this aspect. - In Tables 1 and 2, I don’t understand why the precision is the same for all rows corresponding to the same attack (strength), while the other metrics vary. Overall, I think the paper explores an interesting direction, but would greatly benefit from a revision along the lines outlined above. Minor comments: - P3 3.1 2nd paragraph: “Let B^p...” rather than “Let B^infty...” - P4 bottom: “)” is missing before “..., where” - P6 bottom: (FR) should be (TR). The acronyms FA and FR don’t seem to be introduced. ### Reply to rebuttal: I thank the reviewers for their detailed reply. It seems that the authors agree that some of the issues I raised should be addressed and improved. My assessment that the paper should be revised (and accordingly the rating) therefore remains unchanged.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Learning with Protection: Rejection of Suspicious Samples under Adversarial Environment
We propose a novel framework for avoiding the misclassification of data by using a framework of learning with rejection and adversarial examples. Recent developments in machine learning have opened new opportunities for industrial innovations such as self-driving cars. However, many machine learning models are vulnerable to adversarial attacks and industrial practitioners are concerned about accidents arising from misclassification. To avoid critical misclassifications, we define a sample that is likely to be mislabeled as a suspicious sample. Our main idea is to apply a framework of learning with rejection and adversarial examples to assist in the decision making for such suspicious samples. We propose two frameworks, learning with rejection under adversarial attacks and learning with protection. Learning with rejection under adversarial attacks is a naive extension of the learning with rejection framework for handling adversarial examples. Learning with protection is a practical application of learning with rejection under adversarial attacks. This algorithm transforms the original multi-class classification problem into a binary classification for a specific class, and we reject suspicious samples to protect a specific label. We demonstrate the effectiveness of the proposed method in experiments.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



There is still no universal method to deal with adversarial examples, and introducing a reject option to flag potential attacks seems a sensitive choice for many applications. The considered problem is well-motivated and introduced, and I’m unaware of prior work studying classification with reject option in the context of adversarial examples. However, I think there are different dimensions along which the paper could be improved: - My understanding of classification with a reject option is that the rejection cost c(x) is a design choice that can depend on the specific application. While c(x) is introduced as part of the framework it is then derived in a very specific way, removing the design aspect or at least not explaining very clearly how one would design it. Also, Algorithm 1 doesn’t have corresponding parameters. - The rejection function r(x) relies on z^* which essentially amounts to computing an adversarial perturbation for a given testing point, see (2). The authors state that they use a 30-step of the PGD algorithm to find z^*. The attacks on which the method is tested only uses 10 PGD steps. What if the attacker is stronger in the sense that it runs significantly more PGD iterations than used to compute the rejection function? How sensitive is the rejection function to different initializations? - Similarly, what happens if the attacker and the rejection function rely on different norms to compute the attack and the rejection score, respectively? I think it is important to investigate this aspect. - In Tables 1 and 2, I don’t understand why the precision is the same for all rows corresponding to the same attack (strength), while the other metrics vary. Overall, I think the paper explores an interesting direction, but would greatly benefit from a revision along the lines outlined above. Minor comments: - P3 3.1 2nd paragraph: “Let B^p...” rather than “Let B^infty...” - P4 bottom: “)” is missing before “..., where” - P6 bottom: (FR) should be (TR). The acronyms FA and FR don’t seem to be introduced. ### Reply to rebuttal: I thank the reviewers for their detailed reply. It seems that the authors agree that some of the issues I raised should be addressed and improved. My assessment that the paper should be revised (and accordingly the rating) therefore remains unchanged.
assistant
There is still no universal method to deal with adversarial examples, and introducing a reject option to flag potential attacks seems a sensitive choice for many applications. The considered problem is well-motivated and introduced, and I’m unaware of prior work studying classification with reject option in the context of adversarial examples. However, I think there are different dimensions along which the paper could be improved: - My understanding of classification with a reject option is that the rejection cost c(x) is a design choice that can depend on the specific application. While c(x) is introduced as part of the framework it is then derived in a very specific way, removing the design aspect or at least not explaining very clearly how one would design it. Also, Algorithm 1 doesn’t have corresponding parameters. - The rejection function r(x) relies on z^* which essentially amounts to computing an adversarial perturbation for a given testing point, see (2). The authors state that they use a 30-step of the PGD algorithm to find z^*. The attacks on which the method is tested only uses 10 PGD steps. What if the attacker is stronger in the sense that it runs significantly more PGD iterations than used to compute the rejection function? How sensitive is the rejection function to different initializations? - Similarly, what happens if the attacker and the rejection function rely on different norms to compute the attack and the rejection score, respectively? I think it is important to investigate this aspect. - In Tables 1 and 2, I don’t understand why the precision is the same for all rows corresponding to the same attack (strength), while the other metrics vary. Overall, I think the paper explores an interesting direction, but would greatly benefit from a revision along the lines outlined above. Minor comments: - P3 3.1 2nd paragraph: “Let B^p...” rather than “Let B^infty...” - P4 bottom: “)” is missing before “..., where” - P6 bottom: (FR) should be (TR). The acronyms FA and FR don’t seem to be introduced. ### Reply to rebuttal: I thank the reviewers for their detailed reply. It seems that the authors agree that some of the issues I raised should be addressed and improved. My assessment that the paper should be revised (and accordingly the rating) therefore remains unchanged.