A Unified framework for randomized smoothing based certified defenses
Randomized smoothing, which was recently proved to be a certified defensive technique, has received considerable attention due to its scalability to large datasets and neural networks. However, several important questions still remain unanswered in the existing frameworks, such as (i) whether Gaussian mechanism is an optimal choice for certifying -normed robustness, and (ii) whether randomized smoothing can certify -normed robustness (on high-dimensional datasets like ImageNet). To answer these questions, we introduce a {em unified} and {em self-contained} framework to study randomized smoothing-based certified defenses, where we mainly focus on the two most popular norms in adversarial machine learning, {em i.e.,} and norm. We answer the above two questions by first demonstrating that Gaussian mechanism and Exponential mechanism are the (near) optimal options to certify the and -normed robustness. We further show that the largest radius certified by randomized smoothing is upper bounded by , where is the dimensionality of the data. This theoretical finding suggests that certifying -normed robustness by randomized smoothing may not be scalable to high-dimensional data. The veracity of our framework and analysis is verified by extensive evaluations on CIFAR10 and ImageNet.
This work examines the recently proposed randomized smoothing method for certifying the robustness of neural networks. The authors explain a theoretical framework for analyzing randomized smoothing as a certification method, propose two alternative definitions of robustness (D_MR and D_inf), and prove that using Gaussian noise for smoothing is near “optimal” for L2 robustness, while using exponential noise for smoothing is optimal for L_inf robustness (the authors do this by establishing a lower bound on the noise necessary for smoothing to work). This also leads the authors to the interesting conclusion that randomized smoothing may not be scalable to high dimensional data for L_inf robustness. In its current state, I would vote to weakly reject this paper for one key reason. The notions of robustness defined by the authors (Definitions 3/7/8) is not the same as standard adversarial robustness (Definition 2), and the authors do not explain clearly how to translate their results back to adversarial robustness. Proving results about their own version of robustness is interesting, but it must be related back to the standard notion of adversarial robustness so that the broader machine learning community can understand how the authors’ contributions fit in the literature. It may in fact be quite straightforward to relate the two notions, but I think the authors should explain how to do so clearly. I am happy to reconsider if the authors can address this (and other comments below) in a satisfactory manner. I did not check the authors’ theoretical proofs, but I find the statements of the theorems interesting, especially the results about the maximum certifiable radius for L_inf robustness. This provides significant new insight about the fact that L_inf robustness may not be easy to certify using randomized smoothing methods. However, it is not clear to me how best to translate the authors’ results to a result for the standard notion of adversarial robustness, which I believe would be interesting to present clearly. I would encourage the authors’ to clarify (and tone down) their statement about the “optimality” of Gaussian noise for L2 robustness. Theorem 12 provides a lower bound on the L_inf norm of the noise added, and they show that Gaussian noise is close to “optimal” in terms of expected L_inf norm. I am a bit confused as to why are we providing bounds on the L_inf norm of the added noise (especially since we are verifying L2 robustness) - in what other ways is Gaussian noise (near) optimal? Does it also have the expected lowest L2 norm? Also, why do we want the noise to have low norm? I feel that “optimal” should mean being able to prove the largest possible robust radius, and if that is not what you are proving, I would encourage you to try to avoid overclaiming. Finally, the experimental results should also not just be in terms of D_MR robustness. Otherwise, it is hard to compare with prior work like Cohen et. al. Some additional feedback: - “the Lp-normed robustness” can be replaced with “Lp-norm robustness” everywhere - Page 1, say “the Gaussian mechanism” instead of “Gaussian mechanism” (toward the end of the first paragraph) - Table 1’s formatting can be improved (maybe have a box around the whole table) - Theorem 12 - use “In other words” instead of “In another word”
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
A Unified framework for randomized smoothing based certified defenses
Randomized smoothing, which was recently proved to be a certified defensive technique, has received considerable attention due to its scalability to large datasets and neural networks. However, several important questions still remain unanswered in the existing frameworks, such as (i) whether Gaussian mechanism is an optimal choice for certifying -normed robustness, and (ii) whether randomized smoothing can certify -normed robustness (on high-dimensional datasets like ImageNet). To answer these questions, we introduce a {em unified} and {em self-contained} framework to study randomized smoothing-based certified defenses, where we mainly focus on the two most popular norms in adversarial machine learning, {em i.e.,} and norm. We answer the above two questions by first demonstrating that Gaussian mechanism and Exponential mechanism are the (near) optimal options to certify the and -normed robustness. We further show that the largest radius certified by randomized smoothing is upper bounded by , where is the dimensionality of the data. This theoretical finding suggests that certifying -normed robustness by randomized smoothing may not be scalable to high-dimensional data. The veracity of our framework and analysis is verified by extensive evaluations on CIFAR10 and ImageNet.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This work examines the recently proposed randomized smoothing method for certifying the robustness of neural networks. The authors explain a theoretical framework for analyzing randomized smoothing as a certification method, propose two alternative definitions of robustness (D_MR and D_inf), and prove that using Gaussian noise for smoothing is near “optimal” for L2 robustness, while using exponential noise for smoothing is optimal for L_inf robustness (the authors do this by establishing a lower bound on the noise necessary for smoothing to work). This also leads the authors to the interesting conclusion that randomized smoothing may not be scalable to high dimensional data for L_inf robustness. In its current state, I would vote to weakly reject this paper for one key reason. The notions of robustness defined by the authors (Definitions 3/7/8) is not the same as standard adversarial robustness (Definition 2), and the authors do not explain clearly how to translate their results back to adversarial robustness. Proving results about their own version of robustness is interesting, but it must be related back to the standard notion of adversarial robustness so that the broader machine learning community can understand how the authors’ contributions fit in the literature. It may in fact be quite straightforward to relate the two notions, but I think the authors should explain how to do so clearly. I am happy to reconsider if the authors can address this (and other comments below) in a satisfactory manner. I did not check the authors’ theoretical proofs, but I find the statements of the theorems interesting, especially the results about the maximum certifiable radius for L_inf robustness. This provides significant new insight about the fact that L_inf robustness may not be easy to certify using randomized smoothing methods. However, it is not clear to me how best to translate the authors’ results to a result for the standard notion of adversarial robustness, which I believe would be interesting to present clearly. I would encourage the authors’ to clarify (and tone down) their statement about the “optimality” of Gaussian noise for L2 robustness. Theorem 12 provides a lower bound on the L_inf norm of the noise added, and they show that Gaussian noise is close to “optimal” in terms of expected L_inf norm. I am a bit confused as to why are we providing bounds on the L_inf norm of the added noise (especially since we are verifying L2 robustness) - in what other ways is Gaussian noise (near) optimal? Does it also have the expected lowest L2 norm? Also, why do we want the noise to have low norm? I feel that “optimal” should mean being able to prove the largest possible robust radius, and if that is not what you are proving, I would encourage you to try to avoid overclaiming. Finally, the experimental results should also not just be in terms of D_MR robustness. Otherwise, it is hard to compare with prior work like Cohen et. al. Some additional feedback: - “the Lp-normed robustness” can be replaced with “Lp-norm robustness” everywhere - Page 1, say “the Gaussian mechanism” instead of “Gaussian mechanism” (toward the end of the first paragraph) - Table 1’s formatting can be improved (maybe have a box around the whole table) - Theorem 12 - use “In other words” instead of “In another word”
assistant
This work examines the recently proposed randomized smoothing method for certifying the robustness of neural networks. The authors explain a theoretical framework for analyzing randomized smoothing as a certification method, propose two alternative definitions of robustness (D_MR and D_inf), and prove that using Gaussian noise for smoothing is near “optimal” for L2 robustness, while using exponential noise for smoothing is optimal for L_inf robustness (the authors do this by establishing a lower bound on the noise necessary for smoothing to work). This also leads the authors to the interesting conclusion that randomized smoothing may not be scalable to high dimensional data for L_inf robustness. In its current state, I would vote to weakly reject this paper for one key reason. The notions of robustness defined by the authors (Definitions 3/7/8) is not the same as standard adversarial robustness (Definition 2), and the authors do not explain clearly how to translate their results back to adversarial robustness. Proving results about their own version of robustness is interesting, but it must be related back to the standard notion of adversarial robustness so that the broader machine learning community can understand how the authors’ contributions fit in the literature. It may in fact be quite straightforward to relate the two notions, but I think the authors should explain how to do so clearly. I am happy to reconsider if the authors can address this (and other comments below) in a satisfactory manner. I did not check the authors’ theoretical proofs, but I find the statements of the theorems interesting, especially the results about the maximum certifiable radius for L_inf robustness. This provides significant new insight about the fact that L_inf robustness may not be easy to certify using randomized smoothing methods. However, it is not clear to me how best to translate the authors’ results to a result for the standard notion of adversarial robustness, which I believe would be interesting to present clearly. I would encourage the authors’ to clarify (and tone down) their statement about the “optimality” of Gaussian noise for L2 robustness. Theorem 12 provides a lower bound on the L_inf norm of the noise added, and they show that Gaussian noise is close to “optimal” in terms of expected L_inf norm. I am a bit confused as to why are we providing bounds on the L_inf norm of the added noise (especially since we are verifying L2 robustness) - in what other ways is Gaussian noise (near) optimal? Does it also have the expected lowest L2 norm? Also, why do we want the noise to have low norm? I feel that “optimal” should mean being able to prove the largest possible robust radius, and if that is not what you are proving, I would encourage you to try to avoid overclaiming. Finally, the experimental results should also not just be in terms of D_MR robustness. Otherwise, it is hard to compare with prior work like Cohen et. al. Some additional feedback: - “the Lp-normed robustness” can be replaced with “Lp-norm robustness” everywhere - Page 1, say “the Gaussian mechanism” instead of “Gaussian mechanism” (toward the end of the first paragraph) - Table 1’s formatting can be improved (maybe have a box around the whole table) - Theorem 12 - use “In other words” instead of “In another word”