Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
High sensitivity of neural networks against malicious perturbations on inputs causes security concerns. To take a steady step towards robust classifiers, we aim to create neural network models provably defended from perturbations. Prior certification work requires strong assumptions on network structures and massive computational costs, and thus the range of their applications was limited. From the relationship between the Lipschitz constants and prediction margins, we present a computationally efficient calculation technique to lower-bound the size of adversarial perturbations that can deceive networks, and that is widely applicable to various complicated networks. Moreover, we propose an efficient training procedure that robustifies networks and significantly improves the provably guarded areas around data points. In experimental evaluations, our method showed its ability to provide a non-trivial guarantee and enhance robustness for even large networks.
This paper proposes a computationally efficient calculation technique that lower-bounds the size of adversarial perturbations that can deceive networks, and an effective training procedure, which robustifies networks and significantly improves the provably guarded areas around data points. The contribution of this paper is proposing an intuitive way to measure the slope to calculate the upper-bounds of gradient and provide a widely available and highly scalable method that ensures large guarded areas for a wide range of network structures. There are certain contribution and originality in the literature.  Here I am concerned with the following two questions: 1.    This paper defines a guarded area for a network F and a data point X as a hypersphere with a radius c. This definition is so strong that may conflict with many real world applications. 2.    The authors transfer the problem into solving the Lipschitz constant and prove bounds in diffident functions and do some experiments to evaluate the performance of their method. But lack of the comparative experiments with some existing methods that make the experiment unconvincing.