Efficient Neural Network Robustness Certification with General Activation Functions
Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem. Nevertheless, recently it has been shown to be possible to give a nontrivial certified lower bound of minimum adversarial distortion, and some recent progress has been made towards this direction by exploiting the piece-wise linear nature of ReLU activations. However, a generic robustness certification for general activation functions still remains largely unexplored. To address this issue, in this paper we introduce CROWN, a general framework to certify robustness of neural networks with general activation functions for given input data points. The novelty in our algorithm consists of bounding a given activation function with linear and quadratic functions, hence allowing it to tackle general activation functions including but not limited to four popular choices: ReLU, tanh, sigmoid and arctan. In addition, we facilitate the search for a tighter certified lower bound by adaptively selecting appropriate surrogates for each neuron activation. Experimental results show that CROWN on ReLU networks can notably improve the certified lower bounds compared to the current state-of-the-art algorithm Fast-Lin, while having comparable computational efficiency. Furthermore, CROWN also demonstrates its effectiveness and flexibility on networks with general activation functions, including tanh, sigmoid and arctan.
Summary: This paper proposes a general framework CROWN to efficiently certify robustness of neural networks with general activation functions.  CROWN adaptively bounds a given activation function with linear and quadratic functions, so it can tackle general activation functions including but not limited to the four popular choices: ReLU, tanh, sigmoid, and arctan. Experimental results demonstrate the effectiveness, efficiency, and flexibility of the proposed framework.  Quality: We are glad to find a work which conducts the efficiently certifying of the non-trivial robustness for general activation functions in neural networks. It is also interesting that the proposed framework can flexibly select upper bounds and lower bounds which can reduce the approximation error. This paper is of high quality. The problem is well-motivated, and the experimental comparisons and analysis are sufficient and complete. The proofs of the theorems and bounds are reasonable.     Clarity: This paper is very well written and enjoyable to read. The theory analysis and proof are clear and easy to follow.  Originality: This is an original research work especially of the robustness certification with the general activation functions, which may provide a good direction for the future research on this problem.  Overall impression, this is a high-quality paper with solid analysis, proof, and experimental results.