VERIFICATION OF NON-LINEAR SPECIFICATIONS FOR NEURAL NETWORKS
Prior work on neural network verification has focused on specifications that are linear functions of the output of the network, e.g., invariance of the classifier output under adversarial perturbations of the input. In this paper, we extend verification algorithms to be able to certify richer properties of neural networks. To do this we introduce the class of convex-relaxable specifications, which constitute nonlinear specifications that can be verified using a convex relaxation. We show that a number of important properties of interest can be modeled within this class, including conservation of energy in a learned dynamics model of a physical system; semantic consistency of a classifier's output labels under adversarial perturbations and bounding errors in a system that predicts the summation of handwritten digits. Our experimental evaluation shows that our method is able to effectively verify these specifications. Moreover, our evaluation exposes the failure modes in models which cannot be verified to satisfy these specifications. Thus, emphasizing the importance of training models not just to fit training data but also to be consistent with specifications.
This paper uses convex relaxation to verify a larger class of specifications
for neural network's properties. Many previous papers use convex relaxations on
the ReLU activation function and solve a relaxed convex problem to give
verification bounds.  However, most papers consider the verification
specification simply as an affine transformation of neural network's output.
This paper extends the verification specifications to a larger family of
functions that can be efficiently relaxed.
The author demonstrates three use cases for non-linear specifications,
including verifying specifications involving label semantics, physic laws and
down-stream tasks, and show some experiments that the proposed verification
method can find non-vacuous bound for these problems. Additionally, this paper
shows some interesting experiments on the value of verification - a more
verifiable model seems to provide more interpretable results.
Overall, the proposed method seems to be a straightforward extension to
existing works like [2]. However the demonstrated applications of non-linear
specifications are indeed interesting, and the proposed method works well on 
these tasks.
I have some minor questions regarding this paper:
1) For some non-linear specifications, we can convert these non-linear elements
into activation functions, and build an equivalent network for verification
such that the final verification specification becomes linear. For example, for
verifying the quadratic specification in physics we can add a "quadratic
activation function" to the network and deal with it using techniques in [1] or
[2].  The authors should distinguish the proposed technique with these existing
techniques. My understanding is that the proposed method is more general, but
the authors should better discussing more on the differences in this paper.
2) The authors should report the details on how they solve the relaxed convex
problem, and report verification time. Are there any tricks used to improve
solving time? What is the largest scale of network that the algorithm can
handle within a reasonable time?
3) The detailed network architecture (Model A, Model B) is not shown. How many
layers and neurons are there in these networks? This is important to show the
scalability of the proposed method.
4) For the Mujoco experiment, I am not sure how to interpret the delta values
in Figure 1. For CIFAR I know it is the delta of pixel values but it is not
clear about the delta in Mujoco model. What is the normal range of predicted
numbers in this model?  How does the delta compare to it? Is the delta very
small or trivial?
5) Is it possible to show how loose the convex relaxation is for a small toy
example? For example, the specification involving quadratic function is a
good candidate.
There are some small glitches in equations:
* In (4), k is undefined
* In (20), I am not sure if it is equivalent to the four inequalities after (22).
There are 4 inequalities after (22) but only 3 in (20).
Many papers uses convex relaxations for neural network verification. However
very few of them can deal with general non-linear units in neural networks.
ReLU activation is usually the only non-linear element than we can handle in
most neural network verification works. Currently the only works that can
handle other general non-linear elements are [1][2]. This paper uses more
general convex relaxations than these previous approaches, and it can handle
non-separable non-linear specifications. This is a unique contribution to this
field. I recommend accepting this paper as long as the minor issues mentioned
above can be fixed.
[1] "Efficient Neural Network Robustness Certification with General Activation
Functions" by Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, Luca Daniel.
NIPS 2018
[2] "A dual approach to scalable verification of deep networks." by
Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy Mann, and
Pushmeet Kohli. UAI 2018.