Formal verification of machine learning models has attracted attention recently, and significant progress has been made on proving simple properties like robustness to misclassification under perturbations of the input features. In this context, it has also been observed that folding the verification procedure into training makes it easier to train verifiably robust models. In this paper, we extend the applicability of verified training by extending it to (1) recurrent neural network architectures and (2) complex specifications that go beyond simple adversarial robustness, particularly specifications that capture temporal properties like requiring that a robot periodically visits a charging station or that a language model always produces sentences of bounded length. Experiments show that while models trained using standard training often violate desired specifications, our verified training method produces models that both perform well (in terms of test error or reward) and can be shown to be provably consistent with specifications.
This paper extends bound propagation based robust training method to complicated settings where temporal specifications are given. Previous works mainly focus on using bound propagation for robust classification only. The authors first extend bound propagation to more complex networks with gates and softmax, and designed a loss function that replies on a lower bound for quantitative semantics of specifications over an input set S. The proposed framework is demonstrated on three tasks: Multi-MNIST captioning, vacuum cleaning robot agent and language generation. The authors formulate specifications using temporal logic, train models with bound propagation to enforce these specifications, and verify them after training. 
My questions regarding this paper are mostly on the three demonstrated tasks:
1. For the Multi-MNIST dataset, I have the following questions:
1(a). In Table 2, it is surprising that even with perturbation of 0.5, the verified and adversarial accuracy is very high. At perturbation epsilon=0.5, it should be possible to perturb the entire image to gray (value 0.5), so I believe the accuracy should be very low here. It is hard to believe under this setting the verified accuracy is still 99%.
1(b). For Table 3, nominal accuracy should also be reported.
1(c). Additionally, how do you define the nominal accuracy here? An example is nominally correct when all digits are predicted correctly in the sequence, or just when the sequence length is predicted correctly?
1(d). For the termination accuracy, do we only care about the sequence length being predicted correctly, or does it also cover the case that all digits in the sequence are predicted correctly? If it is only concerning about the sequence length, this property is a little bit weak.
2. For the RL Robot agent experiment, I have the following questions:
2(a). Because T=10, are you saying that we can only guarantee that the battery is always recharged for any rollouts less than 10 steps? After 10 steps beyond the initial position, can we get any guarantees? A 10-step only guarantee seems too restrictive.
2(b). Are all the properties only verified assuming that the agent starts from the center? I think this assumption is probably also too strong in practice.
2(c). Since all the %verified cells reported in Table 4 are all 100%, it is probably better to make the problem more challenging, by increasing T or considering different initial positions. It is important to show when the performance of the proposed method starts to degrade, to understand the power of the proposed method.
3. For the language generation experiment, the perplexity of the verified training model looks significantly worse than nominal or sampled models. With a perplexity as high as this, I believe the model actually produces garbage. Can you provide some examples of generated texts? I feel language generation is probably not a suitable task for the proposed training method. 
Other minor issues:
1. Table 1 should have some horizontal lines - it is hard to align the works with categories on the right.
2. Several papers appear multiple times in references, including "Differentiable abstract interpretation for provably robust neural networks", "Towards fast computation of certified robustness for relu networks" (and probably others). Also on page 2, Shiqi et al., should be Wang et al. (Shiqi is the first name).
3. I feel the writing is a bit rushed and the authors should make a few more passes on the paper.
This paper makes valid technical contributions, especially the conversion from STL specifications to lower bounds of the quantitative semantics is interesting. Although bound propagation based robust training method is simple to extend to softmax/GRU with interval analysis, applying robust training techniques to the three interesting applications are good contributions. Since the main contribution of this paper is the empirical results on the three tasks, my concerns regarding the experiments need to be addressed before I can vote for accepting this paper. Also, because this paper uses 10 pages, I am expecting the paper to meet a higher standard. Thus, I am voting for a weak reject at this time.
****** After author response
The author response addressed some of my concerns. However I do believe this paper is relatively weak in contribution, especially the experiments can be done more thoroughly. I also appreciate that the authors reduced paper length to 8 pages. I am okay with accepting this paper as it does have some interesting bits, but it is clearly on the borderline and can be further improved.