In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement a scaleable version of input gradient regularization which avoids double backpropagation: adversarially robust ImageNet models are trained in 33 hours on four consumer grade GPUs. Finally, we show experimentally and through theoretical certification that input gradient regularization is competitive with adversarial training. Moreover we demonstrate that gradient regularization does not lead to gradient obfuscation or gradient masking.

This paper revisits the method of gradient regularization, which regularizes the loss function by adding the norm of the input gradient, aiming to improve adversarial robustness. The standard gradient regularization is implemented with "double backpropagation", which could be time consuming for large networks. This paper proposes to replace the exact gradient by its discretization via finite difference, which is computationally more efficient comparing to "double backpropagation".  Experiments are conducted to show the effectiveness of the method in reducing training time.
Overall, the paper is well written but the contribution is very limited and I find some of the experimental comparisons unfair.  Thus I do not support publication of the paper. Here are my detailed arguments.
1) The gradient regularization is not novel
The main contribution of the paper is to use the finite difference in the gradient regularization in order to improve the training time.  The method of gradient regularization is not new, hence the contribution is only the computational effectiveness. This would be fine if one could improve the state-of-the-art method's training time by a lot, but according to the experiments, the performance of adversarial robustness is far from the adversarial training, for example in CIFAR10 epsilon=8/255, there is a 12% drop, which is a huge gap.
2) Unfair comparison in the experiments
In table 2, the performance of robustness with respect to the l2 norm is presented. However, in the baseline method of adversarial training, the used attack is in l_{infty} norm. This is unfair since l2 norm and l_{infty} norm has very different characteristics. Moreover, the gradient regularization uses l2 norm instead of l_{1} as in table 1, which makes the comparison unfair.
Furthermore, the finite difference improves the training time versus the standard gradient regularization, but it does not imply that the performance will be the same. In particular, it would be better to include the performance of standard gradient regularization in table 1 and 2 as well.
3) Comment on the motivation/theory
The theory part is fairly straightforward and it clearly shows that the norm of gradient (w.r.t input) itself is not sufficient to guarantee robustness. As a evidence, even under standard training (for example on MNIST), the gradient norm could be very small, in order of 10^{-4} but still have adversarial example with very small perturbation.  Thus, what we also need is to control how fast this gradient changes (Lipschitz constant or w-bound). However, the gradient norm regularization does not take into account how gradient changes. It is claimed that the finite discretization implicitly reduces how the gradient changes, however, I am not convinced by the argument since as long as we take h to be small, it is still a very local measure. An interesting question would be how h affect the performance and it is not discussed in the paper.


We consider the problem of learning models of intuitive physics from raw, unlabelled visual input. Differently from prior work, in addition to learning general physical principles, we are also interested in learning "on the fly" physical properties specific to new environments, based on a small number of environment-specific experiences. We do all this in an unsupervised manner, using a meta-learning formulation where the goal is to predict videos containing demonstrations of physical phenomena, such as objects moving and colliding with a complex background. We introduce the idea of summarizing past experiences in a very compact manner, in our case using dynamic images, and show that this can be used to solve the problem well and efficiently. Empirically, we show, via extensive experiments and ablation studies, that our model learns to perform physical predictions that generalize well in time and space, as well as to a variable number of interacting physical objects.

Summary
The paper proposes an architecture for few-shot video prediction in which a number of videos are summarized through global pooling operations and passed into a video predictor that learns to leverage them for adaptation, similar in spirit to the RNN-based meta-learning approaches such as Santoro’16, Duan’16. Due to global image and feature pooling operations, the proposed approach is computationally efficient. Proof-of-concept experiments are presented in a simple simulated physical prediction setting. It is claimed that the proposed model achieves generalization to longer sequences and larger board sizes, as well as a larger number of objects.
Decision
The work considers an interesting task and shows promise, but the paper lacks a substantial contribution at this time. I recommend weak reject. To improve the paper, I recommend considering harder prediction tasks and lifting some of the restrictions of the method.
Pros
The paper proposes and evaluates a model for few-shot video prediction, which is an interesting and well-motivated task.
Cons
1. The method is only evaluated on a toy task. While a model is proposed that can successfully solve the presented toy task, certain assumptions, such as averaging directly in the image space, may prevent the findings of this paper to scale to more complex tasks, which demands further investigation.
2. It is claimed that the method generalizes to longer sequences and larger board sizes, however, the performance of the method seems to quickly deteriorate in both settings. Authors present anecdotal evidence that the method generalizes to larger number of objects, however, quantitatively the results again are much inferior to the method trained on a larger number of objects. While the presented qualitative generalization findings are interesting, they seem limited in scope, especially when taking into account the simplicity of the considered data.
Minor comments
1. On page 3, the paper states “We are not the first to consider a similar learning problem, although most prior works do require some form of external supervision.” While there is a substantial body of literature that uses extra supervision, the following foundational works do not: Finn et al, 2016a,b; Fragkiadaki, 2016. In addition, “a similar learning problem” is considered without supervision in many papers on video prediction, including: Ranzato et al., 2014; Srivastava et al., 2015; Mathieu et al., 2015; Denton et al., 2017, 2018; Villegas et al., 2017, 2019; Wichers et al., 2018; Castrejon et al., 2019. While it might not be necessary to cite all of these as they do not necessarily apply their models to intuitive physics, the statement in the paper seems rather unsubstantiated.
2. A similar meta-learning prediction problem is addressed in Nagabandi’19a,b, which are not cited.
3. The paper is somewhat cluttered with notation, which overall hampers the flow for the reader. The paper would benefit from slight reorganization to improve the flow. This could be done e.g. be structuring the paper to first present the computational task that it addresses, namely few-shot video prediction, and how each component of the model works to address that problem, and only later go into the detail of how the data were collected and how each individual module was implemented
Santoro et al, Meta-learning with memory-augmented neural networks.
Duan et al, RL2: Fast reinforcement learning via slow reinforcement learning.
Nagabandi et al, LEARNING TO ADAPT IN DYNAMIC, REAL-WORLD ENVIRONMENTS THROUGH META-REINFORCEMENT LEARNING,
Nagabandi et al, DEEP ONLINE LEARNING VIA META-LEARNING: CONTINUAL ADAPTATION FOR MODEL-BASED RL
--------------------- Update 11.19 -----------------------
I appreciate the newly provided qualitative generalization results (in response to R3). However, it is clear that the network has trouble maintaining the exact number of spheres in the scene, especially after collisions or interactions with obstacles. While this generalization finding is undoubtedly interesting, I am not convinced it is enough for a publication in ICLR.
Furthermore, to support the authors' claim that the prediction performance on long sequences is satisfactory, I suggest comparison with a hand-crafted baseline, such as one suggested by R1, or one that makes use of explicit physics.


Adaptive Generation of Unrestricted Adversarial Inputs
Neural networks are vulnerable to adversarially-constructed perturbations of their inputs. Most research so far has considered perturbations of a fixed magnitude under some l p norm. Although studying these attacks is valuable, there has been increasing interest in the construction of-and robustness to-unrestricted attacks, which are not constrained to a small and rather artificial subset of all possible adversarial inputs. We introduce a novel algorithm for generating such unrestricted adversarial inputs which, unlike prior work, is adaptive: it is able to tune its attacks to the classifier being targeted. It also offers a 400-2,000× speedup over the existing state of the art. We demonstrate our approach by generating unrestricted adversarial inputs that fool classifiers robust to perturbation-based attacks. We also show that, by virtue of being adaptive and unrestricted, our attack is able to defeat adversarial training against it.

The paper proposes using GANs to generate unrestricted adversarial examples. They seek to generate examples that are adversarial for a specific classifier, and they do so by using class-conditional GANs and a fine-tuning loss. The fine-tuning loss consists of both the ordinary GAN loss (to fool the discriminator) as well as an adversarial loss (which rewards the GAN for generating examples misclassified by the specific classifier). The authors perform various experiments on their generated examples to check for realism and how adversarial the generated images are.
I would reject this paper for two key reasons. First, I feel that the contributions are not significant enough (in comparison to the prior work of Song et. al). Second, I feel that some of the methods (and some of the writing) are not too principled.
In my opinion, unrestricted adversarial examples are significant if they can be made to be realistic. If our current deep learning models often mislabeled very realistic images, that would properly expose a big failure mode of our current models. However, if our machine learning models perform poorly on images that look fake/generated 40% of the time (which is what the authors state) and don’t look too realistic to humans, it is less worrying.
In comparison to Song et. al, the authors state that their methods result in very similar results in terms of realism and how adversarial their images are (arguably, Song et. al actually produces better results in terms of being adversarial). In my opinion, the authors’ claimed improvements are not significant enough, because I think realism should be the primary metric to evaluate this field. Improving speed of generation is nice, and being able to bypass a simple adversarial training procedure is interesting but not significant unless this insight is expanded upon. The results on MNIST in Fig. 5 and Fig. 6 are not too convincing, as simpler attacks that generate (arguably) more realistic images like translations and rotations [1] or L1/L2 attacks [2] (since the networks are trained for L_inf robustness) can also degrade accuracy. Finally, I can also think of another reasonable baseline that I would have liked to see the authors compare their method against. Because the authors want to attack a specific network, they could have (1) generated realistic images using a pre-trained GAN (2) used a norm-bounded attack on the specific classifier and the generated GAN images. These images could be even more realistic if the norm-bound of the attack is fairly small, and would still be able to attack specific classifiers.
Finally, I am confused by the comparison to a not-fine-tuned GAN in Fig. 14/Fig. 15 and would appreciate a clarification so that I can understand the results. For example, what does it mean for intended true label = 9, target label = 0 to have 90% success in Fig. 15? Does this mean that when you try to generate a 9 with the GAN, the classifier misclassifies it as a 0 90% of the time? In particular, I’m struggling to understand what the target label is for the case of the not-fine-tuned GAN.
Secondly, I feel that there are many instances in the paper where the methods used are not explained in a principled way. For example, one of the key parts of this work is the fine-tuning loss function. Why does the loss function involve multiplying the ordinary GAN loss (with some additional transformation applied to it which seems unnecessary) with the adversarial loss? It seems most reasonable add the adversarial loss and the ordinary GAN loss (without the additional transformation). Is the stochastic loss selection procedure necessary? If all these peculiarities of the method are necessary, it seems that the success of this method is quite brittle.
Additional feedback:
- In the intro, I think citing [3] in addition to Xu et. al is more appropriate.
- You should refer to Figure 1 somewhere in the text of your work
- In section 3.2, you can use “cosine similarity” to describe what you are doing faster.
- When you talk about “global optima of realistic adversarial examples” and “local optimal of unrealistic adversarial examples,” it sounds weird. I would try to reword this because I don’t think you are trying to make a precise mathematical statement but it sounds like one when you write it this way.
- In Table 1, I would format the numbers better to be vertically aligned
- You should provide a citation for MixTrain on page 5
[1] https://arxiv.org/abs/1712.02779,
[2] https://arxiv.org/abs/1905.01034
[3] https://arxiv.org/abs/1802.00420


As deep neural networks become widely adopted for solving most problems in computer vision and audio-understanding, there are rising concerns about their potential vulnerability. In particular, they are very sensitive to adversarial attacks, which manipulate the input to alter models' predictions. Despite large bodies of work to address this issue, the problem remains open. In this paper, we propose defensive tensorization, a novel adversarial defence technique that leverages a latent high order factorization of the network. Randomization is applied in the latent subspace, therefore resulting in dense reconstructed weights, without the sparsity or perturbations typically induced by the randomization. Our approach can be easily integrated with any arbitrary neural architecture and combined with techniques like adversarial training. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We further validate the generalizability of our approach across domains and low-precision architectures by considering an audio classification task and binary networks. In all cases, we demonstrate superior performance compared to prior works in the target scenario.

This paper tackles the problem of designing neural network architectures that are robust to adversarial attacks. Several defense techniques against adversarial attacks have been proposed, mainly adversarial training (train on perturbed inputs) and introducing random perturbation to the weights or activations of the network. The paper claims that one limitation of the second approach is that it introduces artifacts (e.g. sparsity). The authors propose a simple but original idea to address this issue: parameterize the network's weight matrices as low rank tensors (in the Tucker format) and randomize the weights by sketching the core tensor of the Tucker decomposition (in effect, the sketching amounts to randomly setting fibers of the core tensor to 0).
I think this paper can be relevant to the community but I am not confident that this is an important contribution. The idea is interesting and addresses the problem of sparsity artifacts in randomized defense strategies, but it does not appear clearly why using tensor decomposition techniques is a sound approach for designing robust networks (besides overcoming sparsity artifacts). I believe there may be more fundamental (theoretical, principled) arguments to motivate the approach, but this is not explored in the paper: the idea is interesting but not supported by much theoretical insight. Yes, using Tucker decomposition allows one to have randomized but still dense weights. Is it the only reason to use tensor decomposition? Why not do the same with a simple low rank matrix for example?
The experimental section is developed but I find the experimental setting not clearly described (e.g., what is the metric? is it the accuracy over adversarial examples?). Maybe this is because I am not familiar with the adversarial defense literature.
In conclusion, I am a bit on the fence for this paper. The idea is interesting and definitely worth exploring but to me a more thorough discussion and analysis of why tensor decomposition techniques are relevant is missing. Still, the approach is original and this paper may spark future work further exploring these questions, so I recommend acceptance.
* Comments / Questions *
- Paragraph "Latent high-order parametrization of the network". If I understand correctly, the core tensor G in the decomposition as the same size of W, so at this stage W is not parameterized as a low rank tensor (W is actually over-parmeterized). This is only when the stochastic vectors \lambda are introduced that the Tucker rank of W is implicitly reduced. This could be clarified.
- Is stochasticity preserved at test time (unlike when using dropout but like in Wang, et al. (2018))?
- What is the metric used in Table 1 to compare the models?
- Would it make sense to explore other tensor decomposition models (e.g. CP, tensor train, tensor ring, ...)? Are there any particular reasoning motivating the choice of Tucker?


We study adversarial robustness of neural networks from a margin maximization perspective, where margins are defined as the distances from inputs to a classifier's decision boundary. Our study shows that maximizing margins can be achieved by minimizing the adversarial loss on the decision boundary at the "shortest successful perturbation", demonstrating a close connection between adversarial losses and the margins. We propose Max-Margin Adversarial (MMA) training to directly maximize the margins to achieve adversarial robustness. Instead of adversarial training with a fixed , MMA offers an improvement by enabling adaptive selection of the "correct" as the margin individually for each datapoint. In addition, we rigorously analyze adversarial training with the perspective of margin maximization, and provide an alternative interpretation for adversarial training, maximizing either a lower or an upper bound of the margins. Our experiments empirically confirm our theory and demonstrate MMA training's efficacy on the MNIST and CIFAR10 datasets w.r.t. ∞ and 2 robustness. Despite their impressive performance on various learning tasks, neural networks have been shown to be vulnerable to adversarial perturbations (Szegedy et al., 2013; Biggio et al., 2013). An artificially constructed imperceptible perturbation can cause a significant drop in the prediction accuracy of an otherwise accurate network. The level of distortion is measured by the magnitude of the perturbations (e.g. in ∞ or 2 norms), i.e. the distance from the original input to the perturbed input. Figure 1 shows an example, where the classifier changes its prediction from panda to bucket when the input is perturbed from the blue sample point to the red one.
Figure 1 also shows the natural connection between adversarial robustness and the margins of the data points, where the margin is defined as the distance from a data point to the classifier's decision boundary. Intuitively, the margin of a data point is the minimum distance that x has to be perturbed to change the classifier's prediction. Thus, the larger the margin is, the farther the distance from the input to the decision boundary of the classifier is, the more robust the classifier is w.r.t. this input.
Although naturally connected to adversarial robustness, "directly" maximizing margins has not yet been thoroughly studied in the adversarial robustness literature. Instead, the method of minimax adversarial training (Madry et al., 2017; Huang et al., 2015)  is arguably the most common defense to adversarial perturbations due to its effectiveness and simplicity. Adversarial training attempts to minimize the maximum loss within a fixed sized neighborhood about the training data using projected gradient descent (PGD). Despite advancements made in recent years (Hendrycks et al.,  2019; Zhang et al., 2019a; Shafahi et al., 2019; Zhang et al., 2019b; Stanforth et al., 2019; Carmon  et al., 2019), adversarial training still suffers from a fundamental problem, the perturbation length has to be set and is fixed throughout the training process. In general, the setting of is arbitrary, based on assumptions on whether perturbations within the defined ball are "imperceptible" or not. Recent work (Guo et al., 2018; Sharma et al., 2019)  has demonstrated that these assumptions do not consistently hold true, commonly used settings assumed to only allow imperceptible perturbations 1 Under review as a conference paper at ICLR 2020 in fact do not. If is set too small, the resulting models lack robustness, if too large, the resulting models lack in accuracy. Moreover, individual data points may have different intrinsic robustness, the variation in ambiguity in collected data is highly diverse, and fixing one for all data points across the whole training procedure is likely suboptimal.
Instead of further improving adversarial training with a fixed perturbation magnitude, we revisit adversarial robustness from the margin perspective, and propose Max-Margin Adversarial (MMA) training, a practical algorithm for direct input margin maximization. By directly maximizing margins calculated for each data point, MMA training allows for optimizing the "current robustness" of the data, the "correct" at this point in training for each sample individually, instead of robustness w.r.t. a predefined magnitude.
While it is intuitive that one can achieve the greatest possible robustness by maximizing the margin of a classifier, this maximization has technical difficulties. In Section 2, we overcome these difficulties and show that margin maximization can be achieved by minimizing a classification loss w.r.t. model parameters, at the "shortest successful perturbation". This makes gradient descent viable for margin maximization, despite the fact that model parameters are entangled in the constraints.
We further analyze adversarial training (Madry et al., 2017; Huang et al., 2015)  from the perspective of margin maximization in Section 3. We show that, for each training example, adversarial training with fixed perturbation length is maximizing a lower (or upper) bound of the margin, if is smaller (or larger) than the margin of that training point. As such, MMA training improves adversarial training, in the sense that it selects the "correct" , the margin value for each example.
Finally in Section 4, we test and compare MMA training with adversarial training on MNIST and CIFAR10 w.r.t. ∞ and 2 robustness. Our method achieves higher robustness accuracies on average under a variety of perturbation magnitudes, which echoes its goal of maximizing the average margin. Moreover, MMA training automatically balances accuracy vs robustness while being insensitive to its hyperparameter setting, which contrasts sharply with the sensitivity of standard adversarial training to its fixed perturbation magnitude. MMA trained models not only match the performance of the best adversarially trained models with carefully chosen training under different scenarios, it also matches the performance of ensembles of adversarially trained models.
In this paper, we focus our theoretical efforts on the formulation for directly maximizing the input space margin, and understanding the standard adversarial training method from a margin maximization perspective. We focus our empirical efforts on thoroughly examining our MMA training algorithm, comparing with adversarial training with a fixed perturbation magnitude.

Summary:
This paper proposes an adaptive margin-based adversarial training (eg. MMA) approach to train robust DNNs by maximizing the shortest margin of inputs to the decision boundary. Theoretical analyses have been provided to understand the connection between robust optimization and margin maximization. The main difference between the proposed approach to standard adversarial training is the adaptive selection of the perturbation bound \epsilon. This makes adversarial training with large perturbation possible, which was previously unachievable by standard adversarial training (Madry et al.) Empirical results match the theoretical analysis.
Pros:
1. The margin maximization idea has been well-explained, both intuitively and theoretically.
2. Interesting theoretical analyses and understandings of robust optimization from the margin perspective.
3. Clear advantage of MMA over standard adversarial training under large perturbations.
8. The proposed PGDLS is very interesting, actually quite good and much simpler, without extra computational cost. A similar idea was discussed in paper [3], where they gradually increase the convergence quality of training adversarial examples, and show the convergence guarantee  of "dynamic training".
9. The gradient-free SPSA helps confirm the improvements of MMA under large perturbations are not a side effect of gradient masking.
Cons:
1. The idea of "shortest successful perturbation" appears like a type of weak training attack, looking for minimum perturbations to just cross the classification boundary, like deepfool [1] or confidence 0 CW-L2 attack [2].
2. The margin d_\theta in Equation (1)/(2)/... defined on which norm? L_\infty or L2 norm? I assume it's the infinity norm. In Theorem 2.1, the \delta^{*} = argmin ||\delta||, is a norm? Looks like a mistake.
3. The minimum margin \delta^{*} is a bit confusing, is it used in maximization or just in the outer minimization? The last paragraph of page 3, L(\theta, \delta) or L(\delta, \theta), consistency check?
4. Why do we need the "gradients of margins to model parameters" analysis from Proposition 2.1 to remark 2.2? Given the \delta^{*} found in the inner maximization (eg. attacking) process (step 1), minimizing the loss over this \delta^{*}  seems quite a straightforward step 2. Why don't go directly from Theorem 2.1 to Proposition 2.4, since the extensions from LM loss to SLM and CE loss via Proposition 2.3 -> Proposition 2.4., just proves that the standard classification loss CE can already maximize the margin given \delta^{*}?
5. Section 4 Experiments. The experimental settings are not clear, and are not standard. What CIFAR10-\ell_{\infty} means: is it the CW-L2 attack, used for training, or for testing? How the test attacks were generated, the m and N, are confusing: for each test image, you have 260 samples for CIFAR10 (which means 260*10K in total), or just 260 in total (this is far less than a typical setting causing inaccurate results)? How are the d_max determined, and what are their relationship to standard \epsilon? How the m models were trained?
6. Fairness of the comparison. Since MMA changes \epsilon, how to fairly compare the robustness to standard epsilon bounded adversarial training is not discussed. Is it fair to compare MMA-3.0 vs PGD-2.5, since they have different epsilon? Why robustness was not tested against strong, unrestricted attacks like CW-L2 [2], and report the average L2 perturbations required to completely break the robustly trained model (and show MMA-trained models enforce large perturbations to succeed)?
7. Significance of the results. Normally, \epsilon_{infty} > 16/255 will cause perceptual difference. Under 16/255, PGD-8/16, PGDLS-8/16 are still the best. At this level, it is quite a surprise that MMA does not improve robustness, although it does increase clean accuracy. This means the theoretical analysis only stand under certain circumstances. I don't think the optimal \epsilon < margin can explain this, as it does not make sense to me the margin can be larger than 16/255. On the other hand, I thought the theoretical parts were discussing the  ROBUSTNESS, not the CLEAN ACCURACY? But it turns out the MMA benefits a lot the clean accuracy?  Why do we need robustness against large \infty perturbations, this definitely deserves more discussion, as when perturbation goes large, the L2 attack (eg. CW-L2) makes more sense than PGD-\infty.
[1] Moosavi-Dezfooli, Seyed-Mohsen, Alhussein Fawzi, and Pascal Frossard. "Deepfool: a simple and accurate method to fool deep neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[2] Carlini, Nicholas, and David Wagner. "Towards evaluating the robustness of neural networks." 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 2017.
[3] Wang, Yisen, et al. "On the Convergence and Robustness of Adversarial Training." International Conference on Machine Learning. 2019.
============
My rating stays the same after reading through all the responses. I appreciate the authors' clarification on the notations and experimental settings. My 8/9 are positive points. My major concern is still the effectiveness of the proposed approach, and fairness of the comparison.  It seems that MMA only works when the perturbation is large, which often larger than the \epsilon used to train baseline adversarial training methods such as Trades. The authors seem have misunderstood my request for CWL2 results, I was just suggesting that the average L2 perturbation of CWL2 attack can be used as a fair test measure for robustness, instead of the AvgRobAcc used in the paper, and the susceptible comparison between  MMA-12 vs PGD-8, or MMA-32 vs Trades.


For AI systems to garner widespread public acceptance, we must develop methods capable of explaining the decisions of black-box models such as neural networks. In this work, we identify two issues of current explanatory methods. First, we show that two prevalent perspectives on explanations-feature-additivity and feature-selection-lead to fundamentally different instance-wise explanations. In the literature, explainers from different perspectives are currently being directly compared, despite their distinct explanation goals. The second issue is that current post-hoc explainers are either validated under simplistic scenarios (on simple models such as linear regression, or on models trained on syntactic datasets), or, when applied to real-world neural networks, explainers are commonly validated under the assumption that the learned models behave reasonably. However, neural networks often rely on unreasonable correlations, even when producing correct decisions. We introduce a verification framework for explanatory methods under the feature-selection perspective. Our framework is based on a non-trivial neural network architecture trained on a real-world task, and for which we are able to provide guarantees on its inner workings. We validate the efficacy of our evaluation by showing the failure modes of current explainers. We aim for this framework to provide a publicly available, 1 off-the-shelf evaluation when the feature-selection perspective on explanations is needed.

-------------------- AFTER
The original rating of "Weak Reject" still holds as the authors failed to provide proper justification for the raise concerns and support their claims through additional experiments.
"We do not introduce an explanation generation framework, as explainers do. " - The proposed evaluation requires the explainer of the NLP model to agree with the RCNN in-terms of relevant or irrelevant words, to be considered a good explainer. The RCNN model which is defining the relevant and irrelevant tokens for a prediction task is in fact stating that we can explain the decision of an NLP model in terms of relevant and irrelevant tokens. Hence, the proposed RCNN can also be considered as an explainer. The evaluation task is demonstrating if the other explainers are providing explanations consistent with this new explainer based on RCNN.
"The RCNN is not meant to explain any other models except itself." - Unclear
"Regarding the request for more experiments:" - The authors don't provide enough justification to "why they didn't perform more experiments?"
" Hence, with our current instantiations, any domain-agnostic explainer can be evaluated" - The experiment to validate this claim are missing.
"The novelty of our paper consists in the fact that, to our knowledge, it is the first to (1) shed light over a fundamental difference" - This is not a technical novelty. This is an exploratory analysis based observation
"and (2) propose a methodology for evaluating explainers that ...and without human intervention (unlike evaluation type 4)." -  In Section 5 Qualitative Analysis, the authors are also doing human evaluation like other methods in evaluation type-4 of their related works. Also, doing human evaluation is a strong way to justify an explainer. Though expensive, whenever possible it should be done and is in no way a limitation of current evaluation metrics.
Your model needs labelled data for training RCNN. This adds a constraint on the usability and scalability of your proposed evaluation method. Since RCNN is also black-box, one will required another explainer to explain the RCNN.
In the worst-case scenario, if RCNN is trained with data such that it considers all relevant words as irrelevant, the evaluation made by RCNN will be incorrect. Hence, " Success depends on the ability of the RCNN to extract correct subsets of tokens."
-------------------  BEFORE
The paper proposed a verification framework to evaluate the performance of different explanatory methods in interpreting a given target model. Specifically, the authors evaluated three explanatory methods namely, LIME, SHAP and L2X for a target model trained to perform sentiment analysis on text data. Authors assume for each input text, there is a subset of tokens that are most relevant and that are completely irrelevant to the final prediction task.  The proposed framework uses a recurrent convolutional neural network (RCNN) to find these subsets. The performance of an explainer is evaluated in terms of overlap between the RCNN most relevant tokens and the most relevant tokens provided by the explainer as an explanation.
Major
•	The paper lack technical novelty.
•	The proposed architecture uses a RCNN to find the most relevant subset of tokens. Firstly, RCNN is also a black box that provides no intuition behind its selection decision. Secondly, in the absence of the ground truth labels for true relevance and irrelevance of a token in input sentence, this explainer method can also suffer from “assuming a reasonable behavior” assumption. The method assumes that the RCNN is performing reasonably in identifying relevant subsets.
•	The success of the method depends on the ability of the RCNN to extract correct subsets of tokens. The data used for training the RCNN, might have some underlying bias. In that case, the evaluation is not accurate.
•	In related work, for “Interpretable target models” the authors mentioned LIME as an example of explainer functions that explains target models that are “very simple models may not be representative for the large and intricate neural networks used in practice”. LIME locally explains the decision of a complex function for a given data point using simpler models like linear regression. But LIME itself can be used for generating explanation for prediction of complex neural network like Inception Net.
•	The example used to explain the difference between feature additive and feature selection-based explainer methods, is confusing. Its not clear how in health diagnostics, one will prefer feature-selection perspective. Although the most relevant features used for the instance are important to understand the decision, but in clinical settings sometimes low rank features can also be useful to understand the target model.
•	For text, the relevant features are the individual tokens of the input sentence. Similarly, for images relevance can be important regions of the image. The authors did not have any experiments on images or tabular data.
•	In the experiment section, the comparison is made with only 3 explainer models and for just one task. The experiments are inadequate.
•	In Figure 4, the colormap is not readable.


Current literature in machine learning holds that unaligned, self-interested agents do not learn to use an emergent communication channel. We introduce a new sender-receiver game to study emergent communication for this spectrum of partially-competitive scenarios and put special care into evaluation. We find that communication can indeed emerge in partially-competitive scenarios, and we discover three things that are tied to improving it. First, that selfish communication is proportional to cooperation, and it naturally occurs for situations that are more cooperative than competitive. Second, that stability and performance are improved by using LOLA (Foerster et al., 2018a), especially in more competitive scenarios. And third, that discrete protocols lend themselves better to learning cooperative communication than continuous ones.

This paper looks at the question of emergent communication amongst self-interested learning agents. The paper finds that "selfish" (ie. self-interested) agents can learn to communicate using a cheap talk channel as long as the objective is partially cooperative.
The paper makes states that this is is a novel finding that contradicts the previous understanding of emergent communication in the literature (side point: at least some of the papers referenced for this claim did not at all make the claim).
I believe there is a major miss-understanding here: As noted in the paper, self-interested agents can learn to communicate in settings in which the reward function is cooperative. Furthermore, it is also known that in 2 player zero-sum there is no incentive to learn a communication protocol.
This clearly shows that talking about whether or not "selfish" agents can learn to communicate only ever makes sense within the context of a specific game / reward structure.
With this in mind, the main finding, agents learn to somewhat communicate with each other in a simple toy setting, with more communication happening when the payouts are more cooperative, is not very interesting.
This doesn't mean that there isn't a good paper to be written here, in principle. Finding simple settings in which SOTA multi-agent learning "fails", ie. doesn't find Nash policies, understanding why it fails and then finding ways to mend things is generally a good research direction. However, this would require a few things which are currently lacking from the paper: (1) clear understanding of the Nash policies for the different reward settings (2) Implementation of SOTA methods for MARL which are appropriate for this setting (3) In depth analysis of learning successes and failures, ideally in settings which have previously been studied in literature (given how task-specific this analysis necessarily is).
Regarding 2: General sum games will generally have mixed-strategies as Nash equilibria (just think 'rock-paper-scissors'). With this in mind, using a deterministic policy for the receiver is inappropriate for making any claims about learning in general sum games.
Furthermore, it is well known that independent gradient descent (IGD) is not generally going to converge in general sum games (consider the loss functions X * Y and - X *Y or matching pennies). So looking at the outcome of IGD without checking for convergence means the results could be just about anything. Indeed, we don't have to go all the way to writing about emergent communication or complex "sequential social dilemma" to study this, those issues can easily be found in (iterated) matrix games.
This gets us to the second major point of the paper. To the authors' credit,  LOLA [1] has been shown to help with convergence in general sum settings and to lead to the emergence of cooperation and reciprocity in iterated games.
However, the key point for the ‘cooperation’ part is iterated. In a single shot setting (which is explored in this paper), there is simply no way for the agents to reciprocate with each other. So in short, I do not believe the authors' interpretation that agents learn to cooperate with each other because of LOLA, but I do believe that LOLA can help with the learning of mixed strategies (at least for the sender, given that the receiver is deterministic) and with stabilizing convergence. Lastly, the part of the experimental section is dominated by large error bars and graphs that are difficult to interpret.
Other points:
-"..but train agents to emerge their own." (and many other instances). AFAIK "to emerge something" is grammatically wrong (and also sounds really odd).
-"Since the loss is differentiable with respect to the receiver, it is trained directly with gradient descent, so we are training in the style of a stochastic computation graph (Schulman et al., 2015).". This is a weird statement. You don't need SCGs for training a supervised objective. Also, note that the loss is also differentiable with respect to the action of the 1st agent. It is trivial in this setting to compute the true expected return, if that is what you are after. Note my point above about deterministic policies
-"We perform a hyperparameter search to over both agents’" -> spurious "to"
-"We investigate a similar scenario but concern ourselves with learning agents as opposed to fully-rational agents that have full knowledge of the structure of the game, and we do not assume that agents use an existing language, but train agents to emerge their own" .This would be interesting, if the game was complex.
- L_1 vs L - these symbols are used inconsistently, with the subscript _1 sometimes being applied and sometimes not.
-"we can look to extant results" - s/extant/extent?
-"We use the L2 metric only on hyperparameter search and keep L1 as our game’s loss to maintain a constant-sum game for the fully competitive case." - A few points: (a) the game is not in general constant sum (b) By doing this hyperparameter search the evaluation is strongly biased towards 'fair' attributions. This seems highly problematic.
-"We report our results in Figure ??" -> Broken reference.
-"We do not test b = 180◦ because the game is constant-sum and therefore trivially Ls1 + Lr1 = 180◦." -> So? It would still be interesting to see what learning agents do in this setting.
[1]: "Learning with Opponent Learning Awareness", Foerster et al.
[update: I have updated the score based on the discussion with the authors]. While the paper lacks execution and conceptual clarity, I believe the game itself is interesting and could serve as a starting point for more thorough investigation.


The successful application of flexible, general learning algorithms to real-world robotics applications is often limited by their poor data-efficiency. To address the challenge, domains with more than one dominant task of interest encourage the sharing of information across tasks to limit required experiment time. To this end, we investigate compositional inductive biases in the form of hierarchical policies as a mechanism for knowledge transfer across tasks in reinforcement learning (RL). We demonstrate that this type of hierarchy enables positive transfer while mitigating negative interference. Furthermore, we demonstrate the benefits of additional incentives to efficiently decompose task solutions. Our experiments show that these incentives are naturally given in multitask learning and can be easily introduced for single objectives. We design an RL algorithm that enables stable and fast learning of hierarchical policies and the effective reuse of both behavior components and transition data across tasks in an off-policy setting for complex, real-world domains. Finally, we evaluate our algorithm in simulated environments as well as physical robot experiments and demonstrate substantial improvements in data data-efficiency over competitive baselines.

While this paper has some interesting experiments. I am quite confused about what exactly the author are claiming is the core contribution of their work. To me the proposed approach does not seem particularly novel and the idea that hierarchy can be useful for multi-task learning is also not new. While it is possible that I am missing something, I have tried going through the paper a few times and the contribution is not immediately obvious. The two improvements in section 3.2 seem quite low level and are only applicable to this particular approach to hierarchical RL. Additionally, it is very much not clear why someone, for example, would select the approach of this paper in comparison to popular paradigms like Option-Critic and Feudal Networks.
The authors mention that Feudal approaches "employ different rewards for different levels of the hierarchy rather than optimizing a single objective for the entire model as we do." Why reward decomposition at the lower levels is a problem instead of a feature isn't totally clear, but this criticism does not apply to Option-Critic models. For Option-Critic models the authors claim that "Rather than the additional inductive bias of temporal abstraction, we focus on the investigation of composition as type of hierarchy in the context of single and multitask learning while demonstrating
the strength of hierarchical composition to lie in domains with strong variation in the objectives such as in multitask domains." First of all, I should point out that [1] looked at applying Option-Critic in a many task setting and found both that there was an advantage to hierarchy and an advantage to added depth of hierarchy. Additionally, it is well known that Option-Critic approaches (when unregularized) tend to learn options that terminate every step [2].  So, if you generically apply Option-Critic, it would in fact be possible to disentangle the inductive bias of hierarchy from the inductive bias of temporal abstraction by using options that always terminate.
In comparison to past frameworks, the approach of this paper seems less theoretically motivated. It certainly does not seem justified to me to just assume this framework and disregard past successful approaches even as a comparison. While the experiments show the value of hierarchy, they do not show the value of this particular method of creating hierarchy. The feeling I get is that the authors are trying to make their experiments less about what they are proposing in this paper and more about empirical insights about the nature of hierarchy overall. If this is the case, I feel like the empirical results are not novel enough to create value for the community and too tied to a particular approach to hierarchy which does not align with much of the past work on HRL.
[1] "Learning Abstract Options". Matthew Riemer, Miao Liu, and Gerald Tesauro. NeurIPS-18.
[2] "When Waiting is not an Option: Learning Options with a Deliberation Cost" Jean Harb, Pierre-Luc Bacon, Martin Klissarov, and Doina Precup. AAAI-18.


It is widely known that well-designed perturbations can cause state-of-the-art machine learning classifiers to mis-label an image, with sufficiently small perturbations that are imperceptible to the human eyes. However, by detecting the inconsistency between the image and wrong label, the human observer would be alerted of the attack. In this paper, we aim to design attacks that not only make classifiers generate wrong labels, but also make the wrong labels imperceptible to human observers. To achieve this, we propose an algorithm called LabelFool which identifies a target label similar to the ground truth label and finds a perturbation of the image for this target label. We first find the target label for an input image by a probability model, then move the input in the feature space towards the target label. Subjective studies on ImageNet show that in the label space, our attack is much less recognizable by human observers, while objective experimental results on ImageNet show that we maintain similar performance in the image space as well as attack rates to state-of-the-art attack algorithms.

This paper proposes a method for constructing adversarial attacks that are less detectable by humans, by changing the target class to be a class similar to the original class of the image. The resulting attack methodology is then studied in terms of its imperceptibility in label space, and shown to be less perceptible in label space to human observers, while not coming at a cost in image space.
The paper presents compelling evaluation of the method and does seem to succeed in proving that their proposed attack satisfies the stated goal. However, it appears as though this goal is somewhat counter to the main point of adversarial examples---indeed, if the label is reasonable to a human, then what makes the adversarial example adversarial? The main threat in adversarial examples research seems to be that it is possible to induce predictions that are arbitrarily different from humans' on natural-looking in puts. Thus, changing the label to something that a human actually agrees with would actually reduce the impact of the adversarial attack.
In order to improve the paper, I would suggest applying the same (or similar) methodologies to other areas of ML security where imperceptibility in label space is commonly desired---for example, in data poisoning attacks or backdoor attacks. In general, such attacks are much more likely to be "inspected" by humans, and so imperceptibility in both label and image space is very desirable. However, I suspect that this would require significant effort and changes to the paper, and so for now I recommend rejection.


We study the effect of using different object information for visual relation detection with a focus on depth maps. State of the art visual relation detection methods mostly rely on object information extracted from RGB images such as predicted class probabilities, 2D bounding boxes and feature maps. In this paper, we argue that the 3D positions of objects in space can provide additional valuable information about object relations. This information helps not only to detect spatial relations, such as standing behind, but also non-spatial relations, such as holding. Since 3D information of a scene is not easily accessible, we incorporate a pre-trained RGB-to-Depth model within a visual relation detection framework and release a new dataset VG-Depth as an extension to Visual Genome. We note that given the highly imbalanced distribution of relations in the available datasets, typical evaluation metrics for visual relation detection cannot reveal improvements of under-represented relations. To address this problem, we propose using Macro Recall@K for visual relation detection. Our experiments confirm that the performance of state-of-the-art visual relation detection approaches can significantly be improved by utilizing depth map information.
Relational learning is an established research area in machine learning. State-of-the-art approaches (Nickel et al., 2016) describe relations as triples of the form (subject, predicate, object), such as (Man, rides, Bike). The triples, which all together form a knowledge graph, are typically extracted from structured data such as the infoboxes in Wikipedia and other sources.
In the last years, the detection of relations from images, i.e., visual relation detection, has also gained a lot of interest. One reason is that understanding relations between entities can play an important role in decision making. For example, detecting whether a man is on a bike or next to a bike is a crucial challenge in autonomous driving. State-of-the-art works in this area utilize information of objects in the scene such as class labels, bounding boxes and RGB features, to capture pairwise relations using different models. In this paper, we argue that relation detection can additionally benefit from objects' 3D information. This information can help to distinguish between many relations such as behind, in front of and even improve detection in situations where the objects are nearby such as covered in.
Unfortunately, most available datasets, specifically the ones with relational annotations such as Visual Relation Detection (VRD) (Lu et al., 2016) and Visual Genome (VG) (Krishna et al., 2017), lack this 3D information since acquiring them is a cumbersome task requiring specialized hardware. We solve this issue by synthetically generating the corresponding depth maps of images in these datasets. Depth maps provide the objects' distance from the camera and the availability of large corpora of RGB-D pairs, e.g. NYU-Depth-v2 (Nathan Silberman & Fergus, 2012) dataset enables us to learn the mapping between any RGB image to its corresponding depth map using a fully convolutional neural network. We can then apply the trained network on images from VG, converting them into depth maps. We call this dataset extension VG-Depth 1 .
The extracted features from depth maps, together with other object information extracted from the RGB images, are the basis for relation detection in our framework. We note that the typically employed Recall@K metric, which we call as Micro Recall@K, cannot properly reveal the improvements of under-represented relations in highly imbalanced datasets such as Visual Genome. This might be an issue in applications such as autonomous driving where it is important to ensure that the model is capable of predicting also more important but less represented predicates such as walking on Figure 1: An image from the Visual Genome dataset showing several men sitting next to each other and the corresponding generated depth map. Bright colors indicate a larger distance to the camera. Knowing the depth of, e.g., the individuals, the house and the car, plays an important role in detecting relations such as next to and behind.
(648 in VG test set) and not just wearing (20,148 in VG test set). We tackle this issue by proposing to employ Macro Recall@K, where the Recall@K is computed per predicate class and averaged over, thereby eliminating the effect that over-represented classes have in Micro Recall@K setting. Reporting on both Macro@K and Micro@K, we study the effect of each object information on visual relation detection.
In summary, our contributions are as follows:
1. We perform an extensive study on the effect of different object information in visual relation detection, including 3D information. We show in our empirical evaluations using the VG dataset, that our model can outperform competing methods by a margin of up to 8% points, even those with contextualization or using information extracted from external language sources. 2. We propose using Macro Recall@K to better reveal the improvements in visual relation detection when dealing with highly imbalanced datasets such as Visual Genome. 3. To compensate for the lack of 3D data in Visual Genome, we use an RGB-to-Depth model trained on separate corpora of available pairs in both modalities and apply it to VG images and releasing a new dataset, VG-Depth.
In general, in this work we aim to answer the following questions:
1. If we are given only depth maps of unknown objects in a scene, how accurately can we infer the distribution of possible pairwise relations? How do other sources of object information compare to it? 2. Current visual relation detection frameworks commonly rely on extensive object information such as class labels, bounding boxes, RGB features, contextual information, etc. Does it bring any additional information, if we also include depth representations in these frameworks or would that only bring redundant knowledge about the scene? 3. Do the current evaluation metrics properly reflect the improvements of under-represented relations within highly imbalanced datasets?

********* Post Rebuttal *********
I appreciate the authors' effort in providing thorough responses and revised manuscript.
I agree with the authors that "the finding not being surprising" is not a ground for rejection. I tried to word my final decision carefully but it seems it has still caused confusion for the authors. As I have mentioned in my original review, the rating was a result of the 4 points considered *together* .
That is, if one exploits privileged information that needs extra sensory data and/or annotation (point 2), *and*, this privileged information is clearly related and thus should be normally useful for the final task (point 1), *and* achieve marginal improvements (point 3), it can be a ground for rejection. Especially, given that prior works with similar arguments exists (point 4).
The rebuttal has alleviated the issue of marginal improvements (point 3) by introducing meanR@K (or as the revised paper refer to it, Macro R@K). Here, the improvements are more significant both compared to the state of the art and ablated baselines.
The authors also argue that the related [Yang et al. 2018] paper (point 4) should be considered a concurrent submission since the authors original submission was to AAAI18.
The rebuttal also addresses other clarity or experimental issues which improves the quality of the revised work.
Finally, I understand that the privileged information is only required during training time which is a good point.
All in all, *assuming that [Yang et al. 2018] is considered a concurrent work* according to ICLR, I think the revised paper becomes slightly above borderline and thus I change my rating to "weak accept". If [Yang et al. 2018] is not considered concurrent work, then, a conclusive comparison is required for the acceptance of the current work.
********* Summary *********

The paper poses the question of whether depth information is informative for visual relationship prediction using still images. It is intuitive that 3D arrangement of objects in an image can be a useful cue for predicting their relationship. As such it is important to see whether and to what extent depth information complements RGB information for visual relation detection. That is the focus of this paper.
The paper proposes to use an off-the-shelf monocular depth estimation networks to augment the available RGB information towards better visual relation detection. For that, it proposes a specific network two-stream structure working on RGB image and (predicted) depth image. The proposed model demonstrates improved results upon state of the art for visual relation prediction.

********* Strengths and Weaknesses *********

+ A comprehensive set of tests has been conducted.
+ Zero-shot prediction results are particularly interesting.
+ The experiment on ranking the predicate classes based on the change in prediction accuracy before and after using depth information (Figure 4) is interesting and intuitive.
* The final results improve upon the state-of-the-art, especially on the zero-shot learning regime. However, it seems that the improvement is mainly coming from the new architecture as opposed to the inclusion of the depth information. That is, ours_{c,v,l} brings most of the improvement already the last step to ours_{c,v,l,d} is negligible for non-zero-shot case.
- Along the same line, it’s possible that this small difference between ours_{c,v,l} and ours_{c,v,l,d} for the standard predicate prediction, can be due to a hyper-parameter optimization that is (only or more thoroughly) done for ours_{c,v,l,d}. The hyper-parameter optimization scheme for different baselines is not described.
- Given the small difference of ablation levels, the comparison will be stronger if done multiple times and reporting mean and standard deviation of the results.
- For a fair comparison the visual feature vector v_{so} should be tried as the feature of the union bound box of both subject and object same way as it is done for depth feature vector d_{so}.
- The paper refers to “Ours-d’_{so}” as a baseline that *only* uses depth information with no image/label information. However, it seems that the region proposals for this feature are coming from the image-based network that uses image information.

- Important related but uncited works:
(1) [“Visual Relationship Prediction via Label Clustering and Incorporation of Depth Information” ECCV workshops 2018] studies the same question as part of their work.

********* Final Decision *********

I do not find the paper passing the acceptance bar mainly due to the following reasons together:
1) The finding is not surprising since most of the visual relations are either explicitly depth-related (e.g., behind) or are semantically constrained by depth (e.g. riding cannot happen at different depths when the image is taken orthogonal to the rider).
2) an additional depth dataset is used which provides the model with privileged information. Should it have been the case that depth information were inferred without an additional offline dataset, the results would have been more interesting.
3) the improvements due to the additional depth network are not significant or conclusive.
4) there is a prior uncited work with the same research question for effectiveness of depth information in visual relation detection which uses a similar approach.

********* Minor points *********
- the code is not available. This is especially important since the paper is outperforming prior works which could be a contribution if reproducible.
- Section 2.2: is l_{so} concatenation of l_s and l_o?
- Section 2.2: y_{spo} is defined but never used.
- Equation 2: why do we have both e_p and f in the exponents? Aren’t they the same?
- Equation 2:  P is never defined.
- Page 5: “a fully connected hidden layer of 64, 200, 4096 and 20 neurons”: this amounts to 3 hidden layers.
- Why VGG network for visual feature and AlexNet for depth features?
- zero-shot learning results on visual genome is missing
- training procedure is a bit unclear: the text suggest that the fine tuning and/or learning of the three components might happen separately. It is important to clearly state if they are done in an end-to-end fashion and simultaneously or separately; and why.
- It’s good to name the method in table 2 in the same fashion as table 1. With the current naming (based on architecture) it is a bit confusing to understand the content without additional cross referencing. For instance AlexNet-BN - Raw seems to correspond to Ours_{c,v,l,d}
- Figure 4: the frequency represented as different shades of red or blue is really hard to notice especially on a printed paper. The red vs blue color coding is not necessary since the bars going up or down indicate the same quality. So, it might be better to use red/blue for frequency instead (e.g. dark red high frequency to dark blue low frequency)
- Section 3.2: the AlexNet reference seems wrong, it should be "ImageNet Classification with Deep Convolutional Neural Networks" NIPS , 2012
- The structure of section 3.5 is currently flat while the content seems to be nested (two experiments and two sets of corresponding discussions). It will read better if they are organized into subsections.

********* Points of extensions (improvement) *********
- I believe *unsupervised* discovery of depth information for visual relation detection can be an interesting direction since it is not limited to the availability of relevant depth dataset.
- It is not clearly motivated why one should use two separate networks for depth and RGB inputs in light of the additional complexity. For instance, it is good to discuss what is the advantage of the proposed (computationally more expensive) method over the following two simpler baselines:
- Faster RCNN is used on RGBD input to produce a single feature vector
- above case with RGB input but have the Faster RCNN predict the depth map as an auxiliary loss.


Pretrained language models attract lots of attentions, and they take advantage of the two-stages training process: pretraining on huge corpus and finetuning on specific tasks. Thereinto, BERT (Devlin et al., 2019) is a Transformer (Vaswani et al., 2017) based model and has been the state-of-the-art for many kinds of Nature Language Processing (NLP) tasks. However, BERT cannot take text longer than the maximum length as input since the maximum length is predefined during pretraining. When we apply BERT to long text tasks, e.g., document-level text summarization: 1) Truncating inputs by the maximum sequence length will decrease performance, since the model cannot capture long dependency and global information ranging the whole document. 2) Extending the maximum length requires re-pretraining which will cost a mass of time and computing resources. Whats even worse is that the computational complexity will increase quadratically with the length, which will result in an unacceptable training time. To resolve these problems, we propose to apply Transformer to only model local dependency and recurrently capture long dependency by inserting multi-channel LSTM into each layer of BERT. The proposed model is named as BERT-AL (BERT for Arbitrarily Long Document Understanding) and it can accept arbitrarily long input without re-pretraining from scratch. We demonstrate BERT-ALs effectiveness on text summarization by conducting experiments on the CNN/Daily Mail dataset. Furthermore, our method can be adapted to other Transformer based models, e.g., XLNet  and RoBERTa (Liu et al., 2019), for various NLP tasks with long text.

The paper proposes a methodology to overcome the problem of processing long sequences with a pre-trained Transformer model, which suffers from high computational costs due to the complexity being quadratic in the length of the sequence. The authors also point out that BERT needs to be retrained from scratch if sequences longer than the specified maximum length (512) are to be processed. Their method (BERT-AL) chunks the input text into segments of maximum length. Each segment is propagated through several layers of a pre-trained Transformer layer followed by a multi-channel LSTM. Herein, the positional embeddings in each segment are the same. The model is applied to extractive summarization, and directly compared to the BERTSUM model, which can only process documents up to length 512. The comparison is made for 4 application scenarios, each corresponding to an artificial maximum length after which the BERTSUM model truncates the input: after 8, 16, 128, and 256 tokens. The experimental results suggest that BERT-AL outperforms BERTSUM in all 4 scenarios: Substantially for max length 8 and 16, and marginally for 128 and 256. BERTSUM without any truncation still performs best, however.
I think the paper should be rejected for three main reasons: (1) The idea of using RNNs to overcome the long sequence problem in Transformers is already well studied. The specific proposed architecture may be new, but the design choices are not well justified. (2) The experimental evaluation is not convincing: The application scenarios are unrealistic, the dataset is not well-chosen, and the results are not impressive. (3) The structure and language of the paper needs to be polished.
Regarding (1): The problem of dealing with long sequences in Transformer models has been known for a long time, and there already exist several proposed solutions based on an RNN component, which are acknowledged in this paper. The authors propose another model for a similar purpose. While the model is interesting, the individual design decisions are not well-justified, e.g., why the LSTM is applied at each layer and why it needs to be a multi-channel LSTM. Neither a comparison with models from previous works nor an ablation study is provided, so that there is also no empirical justification for the model design.
The authors argue that the novelty in their model comes from its applicability to pre-trained models, e.g., BERT and XLNET. While the motivation for BERT is reasonable, it is questionable whether this will still be a relevant for future research, as XLNET already mitigates the problem through the reliance on Transformer-XL, which is conceptually very similar to BERT-AL. If future pretrained models account for long sequences already during pre-training, the motivation for this work is rather low.
Regarding (2): In order to show the superiority of BERT-AL over BERTSUM, it would've been natural to employ it to datasets whose data are beyond what standard BERT can handle, i.e., longer than 512 tokens. Instead, the authors chose to evaluate on 4 rather unrealistic application scenarios derived from the CNN/DailyMail dataset, which artificially limit the range of the pre-trained BERT model to much less than what it can actually handle. While I understand the idea behind this setup, the insight that BERT performs poorly if the text is truncated to 8 or 16 tokens is not helpful. Since the improvement of BERT-AL over the baselines becomes marginal for longer sequences, and the performance of BERTSUM, without truncating the text, is not reached, I am not convinced of the model's value. An evaluation on more suitable datasets could help here.
Regarding (3): The paper is difficult to read for two reasons. First, the paper is not structured in a way that facilitates the understanding of the proposed method. For example, it is not clear why the pre-training tasks of BERT (masked language modeling and next sentence prediction) are relevant enough to be so prominently described. If the method is applicable to many pre-trained Transformers (as was claimed), then there is no need to go into this level of detail. On the same note, there is no need to explain BERTSUM's Inter-Sentence Transformer and Recurrent Neural Network variants for half a page if they are dismissed in the next paragraph. Second, the paper contains a lot of typos and grammatical mistakes. This is not a deal-breaker by itself, but it makes it obvious that the paper needs substantial polishing before it should be published.
Questions and suggestions to the authors:
1) What is the advantage of choosing a multi-channel LSTM, where each segment is a channel? Wouldn't a single channel suffice that processes aggregated information from the segment (as the Transformer computes)? I think your paper would benefit from exploring different options like that experimentally to better justify your model decisions.
2) It is not clear to me how the information exchange between layers happens. If each layer consists of a single layer from a pre-trained Transformer followed by an LSTM, the input to the following layer of the pre-trained Transformer is the output of the LSTM, which is not part of the pre-trained model. That is, the following Transformer layer receives an input that is very different from what it has seen during pre-training, which we can not expect to work. If there is a misunderstanding on my side, could you please clarify that, or otherwise make it clearer in the paper?
3) I think a good baseline would be to do segment-wise encoding via the pre-trained Transformer, and then feed all sentence representations (i.e., the respective CLS tokens) from all segments into the Transformer-based summarization layer from BERTSUM. This would be a compromise between your model (doing segment-wise encoding but not using an LSTM) and BERTSUM, and thus could potentially highlight the importance of the LSTM.
4) I don't think showing the number of examples in your datasets as in Table 2 contributes much to the paper. Instead, I suggest to show statistics on the length of the documents / number of sentences etc., because these are directly relevant to your study.


Bayesian neural networks, which both use the negative log-likelihood loss function and average their predictions using a learned posterior over the parameters, have been used successfully across many scientific fields, partly due to their ability to 'effortlessly' extract desired representations from many large-scale datasets. However, generalization bounds for this setting is still missing. In this paper, we present a new PAC-Bayesian generalization bound for the negative log-likelihood loss which utilizes the Herbst Argument for the log-Sobolev inequality to bound the moment generating function of the learners risk. We explore the generalization and calibration properties of the learned posterior on several image classification benchmarks, showing that the proposed approach provides better generalization and uncertainty estimates.

Summary:
This paper proposes a PAC-Bayesian generalization bound for Bayesian neural networks. The author discuss that earlier generalization bounds hold for bounded loss function, whereas the proposed generalization bound considers negative log likelihood loss which can be unbounded. Therefore, earlier results do not hold here.
The technical approach used is along lines of PAC Bayes framework and specifically for this loss function which requires bounding the log-partition function, the authors follow the Herbst Argument for bounding the log-partition function.
Contribution:
The paper uses straight forward PAC Bayes approach and the only bottleneck is bounding the log-partition function for which the authors use an earlier result (Herbst Argument for bounding the log-partition function. )
Significance:
My biggest concern with this work is its significance. As we know classification loss is bounded and for regression loss, as long as we have bounded input and a Lipschitz function corresponding to the NN, the output is bounded. Also as authors mention, there have been two earlier results covering other unbounded loss functions. Therefore, I do not feel that extending those results to NLL is a good enough contribution. Especially since the extension uses a known approach  (Herbst Argument for bounding the log-partition function. )
Experiments:
* From the explanations, it seems each architecture is trained once, which is not acceptable. How can one refute the effect of a specific initial value? A good scientific practice entails having mean and variance bar for different values or at least repeating the experiment multiple times and reporting the avg.
* According to the paper, the architectures in table 2, fig 1 are made by keeping the number of parameters roughly the same. Then the authors increase the depth. Note that to keep the # of parameters the same, they have to decrease the width as they increase the depth. Therefore, this cannot be a correct analysis of effect of the depth. As depth is not the only parameter that is changed between the architectures.
Writing:
The writing is overall ok which some vague places such as
*first page, last paragraph, line 1: ".. our PAC Bayesian bound is tightest when we learn..." the authors do not discuss what is the space of options for the bound and only mention the case when it is tightest. Therefore the claim is confusing
*first page, last paragraph, last line: "..better uncertainty than the baseline". The authors do not specify the baseline which makes this whole claim vague.
Title:
This is a minor issue but worth mentioning. The title is vague and confusing. In the first read one might think this paper provides PAC-Bayesian bounds for usual NNs (which has been considered and written about many times in the literature). The authors should mention that the considered networks are Bayesian NNs.


We present a method to infer 3D location and orientation of vehicles on a single image. To tackle this problem, we optimize the mapping relation between the vehicle's wheel grounding point on the image and the real location of the wheel in the 3D real world coordinate. Here we also integrate three task priors, including a ground plane constraint and vehicle wheel grounding point position, as well as a small projection error from the image to the ground plane. And a robust light network for grounding point detection in autopilot is proposed based on the vehicle and wheel detection result. In the light grounding point detection network, the DSNT key point regression method is used for balancing the speed of convergence and the accuracy of position, which has been proved more robust and accurate compared with the other key point detection methods. With more, the size of grounding point detection network is less than 1 MB, which can be executed quickly on the embedded environment. The code will be available soon.

General:  The proposed method tries to improve vehicle identification and tracking by combining model-based and data-driven methods. The idea is appealing, especially the use of domain knowledge combined with a data driven method to improve a model-based task.
 The authors propose a set of model configurations on the waypoint detection, optimization techniques, a deep learning network topology and a data driven and domain knowledge based wheel detection mechanism.
Improvements on vehicle identification and tracking are not shown in the study, but different computer vision methods  are compared to each other.
The study has several shortcomings in consistently stating the problem and the result, completeness, reproducibility, quantification of results and the experimental methodology lacks  a systematic approach to isolate the effects of the different components of the proposed method.
It seems that the full method proposed yields better results compared to other way point based methods in a certain phase space. However, from the material shown, especially the lack of the experimental description and its systematic shortcomings,  I cannot judge if the components of the method can contribute to improved vehicle detection and tracking.
The paper can be strengthened by the following:
    Better framing of the problem
    Improve the descriptions on the experiments carried out
    Including a quantitative discussion on the results and the corresponding uncertainties
    Systematic studies on the behavior of the proposed method components including a mathematical description in the probability space

More detailed comments:
Better framing of the problem:
    The framing of the problem and the task lack  at least a qualitative discussion on the issues arising in using visual based vs Lidar/Radar. I disagree with the authors, that a visual system can replace these other systems, but rather enhance results under certain conditions. I am missing the discussion on the shortcomings of camera sensors with regards to the overall task: Day/Night, Fog, wheels are not visible from all angles and the impact on the target phase space. The authors give an outline, that there is a study to come tackling the performance on vehicle identification, therefore a qualitative discussion could be enough at this point.
Improve the descriptions on the experiments carried out
    So that someone else can reproduce it
Systematic studies on the behavior of the proposed method components including a mathematical description in the probability space
    The Experiment is not described at all and is therefore not reproducible. The results shown are in general the overall Precision and Recall on some dataset. The authors state at several occasions, that isolated measures on the method show better performance. E.g. the choice of the "..online hard keypoints mining method..", "..the fixed range of softmax inputs..", "..ensure the accuracy io vehicle yaw angle…". However, I cannot find proof of these statements in the material shown. Usually only the overall task performance is stated from which deductions on isolated effects cannot be drawn.
    Albeit mentioning in the introduction priors in a Gaussian Process framework, I have not noticed a notion of a random variable, a probability density distribution or the impact of the experiment on the prior pdfs.  A systematic study on the behavior of those pdfs is missing and only a few example picture of bounding boxes are shown.
Including a quantitative discussion on the results and the corresponding uncertainties
    Insert a quantitative  discussion on the experiments.  Insert estimates on uncertainties on the results or give at least a qualitative statement, if these are negligible.
Other: The paper is an application paper and does not offer novel advances for the ICLR community.


There is a growing interest in studying the languages emerging when neural agents are jointly trained to solve tasks requiring communication through a discrete channel. We investigate here the information-theoretic complexity of such languages, focusing on the basic two-agent, one-exchange setup. We find that, under common training procedures, the emergent languages are subject to an entropy minimization pressure that has also been detected in human language, whereby the mutual information between the communicating agent's inputs and the messages is minimized, within the range afforded by the need for successful communication. This pressure is amplified as we increase communication channel discreteness. Further, we observe that stronger discrete-channel-driven entropy minimization leads to representations with increased robustness to overfitting and adversarial attacks. We conclude by discussing the implications of our findings for the study of natural and artificial communication systems.

This paper sets up a couple discrete communication games in which agents must communicate a discrete message in order to solve a classification task.  The paper claims that networks trained to perform this case show a tendency to use as little information as necessary to solve the task.
I vote to reject this paper.
The experiments are not very interesting and I don't at all agree with the assertion of the paper.  The paper claims the networks use only the entropy necessary to solve the task, but there are two main problems with this assertion.  (1) their own experiments don't support this all that strongly, as in the limit of few hidden bits (left half of the x axis in Figure 1), the networks all had noticeable excess information, and (2) and perhaps most damning the paper applies entropy regularization on the sender during training?  Could it perhaps instead be the fact that the entropy of the sender was penalized as an explicit regularization term that the entropy of the senders messages tended to be small?
I also find the experimental design puzzling.  Why both reinforce and the 'stochastic computation graph' approach?  Treating the receiver's output as binary and stochastic without using the log loss of the bernoulli observation model is just giving up on a good gradient as far as the receiver is concerned.
The experiments done are much to simple and the protocol flawed.
The second set of experiments in Figure 3 were not left to converge, so I'm not sure how we can derive a great deal of insight.  Additionally, relaxing the gumbel softmax channel to being continuous rather than discrete technically ruins any argument that there is an entropy bottleneck present anymore, as theoretically even a single dimensional continuous signal could store an arbitrary amount of information.  If the paper wanted to, it could have upper bounded the mutual information between the input and the message using a variational upper bound.
--------------- Response to Response ---------------------------------
I'm editing here in light of continuing to look at the paper and the responses from the author below.
I have to still argue for a rejection of this paper.
I thank the authors for addressing my comments and I admit that at first I thought the paper was minimizing the entropy during training which would have been particularly bad.  While I was mistaken on that point, I still believe the paper is deeply flawed.
In particular, the paper makes a very bold claim, namely that "We find that, under common
training procedures, the emergent languages are subject to an entropy minimization pressure that has also been detected in human language, whereby the mutual
information between the communicating agent’s inputs and the messages is minimized, within the range afforded by the need for successful communication."  But if we are being honest here, the experiments are very lacking to support such a bold claim.
In particular there was one thing I was worried about upon reading the paper again, and is similar to the point raised by the other reviewers.  In Figure 1, we are shown the entropy only of those networks that have succeeded.  Naturally to succeed, the entropy i the message must be large enough to accomodate the size of the remaining bits we are trying to reconstruct.  That is why Figure 1 includes the dotted line, since the networks must be above that line to have good performance.  And the main evidence for the main claim of the paper is that the trained networks are above that line and arguably close to it.
Now, we know that there are clearly solutions to these tasks (in particular the Guess Number task) which could achieve good performance at noticeably higher entropy.  For instance we could take any minimal solution and simply split up each message into 8x different buckets, each of which had exactly the same behavior from the decoder.  This would give us a +3 in the entropy of our message space while having no effect whatsoever on the loss.  The claim of the paper is that under normal training procedures it seems like we don't find those solutions and instead seem to find minimal ones.
But after implementing a simplified version of the experiment in the paper (Notebook available here: https://nbviewer.jupyter.org/urls/pastebin.com/raw/ZF7g34GN ) I suspect something much simpler is going on.  The reason the solutions look minimal in Figure 1 is probably because the initialization chosen for the encoder they used in the paper tended to start at low entropies.  Imagine if all of the networks started out with an initial message entropy of around 3 bits.  Then Figure 1 could be explained by the problems with hidden bits ~< 3 bits simply preserved their entropy, which in order to solve the task with higher numbers of digits hidden we know requires some minimal budget, so they get sort of pushed up.  This could explain the figure, but we wouldn't claim this explains why we observe small entropy for the high number of hidden digits case.
In particular, if we initialized the encoders with higher entropy, we might expect that we fail to see this phenomenon.  That is exactly what I was able to show for myself in that notebook.  If you simply initialize the encoder to have high entropy, all of the solutions have high entropy and the observed effect goes away.
Overall, the paper as I said is low quality.  Several choices were made that don't make a lot of sense.  With the experiments being as small scale as they were, why not explicitly marginalize out the message (as I did in the notebook)?  Why use single layer neural networks to predict 256 x 1024 parameters? Why not just learn them directly?   If the paper aimed to mimic more standard setups and show that under those setups we observe this kind of minimal message entropy, then it would have to much better tease out the effects of all of these choices.
Why does the decoder use a mean field sigmoid bernoulli observation model to try to predict something is in one of ~32 states?  The missing digits are not independent given the message, why model them as so?  Is that part of the purported reason why these models show minimal entropy (cause it isn't discussed).
For such a simple problem, you could presumably analytically compute the gradient with respect to the loss and study whether that correlates with the gradient of the entropy.  There are several things I could imagine checking, none of which are checked in the paper.
The primary question the paper addresses is an interesting one.  But this paper does very little to carefully investigate that question.  I maintain my vote to reject.


Incorporating the desired invariance into representation learning is a key challenge in many situations, e.g., for domain generalization and privacy/fairness constraints. An adversarial invariance induction (AII) shows its power on this purpose, which maximizes the proxy of the conditional entropy between representations and attributes by adversarial training between an attribute discriminator and feature extractor (Xie et al., 2017). However, the practical behavior of AII is still unclear as the previous analysis assumes the optimality of the attribute classifier, which is rarely held in practice. This paper first analyzes the practical behavior of AII, indicating that AII has theoretical difficulty as it maximizes variational upper bound of the actual conditional entropy, and AII catastrophically fails to induce invariance even in simple cases as suggested by the above theoretical findings. We then argue that a simple modification to AII can significantly stabilize the adversarial induction framework and achieve better invariant representations. Our modification is based on the property of conditional entropy; it is maximized if and only if the divergence between all pairs of marginal distributions over representations between different attributes is minimized. The proposed method, invariance induction by discriminator matching, modify AII objective to explicitly consider the divergence minimization requirements. Empirical validations on both the toy dataset and four real-world datasets (related to applications of user anonymization and domain generalization) reveal that the proposed method provides superior performance when inducing invariance for nuisance factors.

*Summary*
The paper proposes a new method to learn data-driven representations, being invariant to some specific nuisance factors which are detrimental for the selected (supervised) classification task.
Authors build upon the existing probabilistic framework termed Adversarial Invariant Induction (AII) from (Xie et al., 2017).
They claim to explore it under a both theoretical and practical point of view, demonstrating the limitations of maximizing a variational upper bound on conditional entropy as a proxy to achieve invariance.
Leveraging these observation, authors propose a novel method, called “invariance induction by discriminator matching” (IIDM) that is based on a regularized classification loss, penalized by a Kullbach-Leibler divergence between conditional distributions of the nuisance factor.
Extremely convincing experiments are carried out on a synthetic and a real benchmark in multi-source domain generalization (PACS).
*Pros*
1. The genesis of the proposed IIDM is extremely paced since smoothly derived from the AII framework.
2. Experimental results on a synthetic benchmark (a version of rotated MNIST) and on a popular benchmark for domain generalization (PACS) proved the effectiveness of IIDM
 *Cons*
1. The paper is hard to get, if the reader is not familiar with related literature
2. It is not fully clear from the paper which parts are original and which are inherited from prior work.
3. The structure of the paper needs to be improved (check my comments in the section beneath)
Some of the proposed methodologies are not clear (IIDM+)

*Detailed Comments*
The problem considered by authors is surely interesting and addressing a popular topic in computer vision and deep learning.
1. Unfortunately, the paper, as it is is hard to get for scholars which are not expert of the AII formalism, which, in my opinion is not enough detailed. Therefore, in my opinion clarity is something that authors should try to work hard on: for instance, during the rebuttal time, authors can write from scratch an entire new Section in which they explain in plain terms the main outcomes of their paper, without entering too much into technical details.
2. Additionally, the structure of the paper needs, in my opinion a major re-styling, still for the sake of better readability:
2.a. A visualization of the proposed method (for instance, using flow-diagrams) in comparison with the existing AII should help in rapidly getting the factors of novelty of the proposed IIDM method. I would also encourage authors to add a pseudo-code
2.b. Since authors claim two major contributions (understanding AII + IIDM), I would like to see those two contributions thoroughly presented and dissected in two separated sections of the paper. I am not fully convinced with the actual writing style in which the two contributions seem to be intertwined together.
3. Although already convincing, the experimental part can be improved:
3.a. Instead of a version of rotated-MNIST, authors can test on the “digits-five” setting (MNIST, MNIST-M, SVHN, UPS, SYN) as done in several works like http://openaccess.thecvf.com/content_cvpr_2018/papers/Xu_Deep_Cocktail_Network_CVPR_2018_paper.pdf.
3.b. In addition to multi-domain generalization, authors could also have tried more classical unsupervised domain adaptation settings or, even, single-source domain generalization as in https://papers.nips.cc/paper/7779-generalizing-to-unseen-domains-via-adversarial-data-augmentation.
*Final Evaluation*
I think that the main aspect that authors should face during the rebuttal is to make the paper more easy to read and better separate the two contributions (understanding AII and IIDM). What I am not convinced at all about the writing style of the authors since when reading the paper I am not always capable of understanding what is novel (since proposed by authors) and what is inherited from prior work. But, maybe, the reason for this is that I am not an expert of the specific related field - but, even so, I think that the paper needs to be understood from the broadest audience possible.
Instead, I am familiar with multi-source/single-source domain generalization (and adaptation) and, after my careful analysis of the experiments, I see a lot of potential in the approach. I would me more than interested in checking the performance of the proposed method over some of the novel benchmarks that I have recommended. It would be nice if authors add more experiments, but I know that this is always a complicated request during a rebuttal period.
Globally, if I were asked to only rate the experimental part, I would have promoted for full acceptance. Unfortunately, the theoretical part of the paper is not fully clear to me and, therefore, I am not confident in calling for a full acceptance only based on the experiments.
In brief, I would go for a weak reject, looking forward to the authors’ rebuttal and the opinion of the other Fellow Reviewers.


The current approaches for machine translation usually require large set of parallel corpus in order to achieve fluency like in the case of neural machine translation (NMT), statistical machine translation (SMT) and example-based machine translation (EBMT). The context awareness of phrase-based machine translation (PBMT) approaches is also questionable. This research develops a system that translates English text to Amharic text using a combination of context based machine translation (CBMT) and a recurrent neural network machine translation (RNNMT). We built a bilingual dictionary for the CBMT system to use along with a large target corpus. The RNNMT model has then been provided with the output of the CBMT and a parallel corpus for training. Our combinational approach on English-Amharic language pair yields a performance improvement over the simple neural machine translation (NMT).

This paper presents a machine translation system based on a combination of a neural machine translation system (NMT) and a context-based machine translation (CBMT). The method is evaluated on a small parallel corpus application of English-Amharic translation. The idea is that in the small corpus setting, the CBMT can leverage a manually built bilingual dictionary
to improve on the standard NMT.
Clarity: The paper is reasonably clear, though there are numerous typos and minor language glitches (missing articles, etc.). Overall the quality of the writing is probably a bit below what's acceptable for publication at ICLR, but nothing that could not be fixed on subsequent revisions.
Novelty: The method proposed appears to be a fairly straightforward variant of one proposed in a previous paper, where an NMT system was combined with a phrase-based MT system (Zoph & Knight, 2016). There seems to be no novel machine learning contribution (nor is it claimed). This paper seems more appropriate for a venue more focused on machine translation rather than a machine learning venue such as ICLR.
Empirical evidence in support of the claims: The authors set out to demonstrate that by combining a CBMT output into an NMT approach, one can get the best of both approaches. Their results do not strongly support this claim. The results suggest that in the context of the small-scale experiments considered, the baseline CBMT model is actually overall the best performing model. It is therefore strange that, in their last sentence of the conclusion, the authors persist in claiming that their combination "outperforms a simple NMT and a basic CBMT system". That being said, the sub-claim that the
NMT/CBMT hybrid improves on the baseline NMT system is well established.
In light of the relatively low novelty and the lack of compelling empirical performance for the proposed combined MT system, I do not feel that this paper is appropriate for ICLR at this time.