This paper investigates the generalization properties of two-layer neural networks in high-dimensions, i.e. when the number of samples n, features d, and neurons h tend to infinity at the same rate. Specifically, we derive the exact population risk of the unregularized least squares regression problem with two-layer neural networks when either the first or the second layer is trained using a gradient flow under different initialization setups. When only the second layer coefficients are optimized, we recover the double descent phenomenon: a cusp in the population risk appears at h ≈ n and further overparameterization decreases the risk. In contrast, when the first layer weights are optimized, we highlight how different scales initialization lead to different inductive bias, and show that the resulting risk is independent of overparameterization. Our theoretical and experimental results suggest that previously studied model setups that provably give rise to double descent phenomenon might not translate to optimizing two-layer neural networks.

This paper provides exact bounds on the risk when training a two-layer neural network in an asymptotic regime. Namely, the paper considers training under the square-loss objective, a two-layer neural network with $h$ hidden units on inputs of dimension $d$ and training on $n$ samples. The asymptotic regime is considered by making all of $d$, $h$, $n$ go to $\infty$, in a way that the ratio $d/n$ approaches $\gamma_1$ and the ratio $h/n$ approaches $\gamma_2$.
This paper considers the following scenarios of training described below, where the data is generated from a linear model on Gaussian inputs and with a zero-mean noise. The emphasis of the results is on understanding when a "double descent" type phenomenon occurs ("Double descent" is a recently coined phenomenon in literature where the risk, as a function of the "complexity of the model", initially has a classical U-shape behavior, but eventually decreases again once the complexity of the model exceeds the number of training points.)
1. Training only the second layer: The risk is first decomposed into a bias and a variance term. An exact bound on the variance term of the risk is obtained. While the exact nature of the bound is rather complex to parse, the takeaway is that a double descent phenomenon is observed in terms of $\gamma_2$, namely, the risk blows up when $h \approx n$, but decreases as $h$ is increased beyond $n$.
2. Training only the first layer: Two different regimes are considered here, depending on the scale of initialization, called "vanishing" and "non-vanishing" initializations. In both regimes, the risk is independent of $\gamma_2$, that is, the risk does not depend on number of hidden units (although the risk bounds are different and there is an additional assumption in the case of non-vanishing initialization to ensure that the initialized network computes the zero function). In other words, a "double descent" phenomenon is not observed in this setting.
Recommendation:
I recommend "weak acceptance". The paper extends prior works that obtain asymptotic risk bounds on linear models to the setting of two-layer neural networks (where only one layer is trained).  However, I am unable to assess the technical novelty of this work as it seems to heavily rely on prior work which in turn use techniques from random matrix theory.
Technical Comments:
- I felt that while it is valuable to have exact bounds on the risk, the form of the bounds are quite complex and hard to parse (especially in Thm 4, case of training only the second layer). Moreover, these bounds are just in the case where the teacher model is linear and while it is claimed that this could be relaxed to a more general class of functions, the specific bounds might change drastically. So any insights on the nature of these bounds will be valuable, especially with some comments on how these bounds change if the teacher model is itself realized as a 2-layer neural network.
- The parameter count of a 2-layer network with $h$ hidden units and input dimension $d$ is $O(dh)$. So perhaps it makes sense to study an asymptotic regime where $dh/n$ approaches $\gamma$, instead of both d and h growing linearly in n. While this issue is hinted at in the discussion section, I don't understand the statement "the mechanism that provably gives rise to double descent from previous works Hastie et al. (2019); Belkin et al. (2019) might not translate to optimizing two-layer neural networks."
- Another future direction that could be included in discussions is the setting where both layers are trained simultaneously.


Deep learning natural language processing models often use vector word embeddings, such as word2vec or GloVe, to represent words. A discrete sequence of words can be much more easily integrated with downstream neural layers if it is represented as a sequence of continuous vectors. Also, semantic relationships between words, learned from a text corpus, can be encoded in the relative configurations of the embedding vectors. However, storing and accessing embedding vectors for all words in a dictionary requires large amount of space, and may strain systems with limited GPU memory. Here, we used approaches inspired by quantum computing to propose two related methods, word2ket and word2ketXS, for storing word embedding matrix during training and inference in a highly efficient way. Our approach achieves a hundred-fold or more reduction in the space required to store the embeddings with almost no relative drop in accuracy in practical natural language processing tasks.

This paper explores two related methods to reduce the number of parameters required (and hence the memory footprint) of neural NLP models that would otherwise use a large word embedding matrix. Their method, inspired by quantum entanglement, involves computing word embeddings on-the-fly (or by directly computing the output of the "word embedding" with the first linear layer of network). They demonstrate their method can save an impressive amount of memory and does not exhibit big performance losses on three nlp tasks that they explore.
This paper is clearly written (with only a couple of typos) but does not yet reach publication standard. Whilst the empirical performance of their approach is promising from the perspective of saving reducing memory requirements, more experiments are required and more careful comparisons to baselines and other methods in the literature for saving memory/parameters. In general the related work and experimental sections are weak and brief, with only superficial analysis. There is  lack of careful analysis and insight into their results, as well as a careful comparisons to other work in this area.
The choice of tasks to evaluate on is broad, which is a strength, but is missing simpler tasks that one would expect to see, such as a text classification dataset, or simple bag-of-vectors style models. In addition, the choice of models are somewhat outdated baselines. It seems that transformers would be an ideal setting for their approach, as transformers have rather high dimensional word embedding matrices, but the authors do not run experiments with the now-ubiquitous Transformer.
The quantum inspiration is largely a distraction, and I think the paper would benefit from this element being scaled back or removed in order to free up space for more experiments.
The authors acknowledge one key weakness of their approach, that both training and inference time are increased (by 28% or 55% longer for DocQA depending on compression)  but much more work could be done to understand the best way to  mitigate for longer training and inference times.
The authors argue that reducing the memory footprint of models is vital to address hardware limitations for training and inference for large models like BERT or ROBERTA, but this argument is not particularly strong. Generally current limitations for training these kinds of models  are the long training times and being able to fit large batches onto our hardware, and the vocabulary matrix is only a constant factor here. And since training time is a bottleneck, the added value of saving memory vs slowing the training speed by 30-50% is debatable.
Here are some questions for the authors that come to mind when reviewing:
How does your method compare to other published methods on your benchmarks?
which choices for r and k lead to the best time/memory/performance tradeoff? how does this compare to other compression methods (on your tasks)
Seq2Seq models usually involve multiplying the the output hidden state with a vocab matrix before softmaxing over all the vocabulary produce word probabilities - did you account for this? Does your method work for the output vocab matrix?
Did you investigate pre-training word2ket like word2vec or Glove?


Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence. Recently, classification-based methods were shown to achieve superior results on this task. In this work, we present a unifying view and propose an open-set method, GOAD, to relax current generalization assumptions. Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations. Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types. The strong performance of our method is extensively validated on multiple datasets from different domains.

UPDATE:
I acknowledge that I‘ve read the author responses as well as the other reviews.
I appreciate the clarifications, additional experiments, and overall improvements made to the paper. I updated my score to 6 Weak Accept.
####################
This paper proposes a deep method for anomaly detection (AD) that unifies recent deep one-class classification [6] and transformation-based classification [3, 4] approaches. The proposed method transforms the data to $M$ subspaces via $M$ random affine transformations and identifies with each such transformation a cluster centered around some centroid (set as the mean of the respectively transformed samples). The training objective of the method is defined by the triplet loss [5] which learns to separate the subspaces via maximizing the inter-class as well as minimizing the intra-class variation. The anomaly score for a sample is finally given by the sum of log-probabilities, where each transformation-/cluster-probability is derived from the distance to the cluster center. Using random affine transformations, the proposed method is applicable to general data types in contrast to previous works that only consider geometric transformations (rotation, translation, etc.) on image data [3, 4]. The paper conclusively presents experiments on CIFAR-10 and four tabular datasets (Arrhythmia, Thyroid, KDD, KDD-Rev) that indicate a superior detection performance of the proposed method over baselines and deep competitors.
I think this paper is not yet ready for acceptance due to the following main reason:
(i) The experimental evaluation needs clarification and should be extended to judge the significance of the empirical results.
(i) I think the comparison with state-of-the-art deep competitors [6, 4] should consider at least another image dataset besides CIFAR-10, e.g. Fashion-MNIST or the recently published MVTec [1] for AD. On CIFAR-10, do you also consider geometric transformations however using your triplet loss or are the reported results from random affine transformations? I think reporting both would be insightful to see the difference between image-specific and random affine transformations.
On the tabular datasets, how do deep networks perform in contrast to the final linear classifier reported on most datasets? Especially when only using a final linear classifier, the proposed method is very similar to ensemble learning on random subspace projections. Figure 1 (right) shows an error curve that is also typical for ensemble learning (decrease in mean error and reduction in overall variance). I think this should be discussed and ensemble baselines [2] should be considered for a fair comparison. Table 2 also seems incomplete with the variances missing for some methods?
Further clarifications are needed. How many transformations $M$ do you consider on the specific datasets? How is hyperparameter $s$ chosen?
Finally, I think the claim that the approach is robust against training data contamination is too early from only comparing against the DAGMM method on KDDCUP (Is Figure 1 (left) wrong labeled? As presented DAGMM shows a lower classification error).
Overall, I think the paper proposes an interesting unification and generalization of existing state-of-the-art approaches [6, 4], but I think the experimental evaluation needs to be more extensive and clarified to judge the potential significance of the results. The presentation of the paper also needs some polishing as there are many typos and grammatical errors in the current manuscript (see comments below).
####################
*Additional Feedback*
*Positive Highlights*
1. Well motivated anomaly detection approach that unifies existing state-of-the-art deep one-class classification [6] and transformation-based classification [3, 4] approaches that indicates improved detection performance and is applicable to general types of data.
2. The work is well placed in the literature. All relevant and recent related work is included in my view.
*Ideas for Improvement*
3. Extend and clarify the experimental evaluation as discussed in (i) to infer statistical significance of the results.
4. I think many details from the experimental section could be moved to the Appendix leaving space for the additional experiments.
5. Maybe add some additional tabular datasets as presented in [2, 7].
6. Maybe clarify “Classification-based AD” vs. “Self-Supervised AD” a bit more since unfamiliar readers might be confused with supervised classification.
7. Improve the presentation of the paper (fix typos and grammatical errors, improve legibility of plots)
8. Some practical guidance on how to choose hyperparameter $s$ would be good. This may just be a default parameter recommendation and showing that the method is robust to changes in s with a small sensitivity analysis.
*Minor comments*
9. The set difference is denoted with a backslash not a forward slash, e.g. $R^L \setminus X$.
10. citet vs citep typos in the text (e.g. Section 1.1, first paragraph “ ... Sakurada & Yairi (2014); ...”)
11. Section 1.1: “ADGMM introduced by Zong et al. (2018) ...” » “DAGMM introduced by Zong et al. (2018) ...”.
12. Eq. (1): $T(x, \tilde{m})$ in the first denominator as well.
13. Section 2, 4th paragraph: $T(x, \tilde{m}) \in R^L \setminus X_{\tilde{m}}$.
14. $m$, $\tilde{m}$, and $m'$ are used somewhat inconsistently in the text.
15. Section 3: “Note, that it is defined everywhere.”?
16. Section 4: "If $T$ is chosen deterministicaly ..." >> "If $T$ is chosen deterministically ..."
17. Section 5, first sentence: “... to validate the effectiveness our distance-based approach ...” » “... to validate the effectiveness of our distance-based approach ...”.
18. Section 5.1: “We use the same same architecture and parameter choices of Golan & El-Yaniv (2018) ...” » “We use the same architecture and parameter choices as Golan & El-Yaniv (2018) ...”
19. Section 5.2: “Following the evaluation protocol of Zong et al. Zong et al. (2018) ...” » “Following the evaluation protocol of Zong et al. (2018) ...”.
20. Section 5.2: “Thyroid is a small dataset, with a low anomally to normal ratio ...” » “Thyroid is a small dataset, with a low anomaly to normal ratio ...”.
21. Section 5.2, KDDCUP99 paragraph: “Tab. ??” reference error.
22. Section 5.2, KDD-Rev paragraph: “Tab. ??” reference error.
####################
*References*
[1] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9592–9600, 2019.
[2] J. Chen, S. Sathe, C. Aggarwal, and D. Turaga. Outlier detection with autoencoder ensembles. In SDM, pages 90–98, 2017.
[3] S. Gidaris, P. Singh, and N. Komodakis. Unsupervised representation learning by predicting image rotations. In ICLR, 2018.
[4] I. Golan and R. El-Yaniv. Deep anomaly detection using geometric transformations. In NIPS, 2018.
[5] X. He, Y. Zhou, Z. Zhou, S. Bai, and X. Bai. Triplet-center loss for multi-view 3d object retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1945–1954, 2018.
[6] L. Ruff, R. A. Vandermeulen, N. Görnitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, and M. Kloft. Deep one-class classification. In International Conference on Machine Learning, pages 4393–4402, 2018.
[7] L. Ruff, R. A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K.-R. Müller, and M. Kloft. Deep semi-supervised anomaly detection. arXiv preprint arXiv:1906.02694, 2019.


Multi-view learning seeks to form better models by making use of multiple feature sets representing the same samples. Exploiting feature correlations during training can enable better models. The traditional method for computing correlations between feature sets is Canonical Correlation Analysis (CCA), which finds linear projections that maximize correlation between feature vectors, essentially computing a shared embedding for two views of data. More recently, CCA has been used for multi-view discriminative tasks; however, CCA makes no use of class labels. Recent CCA methods have started to address this weakness but are limited in that they do not simultaneously optimize the CCA projection for discrimination and the CCA projection itself, or they are linear only. We address these deficiencies by simultaneously optimizing a CCA-based and a task objective in an end-to-end manner. Together, these two objectives learn a non-linear CCA projection to a shared latent space that is highly correlated and discriminative. Our method shows a significant improvement over previous state-of-the-art (including deep supervised approaches) for cross-view classification (8.5% increase), regularization with a second view during training when only one view is available at test time (2.2-3.2%), and semi-supervised learning (15%) on real data.

Originality:
CCA is a generative model that learns a shared subspace based on  two (or multi) views of the data. Being generative, it might not have strong discriminative power for some downstream classification tasks. Previous approaches to infuse discriminative power into the shared subspace estimated by CCA are linear. So, this paper proposes to learn 1) non-linear 2) discriminative subspaces for CCA. The paper accomplishes this by simply adding a task specific term to the optimization objective of DeepCCA (Andrew et. al. 2013), which involves just adding a task-specific MLP on top and minimizing the associated loss-function.
1). The novelty of the proposed approach is limited. It just adds an extra term (extra neural network layer) with a corresponding weighting hyperparameter to the objective function of a previous method (DeepCCA) without much motivation.
2). The experimental setup and results are sound but some of the tasks seem contrived to show the improved performance of TOCCA methods. For instance, in the cross-view MNIST classification the authors use only projection from one view at training time and use the other view at test-time. What's the motivation for this setup? Why not split the data into train and test set by splitting observations, then train on both the views at train time and test on the held-out observations at test-time? I hope I am not missing something.
3). Similarly, for the "Regularization for Cancer Classification" task, it's assumed that only one view is available at test time. Why is that? What are the real-world examples of such setups?
Quality:
The paper is technically sound, though it is a trivial extension of a previous method. The experimental setup is somewhat contrived to show the superiority of the proposed method.
Clarity:
The paper is well organized and is well written in general. The supplementary material contains more results and code will be available after the review period.
Significance:
The paper solves an important problem by infusing discriminative power into generative subspaces learned by CCA but the results are not that important in my eyes. Since the empirical setup is a little contrived it is hard to even know whether a simple two-step approach that first estimates CCA subspace and then uses those projections in a SVM or MLP would perform comparable or better if given a fair-chance to compete.


The loss landscapes of deep neural networks are poorly understood due to their high nonconvexity. Empirically, the local minima of these loss functions can be connected by a learned curve in model space, along which the loss remains nearly constant. Yet, current path finding algorithms do not consider the influence of symmetry in the loss surface caused by weight permutations of the networks corresponding to the minima. We propose a framework to investigate the effect of symmetry on landscape connectivity by directly optimizing the weight permutations of the networks being connected. To learn a locally optimal permutation, we introduce a proximal alternating minimization scheme and prove some convergence guarantees. Using an neuron alignment technique, we derive an inexpensive heuristic for approximating a locally optimal weight permutation. Empirically, optimizing the permutation is critical for efficiently learning a simple, planar, low-loss curve between networks that successfully generalizes. As an application, we also show that there is a performance gain, particularly for models with less parameters, in the accuracy of ensembles constructed from the learned curve with neuron alignment permutation.

The paper investigates the connection between symmetries in the neural network architectures and the loss landscape of the corresponding neural networks. In the previous works, there was shown that the two local minima of a neural network can be connected by a curve with the low validation/train loss along the curve. Despite the loss on the curve being close to the loss at the end points, there are segments of the curve on which the loss in higher than loss at the local minima. To overcome this problem, the authors proposed two-step procedure: 1. Align weights between two neural networks 2. Launch the path finding algorithm between aligned weights. In other words, the authors proposed to connect not original local minima but local minima that parametrize the same functions as the original ones, but have a different order of neurons. The authors also proposed PAM algorithm where they iteratively apply path finding algorithm and weights alignment algorithm.
Novelty and significance. The idea to combine the symmetrization of NN architectures with path finding algorithms is new to the best of my knowledge. Experimentally, the authors showed that ensembling the midpoint and endpoints of the curve found via path finding  algorithm coupled with the neural alignment algorithm delivers a better result than simple averaging of three independent networks. This is a new and significant result, since before the ensembling of points along the curve had the same quality as the ensemble of three independent networks or marginally better.
The weak side of the paper is the PAM algorithm that occupies a significant part of the paper and does not deliver a significantly better result than the simple application of the neural alignment procedure before launching the path finding algorithm.
Clarity. Overall, the related work section contains all relevant references to the previous works to the best of my knowledge. The paper is well written, excluding the section Neuron Alignment that lacks notation description.
The paper contains several typos and inaccuracies:
1. “However, We find its empirical performance is similar to standard curve finding, and we advocate the latter for practical use due to computational efficiency.” The word “We” should start with the lowercase letter.
2. In the sentence “The first question to address is how to effectively deal with the constraints in equation 6” the index i should be replaced with the index l.
3. Notation Π|Kl| introduced after equation 5.
4. In the section describing neuron alignment  algorithm Lie et al. [1] used a different notation. So I would recommend to further extend this section and add all necessary notations. Also, I would recommend to add a direct link to the paper where the problem is described in matrix form.
5. In the Algorithm 1 “Initialize P θ2 := [Wˆ 2 1 ,Wˆ 2 2 , ...,Wˆ 2 L ] as [W2 1 ,W2 2 , ...,W2 L ] for k ∈ {1, 2, ..., L − 1};”  k is not used anywhere in notation.
6. In Figure 3, “model 2 aligned ” sing is out of the plot box for ResNet-32 and VGG-16 architectures.
7. The appendix contains the sketch of the proof that is quite difficult to follow. I would recommend giving all necessary definitions as it is done in the [2] and extend the proof.
Overall, it is quite an interesting paper but it contains some drawbacks.
[1] Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John E Hopcroft.  Convergent learning: Do different neural networks learn the same representations?  In ICLR, 2016
[2]  H́edy Attouch, J́erˆome Bolte, Patrick Redont, and Antoine Soubeyran.  Proximal alternating minimization and projection methods for nonconvex problems:  An approach based on the kurdyka-łojasiewicz inequality.Mathematics of Operations Research, 35(2):438–457, 2010


Time-evolving graphs are ubiquitous such as online transactions on an e-commerce platform and user interactions on social networks. While neural approaches have been proposed for graph modeling, most of them focus on static graphs. In this paper we present a principled deep neural approach that models continuous timeevolving graphs at multiple time resolutions based on a temporal point process framework. To model the dependency between latent dynamic representations of each node, we define a mixture of temporal cascades in which a node's neural representation depends on not only this node's previous representations but also the previous representations of related nodes that have interacted with this node. We generalize LSTM on this temporal cascade mixture and introduce novel time gates to model time intervals between interactions. Furthermore, we introduce a selection mechanism that gives important nodes large influence in both k−depth subgraphs of nodes in an interaction. To capture temporal dependency at multiple time-resolutions, we stack our neural representations in several layers and fuse them based on attention. Meanwhile, our method can process unseen nodes and their interactions. Experimental results on interaction prediction and classification tasks -including a real-world financial application -illustrate the effectiveness of the time gate, the selection and fusion mechanisms of our approach, as well as its superior performance over the alternative approaches.

The paper focuses on the problem of modeling interaction processes over dynamically evolving graphs and perform inference tasks such future interaction prediction and interaction classification. Specifically, the paper proposes a temporal point process based formulation to model the interaction dynamics where the conditional intensity function is parameterized by a recurrent network. With an occurrence of any event, the recurrent architecture updates the embeddings of the nodes involved in that event which then affects the intensity function and hence the likelihood of future events. The paper uses intensity based likelihood to train for future interaction prediction task while cross-entropy based loss for classification task. The paper demonstrates the efficacy of the method through experiments across multiple datasets and compare against representative baselines and further provides ablation analysis for the proposed architecture.
The paper demonstrates markedly improved empirical performance on multiple datasets and also performs the task of interaction classification which is not seen in recent works on evolving graphs, which are plus points. However, there are several concerns with the overall work that makes this paper weaker: (1) The main concern is with the novelty and more importantly the justification/analysis of the contributions proposed approach. (2) Further, while the ablation study provides some insights into architecture, it is not adequate (3) The paper misses comparison with a very important and recently proposed baseline, JODIE [1].
Main Comments:
--------------
- The paper leverages existing techniques built for learning over evolving graphs and augments it with three modifications: explicit use LSTM with time gate, stacked LSTM approach with fusion (Aggregation) and attention mechanism to select important neighbors to contribute to embedding update. The use of LSTM with time gate and fusion mechanism is very incremental contribution.  The attention mechanism proposed here is novel compared to existing works. However, there is very little justification or analysis provide or either of the contributions. This is big drawback of this contribution.

- For instance, the authors mention that stacked LSTM is used to capture multiple resolution. Can they provide some analysis or empirical demonstration that this actually happens? Also, the authors mention they use K in range of {1,2,3,4} but do not provide details what was useful for each dataset and how is it useful. How does the scaling parameter and alpha affect the performance and what are their roles? Also, what does superscript 'task' signify?
Similarly, they propose coattention mechanism with adaptive gate functions but does not provide any analysis of why they are useful and what characteristics they capture in the data that allows it to select most relevant neighbors. Is the attention mechanism temporally dependent?
- The authors perform ablation studies by switching off each component as a whole but considering the way this architecture is built, this is not a very useful exercise except knowing that each component contributes to the performance. A more detailed analysis and ablation is required. For instance, can the authors show performance with different K and  how it deteriorates/improves with it? Also, for stacked LSTM case, the authors show what happens when you use last layer, but what happens if the authors use only one layer (I guess this is K=1?) or don't use residual connections? When the time gate is switched off, does the authors also remove deltas from intensity function? what happens in this scenario? How does subgraph depth affect the quality of performance? What happens if authors don't sue adaptive gate functions?
- Figure 2 shows an example of bipartite graph, however, it seems datasets in experiments does have non-bipartite case? Is this true or the method only works for bipartite case?
- The use of proposed Algorithm 2 is not well justified. Why does the author need coloring and hashing mechanism instead of simpler BFS/randomwalk routine to collect previous interactions? Also, is this subgraph created for each event or it is computed offline during training? Further, the subgraph used for selection mechanism same as subgraph used for backtracking in LSTM?
- Further, is it true that the training is done in order of ColorGraphSeq or is it done in order of dataset? How does the authors capture dependencies across dataset in later case?
- Do you also update cell states with selection mechanism? The DIP-UNIT equation in selection section does not show that update. Also, are the embeddings updated only during train or also during validation/evaluation?
- The authors only present the results as-is without any insights on the performance of DIP model vs others and why they are able to demonstrate good performance. It is highly desired that authors add discussion section for each set of results to provide such information
- The authors include support for new nodes for interaction classification task but remove them for interaction prediction task which is strange. Is there a specific reason for this? What is the effect on the performance if new nodes are allowed in test? Further, why is interaction classification not compared with temporal baselines? All baselines produce embeddings and the authors mention that classification for this paper is independent of marker history. While the temporal baselines do not train for the task, the authors can train a second stage classifier with learned embeddings to perform classification
- The authors do not compare with recently proposed JODIE [1] which is a big miss. The comparison is required as it also models interaction processes in a  novel way by actually predicting the next embedding directly instead of modeling the intensity. An empirical comparison and discussion of this method is required to compare with various state-of-art methods.
Minor:
-------
- The authors need to use better and consistent notations. Also, as the overall approach uses similar flow as previous papers such as DeepCoevolve, it is recommended that the authors make the presentation simpler to position it clearly with existing works. On page 3, section 3.2 both bold-face and normal letters are used as vectors. Is $\hat{x}_{u(t)}$ a vector?
- Please provide numbers to equations for better referencing
[1] Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks, Kumar et. al. KDD 2019


Graph neural networks have recently achieved great successes in predicting quantum mechanical properties of molecules. These models represent a molecule as a graph using only the distance between atoms (nodes). They do not, however, consider the spatial direction from one atom to another, despite directional information playing a central role in empirical potentials for molecules, e.g. in angular potentials. To alleviate this limitation we propose directional message passing, in which we embed the messages passed between atoms instead of the atoms themselves. Each message is associated with a direction in coordinate space. These directional message embeddings are rotationally equivariant since the associated directions rotate with the molecule. We propose a message passing scheme analogous to belief propagation, which uses the directional information by transforming messages based on the angle between them. Additionally, we use spherical Bessel functions and spherical harmonics to construct theoretically well-founded, orthogonal representations that achieve better performance than the currently prevalent Gaussian radial basis representations while using fewer than 1 /4 of the parameters. We leverage these innovations to construct the directional message passing neural network (DimeNet). DimeNet outperforms previous GNNs on average by 76 % on MD17 and by 31 % on QM9.

This is a sophisticated paper on predicting molecular properties at atom as well as molecular levels using physics inspired, extended GNN architectures. Two key extensions are provided above and beyond previous GNN models that operated on graphs derived from pairwise distances between atoms. First, the encoding of atom distances for use in neural messages is no longer done in terms of Gaussian radial basis function representations but in terms of spherical Bessel functions. The provide an orthogonal decomposition at resolutions controlled by associated frequencies. The orthogonality is though lost due to the use of an envelop function to ensure differentiability at cutoff distance defining the graph (essential for molecular simulations) but this appears not to affect performance. The second and the primary contribution of the paper, beyond the architecture itself, is the use of directional embeddings of messages where angles are transformed into cosine basis for neural mappings. In other words, the message sent from atom i to j aggregates messages from i's other neighbors in a manner that takes into account the angle formed by i, j, and k's respective atom positions. Since local directions are equivariant with an overall molecular rotation, the message passing architecture in this new representation remains invariant to rotations/translations. The added directional information, embedded in local basis representation, clearly makes the network more powerful (able to distinguish higher order structures).
- the authors suggest that the radial information can be transformed simply by element-wise multiplication while angular information requires more complex transformations in message calculations. Is there a physical insight to this or is this simply an empirical finding?
- there are many layers of transformations introduced from the atom embeddings before reaching the property of interest. Are so many layers really necessary?
- it seems models for QM9 data were trained separately for each different physical target. Is this really necessary? Given the many layers of transformations until the properties are predicted, couldn't the message passing component be largely shared?
- what exactly is the training data for the molecular simulation tests? The description in the paper is insufficient. A separate model is trained for each molecule, presumably based on samples resulting from physical simulations (?). What is provided to the models based on each "sample"?
- the ablation studies are helpful to assess the impact of the main differences (directionality, bessel vs Gaussian, node embeddings vs message) though I would wish to see what the degradation effect is on QM9 if one used a shared message passing architecture (just sharing the messages, resulting embeddings could be transformed separately for different predictions).
There's a recent workshop paper also making use of directional information (local coordinate transformations along a protein backbone chain) in message passing/transformer architectures: Ingraham et al., Generative models for graph-based protein design, ICLR workshop 2019


This paper presents a generic framework to remedy the misalignment in unsupervised domain adaptation (UDA). Previous adversarial learning methods for UDA condition domain alignment only on pseudo labels, but pseudo labels may be inaccurate, hence bringing insufficient alleviation to the misalignment. Compared with pseudo labels, class prototypes are more accurate and reliable since they are summarized over all the instances and are able to represent the inherent semantic structures shared across domains. Therefore, we propose a novel Prototype-Assisted Adversarial Learning (PAAL) scheme, which incorporates instance probabilistic predictions and class prototypes together to provide reliable indicators for adversarial domain adaptation. With the PAAL scheme, we align both the instance feature representations and class prototypes to alleviate the misalignment among semantically different instances. Also, we exploit the class prototypes as proxy to minimize the intra-class variance in the target domain to mitigate the misalignment among semantically similar instances. With these novelties, we constitute a Prototype-Assisted Conditional Domain Adaptation (PACDA) framework which achieves state-of-the-art results on two UDA tasks, i.e., object recognition (OfficeHome, ImageCLEF-DA, and Office) and synthetic-to-real semantic segmentation (GTA5→Cityscapes and Synthia→Cityscapes).

Summary:
- key problem: address "class mismatch" in adversarial learning methods for unsupervised domain adaptation (UDA);
- contributions: 1) extension of the domain adversarial learning objective to leverage class prototypes (exponential moving average of features weighted by predicted class probabilities) in addition to pseudo-labels and intermediate representations (cf. eqs.5-11), 2) state-of-the-art results on several UDA tasks (Office-Home, ImageCLEF-DA, sim2real on Cityscapes).
Recommendation: weak accept (with some reservations below).
Key reason: interesting and effective use of prototypes for UDA.
- The formulation of the prototypes and additional learning objectives for UDA are clear and seem novel, although I would like to see a discussion of additional related works:
-- "Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results", Tarvainen and Valpola, NeurIPS'17;
-- "Unsupervised Domain Adaptation with Similarity Learning", Pinheiro, CVPR'18;
-- "Transferable Prototypical Networks for Unsupervised Domain Adaptation", Pan et al, CVPR'19.
- The effectiveness of the contributions is validated on multiple UDA tasks, and the ablative analysis supports the claims (that prototype-level alignment and within-class compactness helps).
Main reservation: the specific problem is not clearly formalized.
- What is the often mentioned but not clearly described "class mismatch" problem in UDA? To the best of my knowledge, this not a standard problem (could not find any mention in the previous literature, no citations or definitions in the submission). Is it that the target label space is different than the source label space (e.g., different ontologies)? In this case, what is the information on the target label space that enables unsupervised adaptation from the source one? What is the inductive bias / prior / assumptions?
- Alternatively, is the tackled problem only the noise in the pseudo-labels?
- In any case, the submission would greatly benefit from a clearer mathematical formalism and experimental characterization of the specific problem tackled here, especially in light of claims like "conditioning the alignment on pseudo labels can not well address the mismatch problem. Compared with the pseudo labels, the class prototypes are more robust and reliable in terms of representing the distribution of different semantic classes."
Additional Feedback:
- missing references on sim2real UDA: "DADA: Depth-aware Domain Adaptation in Semantic Segmentation" (Vu et al, ICCV'19), "SPIGAN: Privileged Adversarial Learning from Simulation" (Lee et al, ICLR'19)
## Post rebuttal update
I would like to thank the authors for replying to our questions. The clarifications with respect to related works and missing references is helpful, although a bit high-level (i.e. not necessarily describing the relative advantages of the proposed method). Nonetheless, the expected benefits of prototypes is still not entirely clear enough here, for instance regarding the main statistical assumptions that the method needs to make to get robust prototypes (e.g., in the presence of outliers or specific forms of "inaccuracies" in the pseudo-labels or "domain misalignment"). Therefore, due to the overall lack of mathematical clarity in the text and rebuttal, my main reservation remains, and I will change my "weak accept / borderline" score to weak reject. I encourage the authors to formalize the problem in a clearer, non-ambiguous way, discussing more explicitly the limitations of the proposed method.


Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. This is a critical shortcoming for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being deployed to interact with the environment -e.g. systems that learn from human interaction. Thus, we develop a novel class of off-policy batch RL algorithms which use KLcontrol to penalize divergence from a pre-trained prior model of probable actions. This KL-constraint reduces extrapolation error, enabling effective offline learning, without exploration, from a fixed batch of data. We also use dropout-based uncertainty estimates to lower bound the target Q-values as a more efficient alternative to Double Q-Learning. This Way Off-Policy (WOP) algorithm is tested on both traditional RL tasks from OpenAI Gym, and on the problem of open-domain dialog generation; a challenging reinforcement learning problem with a 20,000 dimensional action space. WOP allows for the extraction of multiple different reward functions post-hoc from collected human interaction data, and can learn effectively from all of these. We test real-world generalization by deploying dialog models live to converse with humans in an open-domain setting, and demonstrate that WOP achieves significant improvements over state-of-the-art prior methods in batch deep RL.

This paper presents the Way Off-Policy (WOP) algorithm which penalize divergence fro a strong pre-trained prior model as a way to avoid extrapolation error and instability in batch reinforcement learning (BRL). KL-divergence penalties have previously been explored extensively in the on-policy setting (mainly with policy gradient methods) and the authors claim they are the first to apply this idea to off-policy RL. I am not familiar enough with related work to evaluate this statement, but given it’s true, this is an interesting application of these ideas. I also like the application to dialog because pre-trained language models have been shown to offer very good priors for high-quality natural language generation.
Unfortunately, before any other comments I would like to point out that this paper is not properly anonymized. Both the submitted code and the online demo that is linked to from the paper contain the author names and their affiliation (the page linked to in Section 1 contains the author names but the other one in Section 5 does not—it mentions they have been redacted). Apparently there is also an arXiv version of the paper which I was not aware of and which would be fine if this version was not cited from the README file in the submitted code. Please be more careful with respect to anonymizing your submission!
Overall the paper proposes an interesting algorithm that combines existing ideas in a new way and arrives at an algorithm that achieves good performance, based on the experimental evaluation provided in the paper. The experiments performed on dialog are good and I really like that human evaluation was used. However, the traditional RL experiments are lacking. The authors only present results for the CartPole environment in the main paper, which is a very easy problem and so not that interesting. In the appendix they also show results for the Acrobot environment, which are a bit weaker for WOP. Both are quite simple problems and it’s quite easy to learn a good prior model. Why not present results for harder problems? It should be easy to run on more Gym environments and see if WOP still does well in harder problems. Other than that, I like the main ideas in the paper, but I think they could be presented better. Given this and the anonymization issue I lean towards rejecting this paper for now.
Other comments:
(1) I don’t necessarily agree with the statement “Because learning language online from humans on the internet can result in inappropriate behavior (see Horton (2016)), learning offline using BRL is imperative.” I would rather just say that learning offline is one way of dealing with this issue but it’s not the only one or necessarily the best one. In general, the paper defends the batch RL setting quite extensively, but I feel some of the statements may be overly strong. There is a lot of prior work that shows when and why the batch RL setting is useful and so I would add more references to such work.
(2) In Section A.1 you present the implicit metrics that were used and the weight of each metric. These weights seem *extremely* arbitrary to me so could you please provide some insight (and references) as to how they were chosen? I like the results presented in Table 3, in terms of the insight they provide. Are these what you used to derive the weights presented in Section A.1?
(3) The yellow shading in Figure 1 is not really visible. You could try using a different color for the shading (e.g., use gray and make the walls blue or something like that).
(4) The axis labels in the plots of Figure 2 are too tiny. These plots should be made bigger to be readable in printed form. Also, I would recommend you order the labels in the legend in maybe chronological order or something like, which would also put WOP at the top to clarify it’s the proposed method.
Disclaimer: I am not too familiar with this area of research and the related work and so my comments and evaluation should be taken into account in that context.


Most computer vision tasks use textured images. In this paper we consider the geo-localization task -finding the pose of a camera in a large 3D scene from a single lean image, i.e. an image with no texture. We aim to experimentally explore whether texture and correlation between nearby images are necessary in a CNNbased solution for this task. Our results may give insight to the role of geometry (as opposed to textures) in a CNN-based geo-localization solution. Lean images are projections of a simple 3D model of a city. They contain solely information that relates to the geometry of the scene viewed (edges, faces, or relative depth). We find that the network is capable of estimating the camera pose from lean images for a relatively large number of locations (order of hundreds of thousands of images). The main contributions of this paper are: (i) demonstrating the power of CNNs for recovering camera pose using lean images; and (ii) providing insight into the role of geometry in the CNN learning process;

The paper tackles the problem of geolocating an image from a 3D model of a city. This is an interesting and challenging problem since it involves solving the matching under 3D transformations at large scale.
The proposed approach is to use geometric features such as edges, faces and depths of the image, which is called "lean image" by the paper. The paper demonstrate that by using the "lean image" a neural network can recognize its geolocation with substantial accuracy. However, the idea of using geometric features has been investigated in quite a few previous literature and the paper fails to discuss relevant works, e.g., [1,2,3] and distinguish their differences.
[1] Li et al. Planar Structure Matching Under Projective Uncertainty for Geolocation. ECCV 2014.
[2] Mousavian and Kosecka. Semantic Image Based Geolocation Given a Map. ArXiv 2016.
[3] Kim et al. Learned Contextual Feature Reweighting for Image Geo-Localization. CVPR 2018.
The paper contains mostly empirical evaluations however the provided experiments do not well support the claim that CNN works well with geometric structures in geolocalization. The paper should compare with CNN on RGB images on the same data in order to demonstrate its effectiveness.
Review:
## Contribution of the paper:
1. The paper argues that "lean images" (geometric structure of a 2D scene) contains sufficient information for solving the memorization of geolocation tasks.
2. The paper provide an approach to evaluate the argument.
3. The paper provide experimental validation that the proposed method works fairly on a certain Berlin dataset.
## Feedback
The paper is interesting but it does not meet the standard of this conference. There are some key reasons:
1. The writing is poor. The paper is hard to follow. There are also missing references. The importance of geometry in geolocation has been studied in many previous works.
2. The paper is empirical so it has to provide strong empirical evidence to support the claim. The experiment is only conducted in Berlin data whose scope is fairly small. Without large scale experiments, it is hard to convince readers that CNN is able to "memorize" all the locations using just geometric features. Without comparing to RGB baseline or other existing approaches, it is hard to convince readers that the proposed approach works fairly well. It also seems the input images are just 2D projections so without experiments using real 2D images, it is hard to say the approach is effective.
## Improvements that could be done
1. Please make sure everything in the tables are clearly explained. The metrics should be explained somewhere in the caption. It is hard to find the meaning of some notations in the table such as "37K images", "Area 400x400". Please clearly state the unit.
2. Please include some baseline results into the tables so that readers know how much improvments achieved by your approach.
3. Please reorganize the paper to make it more clear.
4. Please provide more literature review and discuss the difference of your work to existing ones.
## Questions
1. How large is the Berlin data?
2. How would your method work when there is only a street image? How would you method scale when 3D model is not always available.
3. How would you method work when there is some places without much structure? Rural area could be an example.


In this work we propose a novel end-to-end imitation learning approach which combines natural language, vision, and motion information to produce an abstract representation of a task, which is in turn used to synthesize specific motion controllers at run-time. This multimodal approach enables generalization to a wide variety of environmental conditions and allows an end-user to direct a robot policy through verbal communication. We empirically validate our approach with an extensive set of simulations and show that it achieves a high task success rate over a variety of conditions while remaining amenable to probabilistic interpretability.

The paper addresses the problem of using multiple modalities for learning from demonstration. Approaches that take in task or joint space data to learn a policy for replicating that task are numerous. Doing the same with multiple modalities involved, in particular vision, language and motion, has only been recently considered, so this is a timely paper.
The core contribution is pretty well summarised by the architecture in figure 1, which involves a combination of encodings of the words and sentences, images and parameters of a DMP in order to generate movement commands from a high level instruction.
Unless I have missed something in the experimental setup, all of the considered task variations are movement commands of the form <Move> to <Object>. The network setup allows for synonyms of two kinds, so <Move> can be replaced by numerous verbal synonyms such as advance and go, and the object can be specified in terms of shapes, colors and so on, but otherwise this is the only specification of the task. This has been addressed in the recent literature using neural network architectures similar to the one being proposed here, e.g., see the following papers. These papers already solve the proposed problem and provide similar explanations. It would be helpful to see comparative discussion with respect to those methods and a clear statement of novelty with respect to such prior work:
[R1] M. Burke, S. Penkov, S. Ramamoorthy, From explanation to synthesis: Compositional program induction for learning from demonstration, Robotics: Science and Systems (R:SS), 2019.
[R2] Y. Hristov, D. Angelov, A.Lascarides, M. Burke, S. Ramamoorthy, Disentangled Relational Representations for Explaining and Learning from Demonstration, Conference on Robot Learning (CoRL), 2019.
An interesting feature in R2 that the authors do not explicitly address here is the issue of relational specifications in the language, e.g., in addition to saying "move to the red bowl", we may also wish to say "place on top of red block". In the way that MPN is currently set up to map from the language input directly to hyperparameters of the DMP, and considering the embedding structure, it is not clear if MPN is capable of handling such specifications. If so, the claim of generalisation on the language input should be stated more clearly.
The ablation study is setup somewhat differently than what I would have expected. The authors consider the effect of changing the training set size and if the language input includes synonyms or not. Those two aspects seem to produce the expected results. It would also be interesting to see an ablation study in the sense of replacing or removing aspects of the architecture to see its relative effect on the overall model performance. So, for instance, if one did not have a DMP with the hyperparameters being estimated by a network and instead had a more straightforward encoding of where to move to - does it make a difference and how much? Likewise, how much performance benefit, if any, is being derived from an uninterpreted image I being combined as described in the embedding as opposed to an alternative that detects an objects and combines that position differently. The paper would have been stronger if such architectural choices were better justified and also demonstrated in the experiments.


We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of the observer. The resulting predictive F-information encompasses mutual information and other notions of informativeness such as the coefficient of determination. Unlike Shannon's mutual information and in violation of the data processing inequality, F-information can be created through computation. This is consistent with deep neural networks extracting hierarchies of progressively more informative features in representation learning. Additionally, we show that by incorporating computational constraints, F-information can be reliably estimated from data even in high dimensions with PAC-style guarantees. Empirically, we demonstrate predictive F-information is more effective than mutual information for structure learning and fair representation learning.

Summary
The paper introduces a framework for quantifying information about one random variable, given another random variable (“side information”) and, importantly, a function class of allowed transformations that can be applied to the latter. This matches the typical scenario in machine learning, where observations (playing the role of side information) can be transformed (with a restricted class of transformation-functions) such that they become maximally predictive about another random variable of interest (“labels”). Using this framework, the paper defines the notion of conditional F-entropy and F-entropy (by conditioning on an empty set). Interestingly, both entropic quantities are shown to have many desirable properties known from Shannon entropy - and when allowing the function class of transformations to include all possible models F-entropies are equivalent to Shannon entropies. The paper then further defines “predictive F-information” which quantifies the increase in predictability about one random variable when given side information, under a restricted function-class of allowed transformations of the side information. Importantly, transformations of side information can increase predictive F-information (which is the basis for the notion of “usable” information), which is in contrast to the data processing inequality that applies to Shannon information and states that no transformation of a variable can increase predictability of another variable further than the un-transformed variable (information cannot be generated by transforming random variables). The paper highlights interesting properties of the F-quantities, most notably a PAC bound on F-information estimation from data, which gives reason to expect F-information estimation to be more data-efficient than estimating Shannon-information (particularly in the high-dimensional regime). This finding is confirmed by four types of interesting experiments, some of which make use of a modified version of a tree-structure learning algorithm proposed in the paper (using predictive F-information instead of Shannon mutual information).
Contributions
i) Proposal of a framework for measuring and reasoning about information that transformed random variables have about other random variables, when the class of transformation functions is restricted. Interesting properties are highlighted and corresponding proofs are given. Important conclusions to Shannon-information measures are drawn.
ii) PAC guarantees for estimating F-information quantities from data. A nice result that justifies some optimism about the scalability of F-information estimation.
iii) Modification of a tree-structure learning algorithm, and application to four types of experiments with comparisons against methods for estimating Shannon(-mutual)-information.
Quality, Clarity, Novelty, Impact
The paper is very well written, the motivation and main results are clear and connections to known measures for information in complex systems are drawn (which often appear as corner-cases, or unrestricted cases of F-information). I am not an expert on various information measures, thus I cannot fully judge the novelty of the framework (given that the central idea is fairly simple and quite elegant, the main work lies in the proofs and connections to other frameworks). However, I have not seen the framework being discussed in the machine learning literature before. I personally would rate the potential impact of the F-information framework as high because it addresses many problems that Shannon-(mutual-)information has (hard to estimate, generality means complete blindness against model-classes). The experiments in the paper already illustrate how F-information could be very useful for a range of ML problems that cannot be tackled by strong competitor methods based on Shannon-information estimation. My only criticism is that the paper does not clearly state current limitations and shortcomings and does not comment on the difficulties / potential problems with solving the variational problem that is part of the definition of (conditional) F-information. I currently vote and argue for accepting the paper, though my assessment is of medium confidence only, and I am happy to take issues raised by the other reviewers and the rebuttal into account. I have not checked the proofs in the appendix in great detail.
Improvements
i) Please add a short section of current shortcomings and caveats, especially with regard to applying the methods in practice.
ii) Please comment on solving the variational optimization problem (the infimum) which is part of the definition of (conditional) F-information. In particular, are there any theoretical statements / bounds / etc. to be made for the case where the infimum is not found exactly - does the measure degrade gracefully or can small errors in this optimization lead to wildly varying/divergent F-information? From a practical point-of-view: how was this optimization done in the experiments (particularly when involving a neural network model), how much computational overhead did this optimization add (and how does it compare against other methods, e.g. in terms of wall-clock time or other reasonable metrics, the more the better)?
iii) This is a minor one and feel free to completely ignore it. The name F-information might easily get confused with the use of f-divergences, perhaps there is a better, more informative name. Also, while I personally like the term “usable” in the title, I’m not so sure about “computational constraints” - the latter somehow suggests that the method has small computational footprint, or can easily scale to different computational budget. Perhaps there is a way that more strongly indicates that this refers to restrictions on the model-/function-class (which the term “usable” does already to some degree admittedly).
Minor Comments
a) Have you had any thoughts on how F-information could be used in a rate-distortion / information-bottleneck type framework for a theory of “relevant usable information”? This is probably beyond the scope of this paper, just out of curiosity.
b) The paragraph above 3.3 almost sounds a bit like Shannon (and the data processing inequality) was wrong. I’d rather phrase this as a “no-free-lunch problem” - while the DPI and Shannon (mutual) information is very elegant, it is necessary to make further assumptions/restrictions (the function class of allowed transformations) to make more fine-grained statements and define more precise (but less general) informational-quantities tailored to the specific function class.
c) When choosing function classes that allow for universal function approximation, would F-information degrade to Shannon information?


Integrating distributed representations with symbolic operations is essential for reading comprehension requiring complex reasoning, such as counting, sorting and arithmetics, but most existing approaches rely on specialized neural modules and are hard to adapt to multiple domains or multi-step reasoning. In this work, we propose the Neural Symbolic Reader (NeRd), which includes a reader, e.g., BERT, to encode the passage and question, and a programmer, e.g., LSTM, to generate a program for multi-step reasoning. By using operators like span selection, the program can be executed over text to generate the answer. Compared to previous works, NeRd is more scalable in two aspects: (1) domain-agnostic, i.e., the same neural architecture works for different domains; (2) compositional, i.e., complex programs can be generated by compositionally applying the symbolic operators. Furthermore, to overcome the challenge of training NeRd with weak supervision, we apply data augmentation techniques and hard ExpectationMaximization (EM) with thresholding. On DROP, a challenging reading comprehension dataset requiring discrete reasoning, NeRd achieves 1.37%/1.18% absolute gain over the state-of-the-art on Exact-Match/F1 metrics. With the same architecture, NeRd significantly outperforms the baselines on MathQA, a math problem benchmark that requires multiple steps of reasoning, by 25.5% absolute gain on accuracy when trained on all the annotated programs, and more importantly, still beats the baselines even with only 20% of the program annotations.

This paper presents a semantic parser that operates over passages of text instead of a structured data source.  This is the first time anyone has demonstrated such a semantic parser (Siva Reddy and several others have essentially used unstructured text as an information source for a semantic parser, similar to OpenIE methods, but this is qualitatively different).  The key insight is to let the semantic parser point to locations in the text that can be used in further symbolic operations.  This is excellent work, and it should definitely be accepted.  I have a ton of questions about this method, but they are good questions.  The rest of this review focuses on things that I thought could be more clear, or that raise new questions, and might sound negative.  Please understand them, however, in terms of my overall score and what I said above.
The three claimed contributions are (1) better numbers, (2) better compositionality / domain applicability, and (3) better interpretability.
(2) and (3) sound a bit like overclaiming in the introduction to me, as there isn't a whole lot of nested composition in the language used by NeRd, and the BERT calculator in principle is almost as compositional and interpretable (also, e.g., NAQANet can add and subtract an arbitrary number of numbers, also, and it tells you which ones they are, just as NeRd does).  Later in the paper the specifics of those claims are made more clear, and while they are justified, they are very narrow claims.  To me, someone who is intimately familiar with this research area, the key contributions (the things that I learned) are (1) using passage-span and key-value predicates actually works, (2) how much difference hard EM and thresholding make, and (3) the data augmentation in this work is pretty clever.  (2) was intuitively clear to me after seeing Dasigi's iterative search paper and Min's hard EM paper, but the difference in results presented here is pretty striking.
Compositionality:
The authors claim that their method is compositional and domain agnostic, while all previous methods had hand-crafted modules for specific question types.  However, I see little reason to believe there's much of a difference here.  You also defined operations that are tailored to the dataset, and are basically identical to the operations that others have used.  I see no evidence that NeRd actually generalizes to program types that are beyond what is captured by other methods.  It's possible that this happens, but there is no evaluation that discusses this, and from all of the examples I'm led to believe that this is basically also just learning a few program templates, the same ones learned by previous methods.  With the weak supervision that you have, are you actually able to find more complex programs during your search?  Some kind of demonstration of actual compositionality on the more complex questions in DROP would make a very strong argument for your claims; without that, they ring a little hollow.
Interpretability:
The use of passage-span as a predicate is really interesting, and it raises a lot of questions.  This predicate lets the model shortcut any interpretable reasoning and do operations entirely inside the encoder/parser.  For example, your first example in table 2 ostensibly requires filtering the numbers in the passage to those that are percentages associated with groups, then filtering them again to those where the percentage is larger than 16, then returning the associated groups. But your method jumps straight to returning a set of passage spans.  This is hardly interpretable.  (In fairness, no prior method provides interpretable reasoning for this kind of operation either.)  But the fact that you have this predicate lets the model do these filters and greater-than comparisons inside the network in an opaque way, while also getting interpretable operations for some questions (table 5 is further confirmation of this, and of the fact that you probably are not capturing many of more the complex, compositional questions in DROP).  But how does the network decide which to do?  Any argmax or max question, and many count questions, could be answered by passage-span alone.  With only weak supervision, and with the parser having the ability to shortcut these more interpretable operations, how often are you actually getting the interpretable one, and what's causing the model to choose it?
Similarly, how often does an argmax or a max operation actually operate on the full set that you would expect it to?  Or does it just do the argmax internal to the network, and output only one item as an argument to the argmax?  If the later, this again hurts your claims of better interpretability over prior methods, as the logic is just as opaque as before.  This also seems like a really hard search problem in how you've set up your DSL - what would make your search over programs actually select all of the correct arguments?  Because you're selecting passage spans directly instead of performing some kind of matching operation, you have to have your search select all of the appropriate spans for this to be "interpretable", and not just hiding the logic inside of the network.  But that seems like a totally intractable search.  You found a clever way to get around this for count questions (even though that still implicitly hides a bunch of filtering logic, as noted above), but I don't know how to make it work for maxes and argmaxes.
Another question raised by the passage-span predicate: the more you use bare passage-span programs for training, the more the network learns to put all of its compositional reasoning inside, in an opaque way, instead of giving you interpretable compositionality.  At one extreme, you end up with something like NABERT (or even less compositional), where basically everything is inside the network.  At the other extreme, where you don't have passage-span, you are left with a crippled semantic parser that can't handle most of the questions.  But using the predicate introduces tension in the model between interpretability and flexibility.  How do we resolve this tension?  (This isn't something I expect your paper to address, it's just a really interesting and important question raised by your work.)
Parser:
Prior work has found benefit in using runtime constraints on parser outputs, or grammar-based decoding.  It looks like you are doing neither of those, yet you're able to output specific token indices and number indices in your programs.  Are you really not doing anything special to handle those?  How does the decoder know token indices?  I feel like something must be missing here, or a simple LSTM decoder is more magical than I thought.
Evaluation:
Why only show results on DROP dev, and not on the test set?  It's possible that your higher numbers are because you were better able to overfit to the dev set, which you presumably used during training.  (I don't think that that's likely, but it's a concern that would be easily avoided by evaluating on test.)


Convolutional layer utilizes the shift-equivalent prior of images which makes it a great success for image processing. However, commonly used down sampling methods in convolutional neural networks (CNNs), such as max-pooling, averagepooling, and stride convolution, are not shift-equivalent. This destroys the shiftequivalent property of CNNs and degrades their performance. In this paper, we provide a formal definition of shift-equivalent when down sampling involved. We propose a strict shift equivalent and anti-aliasing pooling method. This is achieved by (inverse) Discrete Fourier Transform and we call our method frequency pooling. Experiments on image classifications show that frequency pooling improves accuracy and robustness w.r.t shifts of CNNs.

This paper researches the pooling operation, which is an important component in convolutional neural networks (CNN) for image classification. Taking the perspective from signal processing, this paper proposes a pooling operation called frequency pooling (F-pooling). The key motivation is to make the pooling operation shift-equivalent and anti-aliasing. This paper gives an improved definition on shift-equivalent functions and shows that the proposed F-pooling is optimal in the sense of reconstructing the orignal signal. The F-pooling is then implemented with matrix multiplications and tested with recent convolutional neural networks for image classifiation on CIFAR-100 and a subset of ImageNet dataset.
It is interesting to take the perspective from signal processing to give pooling operation in CNN a formal treatment. As indicated in the recent literature, enforcing shift-invariance does help to improve the performance of a CNN on classification accuracy and the robustness with respect to image shift. At the same time, this work can be further enhanced at the following aspects:
1. This work can make it clearer in principle how anti-aliasing contributes to improving the classification performance and robustness. This will help to make this paper more self-contained.
2. When showing the optimality of F-pooling in Section 2.3, the criterion is to reconstruct the original signal x. Considering that the ultimate goal is classification, the information to be maximally preserved through each operation through the layers shall be the information that relates to the class label y. In light of this, some justification and explanation shall be provided for using this criterion for optimality.
3. The experimental study is weak. Experiments could be conducted on more benchmark datasets with more CNN architectures to convincingly show the effectiveness of the proposed F-pooling. Also, from the three Tables in the experimental part, the improvement of F-pooling over AA-pooling (developed by the main reference of this work) does not seem to be significant or consistent. For example, in Table 2, the F-pooling only wins at either accuracy (marginally) or consistency, but not both. In Table 3, the F-pooling consistently shows inferior classification performance, although obtaining slightly higher consistency. This makes the advantage of F-pooling over the existing AA-pooling unclear.


Optimistic initialisation is an effective strategy for efficient exploration in reinforcement learning (RL). In the tabular case, all provably efficient model-free algorithms rely on it. However, model-free deep RL algorithms do not use optimistic initialisation despite taking inspiration from these provably efficient tabular algorithms. In particular, in scenarios with only positive rewards, Q-values are initialised at their lowest possible values due to commonly used network initialisation schemes, a pessimistic initialisation. Merely initialising the network to output optimistic Q-values is not enough, since we cannot ensure that they remain optimistic for novel state-action pairs, which is crucial for exploration. We propose a simple count-based augmentation to pessimistically initialised Q-values that separates the source of optimism from the neural network. We show that this scheme is provably efficient in the tabular setting and extend it to the deep RL setting. Our algorithm, Optimistic Pessimistically Initialised Q-Learning (OPIQ), augments the Q-value estimates of a DQN-based agent with count-derived bonuses to ensure optimism during both action selection and bootstrapping. We show that OPIQ outperforms non-optimistic DQN variants that utilise a pseudocount-based intrinsic motivation in hard exploration tasks, and that it predicts optimistic estimates for novel state-action pairs.

Optimistic Exploration even with a Pessimistic Initialisation
================================================================
This paper presents an exploration algorithm based on "optimism in the face of uncertainty" via count-based bonus.
The authors observe that typical neural net initializations close to zero can be pessimistic, but show that augmenting a count-based bonus for acting and bootstrapping can overcome this.
The authors support their claim with an adaptation of a regret bound for the tabular case, and a series of didactic experiments with neural net models.
There are several things to like about this paper:
- Exploration with generalization in Deep RL is a large outstanding problem, with few effective options and none really commonly used in the field beyond epsilon-greedy or Boltzmann.
- This algorithm is reasonably well thought out, building on an established literature of exploration bonuses, but with a slightly different take on the structure of the bonus.
- The paper is well structured, building from intuition to theory to toy examples in DQN setting.
- Overall the algorithm appears to perform well against a wide variety of related variants (although that presentation is a little confusing / overwhelming).
There are several places the paper might be improved:
- I don't think the authors make a clear enough case for why this method is *better* than the other optimistic bonus approaches listed... Yes there is a regret bound, but this is not as good as some other methods... Yes there are ablations... but they're not really clear about what the mechanism that makes this method better than others!
- Although this algorithm is motivated by applications to *deep* RL, the key choice of the "count" (and thus the method for optimism bonus) is mostly sidestepped. It amounts to an essentially tabular bonus in the space of the hashing function... and it's not clear why this approach should work any better or worse than other similar approaches that the paper complains about. For example, if you used "Randomized Prior Functions" or "Random Network Distillation" with that same hashing functions you would likely end up with similar results?
- The comparison to benchmark algorithms seems quite confusing and I'm not sure if it's really presented well. It might be good to focus on fewer comparisons at a time and push remaining less important ones to the appendix.
- It would be great to get an evaluation of these algorithms on a standardized and open-source benchmark... and I think that bsuite could be a really good candidate for this paper https://github.com/deepmind/bsuite particularly the "deep sea" experiments.
Overall I think this is a reasonable paper, and I expect it to improve during the review process.
At the moment I have to say that I don't think there is a clear enough case for why this method is preferable to other similar approaches, or enough insight into the pros and cons to accept.
============
Updating to "weak accept" as part of rebuttal.


The emergence of language in multi-agent settings is a promising research direction to ground natural language in simulated agents. If AI would be able to understand the meaning of language through its using it, it could also transfer it to other situations flexibly. That is seen as an important step towards achieving general AI. The scope of emergent communication is so far, however, still limited. It is necessary to enhance the learning possibilities for skills associated with communication to increase the emergable complexity. We took an example from human language acquisition and the importance of the empathic connection in this process. We propose an approach to introduce the notion of empathy to multi-agent deep reinforcement learning. We extend existing approaches on referential games with an auxiliary task for the speaker to predict the listener's mind change improving the learning time. Our experiments show the high potential of this architectural element by doubling the learning speed of the test setup.

This paper starts with a conceptual claim that incorporating a notion of “empathy” in language emergency would help agents learn faster.  The paper then proposes a learning mechanism for implementing this, and looks at its empirical effect for the case of a Speaker-Listener game.
The concept at the core of the paper is thought-provoking, somewhat grounded in human communication, and it’s interesting to see how this can be translated into a learning mechanism for the multi-agent setting.   The specific implementation proposed seems reasonable at a high level, however there are many technical details missing which really hamper the paper’s message & potential scientific impact.   The results are limited to a single game, with just a pairwise comparison (with and without “empathy”), and provide a narrow view into the effectiveness of the proposed technique.
My main problem with the paper is the clarity & organization problems.  Usually I tend to be lenient on this, thinking poor writing is much easier to fix than poor science.  But in this case the problems are large enough that the paper is just too far from the standard for ICLR publication.  It also fits in 5 pages, so the authors had lots of space to write a much better paper.   I encourage them to do this for a future submission, in addition to more extensive results, because I think the ideas are worthwhile.
Specific comments:
Design of the empathy mechanism. Can you motivate why it’s reasonable to “achieve a high relation between the hidden states of both agents”? Is this necessary / sufficient for empathy?  What are alternate framings of this?  What are properties and pros/cons of this framing?
Sec.4 needs a lot more detail!   Sections Agent setup, Learning and Empathy Extension in Sec.5 should be moved to Sec.4, since they describe the method, rather than the experiments.
Sec.5 needs better clarity.
o	What are m^l_t and M^{<l}_t in Eqn 5?  Define how h_S and h_L are parameterized, and how each is trained.
o	Fig.2 gives a high-level view of the approach, but lacks important details.  Do you apply a loss at both the Speaker’s Decoder output, and the Listener’s Decoder output?  Or just the latter?  What is the loss specifically? I assume combination of Eqn 5 and Eqn 6, but not sure.
o	If you train just on the loss of the Listener’s Decoder, does this mean this is backpropagated all the way to train the Speaker?  How would this be done in a real system?  It’s a very strong assumption to say that the Listener will share gradients with the Speaker.  It seems more realistic to assume they will each observe a loss and train independently.
Results are very brief.
o	How robust are the results to the specification of the \alpha (the loss weight from Eqn 6)?  How much data goes to finding a good \alpha?
o	What is the difference between the left and the right plot?  Is one for the Speaker and the other for the Listener?
o	How do you measure “learning speed”, which is the main metric discussed in the text of Sec.6?
o	How do the results change by number of concepts in the game?
o	Do you do any pre-training of the encoder/decoder networks?
o	Can you show confidence intervals on each curve?
o	Can you show test performance?
o	Are there other related games to consider?
Many references missing throughout to support statements, e.g.
o	“Natural language is not as rule-based as…”
o	“These considerations led to the research field of emergent communication…”
o	Sec.2:  Earlier refs to RL in general (e.g. work Sutton in the 1980’s). Earlier refs to RL with neural networks (e.g. work of G. Tesauro; work of M. Riedmiller).
o	Referential game in Fig.1 caption.
Some minor language issues, e.g.
o	“The field was then alleviated” -> Do you mean elevated?


We present a generative adversarial network (GAN) that conducts manifold learning and alignment (MLA): A task to learn the multi-manifold structure underlying data and to align those manifolds without any correspondence information. Our main idea is to exploit the powerful abstraction ability of encoder architecture. Specifically, we define multiple generators to model multiple manifolds, but in a particular way that their inverse maps can be commonly represented by a single smooth encoder. Then, the abstraction ability of the encoder enforces semantic similarities between the generators and gives a plausibly aligned embedding in the latent space. In experiments with MNIST, 3D-Chair, and UT-Zap50k datasets, we demonstrate the superiority of our model in learning the manifolds by FID scores and in aligning the manifolds by disentanglement scores. Furthermore, by virtue of the abstractive modeling, we show that our model can generate data from an untrained manifold, which is unique to our model.

The paper suggests performing simultaneous manifold learning and alignment by multiple generators that share common weight matrices and a constructed inverse map that is instantiated by a single encoder. The method utilizes a special regularizer to guide the training. It has been empirically well tested on a multi-manifold learning task, manifold alignment, feature disentanglement, and style transfer.
Overall, this is an interesting idea with a motivated approach, however, I would like several points to be addressed before I could increase a score.
1. It seems that the method makes direct use of the number of classes in the datasets used. How would it fare compared to other models when the number of manifolds is not known (e.g. CelebA dataset)?
2. In MADGAN (Ghosh et al., 2017) generators share first layers which possibly makes them not independent as claimed in the paper, thus it is worth checking if MADGAN exhibits any kind of manifold alignment and could be a baseline for disentanglement with multiple generators.
3. There are hyperparameters \lambda and \mu for the regularizers in the model. It would be helpful to study their effect of different values on the training and encoding.
4. Is there a reason for DMWGAN/InfoGAN scores being omitted in Table 1?
Minor remark - there are a number of typos in the text.


Binary Neural Networks (BNNs) have been garnering interest thanks to their compute cost reduction and memory savings. However, BNNs suffer from performance degradation mainly due to the gradient mismatch caused by binarizing activations. Previous works tried to address the gradient mismatch problem by reducing the discrepancy between activation functions used at forward pass and its differentiable approximation used at backward pass, which is an indirect measure. In this work, we use the gradient of smoothed loss function to better estimate the gradient mismatch in quantized neural network. Analysis using the gradient mismatch estimator indicates that using higher precision for activation is more effective than modifying the differentiable approximation of activation function. Based on the observation, we propose a new training scheme for binary activation networks called BinaryDuo in which two binary activations are coupled into a ternary activation during training. Experimental results show that BinaryDuo outperforms state-of-the-art BNNs on various benchmarks with the same amount of parameters and computing cost.

This paper proposes a new measure of gradient mismatch for training binary networks, and additionally proposes a method for getting better performance out of binary networks by initializing them to behave like a ternary network.
I found the new measure of gradient deviation fairly underdeveloped, and I suspect the method of converting ternary activations into binary activations works for a different reason than that proposed by the authors.
There were English language issues that somewhat reduced clarity, though the intended meaning was always understandable.
Detailed comments:
"Binary Neural Network (BNN) has been gaining interest thanks to its computing cost reduction and memory saving." --> "Binary Neural Networks (BNNs) have been garnering interest thanks to their compute cost reduction and memory savings." (will stop making English language corrections from here on)
"Therefore, we argue that the sharp accuracy drop for the binary activation stems from the inefficient training method, not the capacity of the model."
This could also be due to poor initialization in the binary case. e.g., it might make sense to initialize the binary network with bias=-0.5, so that the nonlinearity has a kink at pre-activation=0, rather than pre-activation=0.5.
"Unfortunately, it is not possible to measure the amount of gradient mismatch directly because the
true gradient of a quantized activation function is zero almost everywhere. " It *is* possible to measure the mismatch to the true gradient exactly. One could even train using the true gradient. It's just that the true gradient is useless.
Fig 1b -- this is a nice baseline.
"the steepest descent direction, which is the direction toward the point with the smallest loss at given distance"
This is not the usual definition of steepest descent direction. If you're going to redefine this, should do so mathematically and precisely (for instance, you are going to run into trouble with the word "distance", since your coordinate discrete gradient more closely resembles an L\infty-ball perturbation, rather than an L2-ball perturbation.
eq. 3:
Note that this equation is equivalent to taking the true gradient of a function which has been boxcar-smoothed along each parameter. This may more closely resemble existing measures of deviation than you like.
You should also consider the relationship to an evolutionary strategies style gradient estimate, which similarly provides an unbiased gradient estimate for a smoothed function, and which allows that estimate to be computed with fewer samples (at the cost of higher error).
Sec. 4.2 / Figure 3:
The results in this section will be *highly* sensitive to the choice of epsilon. You should discuss this, specify the epsilon used, and experimentally explore the dependence on epsilon.
"The results indicate that the cosine similarity between coarse gradient and CDG can explain the relationship between gradient mismatch and performance of model better than previous approaches. "
Don't know that I followed this. Gradient mismatch is never formally defined, so it's hard to know what this says about its relationship. Additionally, CDG sounds more like something which is correlated with, rather than an explanation for, performance.
" cosine similarity between coarse gradient and CDG can explain the relationship between gradient mismatch and performance of model better " --> " cosine similarity between coarse gradient and CDG can explain the relationship between gradient mismatch and performance of model better "
"we shift the bias of BN layer which comes right before the activation function layer. "
Did you try using these bias values without pre-training as a ternary network? I suspect it would work just as well!
"Please note that BN layers followed by binary activation layer can be merged to the threshold of the binary activation layer, incurring no overhead at inference stage."
Did not understand this.
"it is expected that the fine-tuning increases the accuracy even further"
Does it improve the accuracy further? Should state this as result, not prediction, and should have an ablation experiment showing this.
"Table 2 shows the validation accuracy of BNN in various schemes."
Why not test accuracy?
Figure 6:
What are the filled circles?
What was the sampling grid for the HP search? The images have high spatial frequency structure that I suspect is an artifact of the interpolation function, rather than in the data.
----
Update post-rebuttal:
The authors have addressed the majority of my concerns, through both text changes and significant additional experiments. I am therefore increasing my score. Thank you for your hard work!


Overparameterized transformer networks have obtained state of the art results in various natural language processing tasks, such as machine translation, language modeling, and question answering. These models contain hundreds of millions of parameters, necessitating a large amount of computation and making them prone to overfitting. In this work, we explore LayerDrop, a form of structured dropout, which has a regularization effect during training and allows for efficient pruning at inference time. In particular, we show that it is possible to select sub-networks of any depth from one large network without having to finetune them and with limited impact on performance. We demonstrate the effectiveness of our approach by improving the state of the art on machine translation, language modeling, summarization, question answering, and language understanding benchmarks. Moreover, we show that our approach leads to small BERT-like models of higher quality compared to training from scratch or using distillation.

This paper presents LayerDrop, a simple method for dropping groups of weights (typically layers) jointly. Despite its simplicity (which is actually a big plus), the method seems to improve performance quite consistently on a range of NLP tasks. Moreover, it allows the authors to train very deep networks, that are very hard to train otherwise (according to the authors). For me the most exciting thing about this approach is that this training regime allows to prune the trained network at test time *without finetuning*, effectively getting a smaller, more efficient network for free. This is a great benefit compared to existing approaches that require retraining a smaller network for each costume size. While the method isn't really applicable to any size, and largely depends on the dropout rate the full model was trained on, I imagine it could serve as a starting point for other researchers to develop more flexible extensions that would allow for any size of network to be pruned at test time. I think this is a very strong submission and strongly advocate accepting it to ICLR.
Questions and comments:
1. The main thing missing for me is some more analysis on the runtime/energetic savings (e.g., in terms of FLOPs) of the proposed method. The authors argue (3.2.1) that approaches such as DropConnect are not necessarily more efficient, but do not analyze the efficiency of their pruned networks apart from the size of the pruned network.
2. Similarly, details about the experiments are also somewhat lacking:
a. how many GPUs were used to train the models? the authors mention 8 v100 in A.3, but I am not sure if this was the setup for all experiments.
b. Figure 7, which shows that LayerDrop also improves training speed, is very interesting and should be part of the main text in my opinion. Was this trend consistent for all experiments?
c. Similarly, presenting the total running time of the models (and not just words per second) would be helpful for reproducibility.
d. Finally, reporting dev and not only test results (e.g., in tables 1 and 2) would also facilitate future reproducibility efforts.
3. Did the authors use a regular dropout? If I understand correctly, in A.1.3, the authors mention tuning the dropout rate between {0.2,0.3}. Was this done for all tasks? and was it done for the baseline models as well? Using dropout in the baseline model with a similar proportion as LayerDrop seems like an important baseline, and in particular it would be interesting to see whether the deep experiments (e.g., 40 layers on WT103) that are hard to train without LayerDrop could converge with regular dropout.
Minor:
- 3.2: "We present *an* regularization approach ..." (should be "a")
- Table 2 is referred to before table 1, it might be clearer to switch them.
- In figure 4, it wasn't clear to me why "Layer" on the lefthand side is much better than "Every other" on the righthand side. Aren't these the same model variant?
- Missing year for paper "Language models are unsupervised multitask learners".