The incorporation of prior knowledge into learning is essential in achieving good performance based on small noisy samples. Such knowledge is often incorporated through the availability of related data arising from domains and tasks similar to the one of current interest. Ideally one would like to allow both the data for the current task and for previous related tasks to self-organize the learning system in such a way that commonalities and differences between the tasks are learned in a datadriven fashion. We develop a framework for learning multiple tasks simultaneously, based on sharing features that are common to all tasks, achieved through the use of a modular deep feedforward neural network consisting of shared branches, dealing with the common features of all tasks, and private branches, learning the specific unique aspects of each task. Once an appropriate weight sharing architecture has been established, learning takes place through standard algorithms for feedforward networks, e.g., stochastic gradient descent and its variations. The method deals with meta-learning (such as domain adaptation, transfer and multi-task learning) in a unified fashion, and can easily deal with data arising from different types of sources. Numerical experiments demonstrate the effectiveness of learning in domain adaptation and transfer learning setups, and provide evidence for the flexible and task-oriented representations arising in the network.

The work proposed a generic framework for end-to-end transfer learning / domain adaptation with deep neural networks. The idea is to learn a joint autoencoders, containing private branch with task/domain-specific weights, as well as common branch consisting of shared weights used across tasks/domains, as well as task/domain-specific weights.  Supervised losses are added after the encoders to utilize labeled samples from different tasks. Experiments on the MNIST and CIFAR datasets showed improvements over baseline models. Its performance is comparable to / worse than several existing deep domain adaptation works on the MNIST, USPS and SVHN digit datasets.
The structure of the paper is good, and easy to read.  The idea is fairly straight-forward. It reads as an extension of "frustratingly easy domain adaptation" to DNN (please cite this work). Different from most existing work on DNN for multi-task/transfer learning, which focuses on weight sharing in bottom layers, the work emphasizes the importance of weight sharing in deeper layers. The overall novelty of the work is limited though.
The authors brought up two strategies on learning the shared and private weights at the end of section 3.2. However, no follow-up comparison between the two are provided. It seems like most of the results are coming from the end-to-end learning.
Experimental results:
section 4.1: Figure 2 is flawed. The colors do not correspond to the sub-tasks. For example, there are digits 1, 4 in color magenta, which is supposed to be the shared branch of digits of 5~9. Vice versa.
From reducing the capacity of JAE to be the same as the baseline, most of the improvement is gone. It is not clear how much of the improvement will remain if the baseline model gets to see all the samples instead of just those from each sub-task.
section 4.2.1: The authors demonstrate the influence of shared layer depth in table 2. While it does seem to matter for tasks of dissimilar inputs, have the authors compare having a completely shared branch or sharing more than just a single layer?
The authors suggested in section 4.1 CIFAR experiment that the proposed method provides more performance boost when the two tasks are more similar, which seems to be contradicting to the results shown in Figure 3, where its performance is worse when transferring between USPS and MNIST, which are more similar tasks vs between SVHN and MNIST. Do the authors have any insight?


Graph classification is currently dominated by graph kernels, which, while powerful, suffer some significant limitations. Convolutional Neural Networks (CNNs) offer a very appealing alternative. However, processing graphs with CNNs is not trivial. To address this challenge, many sophisticated extensions of CNNs have recently been proposed. In this paper, we reverse the problem: rather than proposing yet another graph CNN model, we introduce a novel way to represent graphs as multi-channel image-like structures that allows them to be handled by vanilla 2D CNNs. Despite its simplicity, our method proves very competitive to state-ofthe-art graph kernels and graph CNNs, and outperforms them by a wide margin on some datasets. It is also preferable to graph kernels in terms of time complexity. Code and data are publicly available 1 .

The authors propose to use 2D CNNs for graph classification by transforming graphs to an image-like representation from its node embedding. The approach uses node2vec to obtain a node embedding, which is then compacted using PCA and turned into a stack of discretized histograms. Essentially the authors propose an approach to use a node embedding to achieve graph classification.
In my opinion there are several weak points:
1) The approach to obtain the image-like representation is not well motivated. Other approaches how to  aggregate the set of node embeddings for graph classification are known, see, e.g., "Representation Learning on Graphs: Methods and Applications", William L. Hamilton, Rex Ying, Jure Leskovec, 2017. The authors should compare to such methods as a baseline.
2) The experimental evaluation is not convincing:
- the selection of competing methods is not sufficient. I would like to suggest to add an approach similar to Duvenaud et al., "Convolutional networks on graphs for learning molecular fingerprints", NIPS 2015.
- the accuracy results are taken from other publications and it is not clear that this is an authoritative comparison; the accuracy results published for state-of-the-art graph kernels are superior to those obtained by the proposed method, cf., e.g., Kriege et al., "On Valid Optimal Assignment Kernels and Applications to Graph Classification", NIPS 2016.
- it would be interesting to apply the approach to graphs with discrete and continuous labels.
3) The authors argue that their method is preferable to graph kernels in terms of time complexity. This argument is questionable. Most graph kernels compute explicit feature maps and can therefore be used with efficient linear SVMs (unfortunately most publications use a kernelized SVM). Moreover, the running of computing the node embedding must be emphasized: On page 2 the authors claim a "constant time complexity at the instance level", which is not true when also considering the running time of node2vec. Moreover, I do not think that node2vec is more efficient than, e.g., Weisfeiler-Lehman refinement used by graph kernels.
In summary: Since the technical contribution is limited, the approach needs to be justified by an authoritative experimental comparison. This is not yet achieved with the results presented in the submitted paper. Therefore, it should not be accepted in its current form.


Published as a conference paper at ICLR 2018 META-LEARNING FOR SEMI-SUPERVISED FEW-SHOT CLASSIFICATION
In few-shot classification, we are interested in learning algorithms that train a classifier from only a handful of labeled examples. Recent progress in few-shot classification has featured meta-learning, in which a parameterized model for a learning algorithm is defined and trained on episodes representing different classification problems, each with a small labeled training set and its corresponding test set. In this work, we advance this few-shot classification paradigm towards a scenario where unlabeled examples are also available within each episode. We consider two situations: one where all unlabeled examples are assumed to belong to the same set of classes as the labeled examples of the episode, as well as the more challenging situation where examples from other distractor classes are also provided. To address this paradigm, we propose novel extensions of Prototypical Networks (Snell et al., 2017) that are augmented with the ability to use unlabeled examples when producing prototypes. These models are trained in an end-to-end way on episodes, to learn to leverage the unlabeled examples successfully. We evaluate these methods on versions of the Omniglot and miniImageNet benchmarks, adapted to this new framework augmented with unlabeled examples. We also propose a new split of ImageNet, consisting of a large set of classes, with a hierarchical structure. Our experiments confirm that our Prototypical Networks can learn to improve their predictions due to unlabeled examples, much like a semi-supervised algorithm would.

This paper proposes to extend the Prototypical Network (NIPS17) to the semi-supervised setting with three possible
strategies. One consists in self-labeling the unlabeled data and then updating the prototypes on the basis of the
assigned pseudo-labels. Another is able to deal with the case of distractors i.e.  unlabeled samples not beloning to
any of the known categories. In practice this second solution is analogous to the first, but a general 'distractor' class
is added. Finally the third technique learns to weight the samples according to their distance to the original prototypes.
These strategies are evaluated in a particular semi-supervised transfer learning setting:  the models are first trained
on some source categories with few labeled data and large unlabeled samples (this setting is derived by subselecting
multiple times a large dataset), then they are used on a final target task with again few labeled data and large
unlabeled samples but beloning to a different set of categories.
+ the paper is well written, well organized and overall easy to read
+/-  this work builds largely on previous work. It introduces only some small technical novelty inspired by soft-k-means
clustering that anyway seems to be effective.
+ different aspect of the problem are analyzed by varying the number of disctractors and varying the level of
semantic relatedness between the source and the target sets
Few notes and questions
1) why for the omniglot experiment the table reports the error results? It would be better to present accuracy as for the other tables/experiments
2) I would suggest to use source and target instead of train and test -- these two last terms are confusing because
actually there is a training phase also at test time.
3) although the paper indicate that there are different other few-shot methods that could be applicable here,
no other approach is considered besides the prothotipical network and its variants. An further external reference
could be used to give an idea of what would be the experimental result at least in the supervised case.


We propose a model to tackle classification tasks in the presence of very little training data. To this aim, we introduce a novel matching mechanism to focus on elements of the input, by using vectors that represent semantically meaningful concepts for the task at hand. By leveraging highlighted portions of the training data, a simple, yet effective, error boosting technique guides the learning process. In practice, it increases the error associated to relevant parts of the input by a given factor. Results on text classification tasks confirm the benefits of the proposed approach in both balanced and unbalanced cases, thus being of practical use when labeling new examples is expensive. In addition, the model is interpretable, as it allows for human inspection of the learned weights.

After responses:
I read the authors response and decided to stick to my original score mostly because:
1 - I understand that interpretability is hard to define. I also agree with the authors response. However, this is still not reflected in the paper in any way. I expect a discussion on what is the relevant definition used in the paper and how does it fit to that definition. Currently, it is very confusing to the reader.
2 - I understand the authors' response that few-shot learning is a different empirical setting. However, authors also agree that settings are some-what relevant. I really do not see any gain by NOT discussing the few-shot learning literature. At the end, a reader is interested in this work if they have limited data. Moreover, other ways to address limited data issue should be discussed.
-----
The manuscript is proposing a few-shot classification setting in which training set includes only few examples. The main contribution is using prototype embeddings and representing each word as cosine distances to these prototype embeddings. Moreover, the final classification is weighted summation of the per-token decisions followed by a soft-max. Per-token classifiers are obtained with an MLP using the cosine distances as features. When the relevance labels are available, they are used in training to boost gradients.
PRO(s)
The proposed method is interesting and addressing an important problem. There are many few-shot scenarios and finding good models for them is impactful.
The results are promising and the proposed method is more interpretable than the existing NLP classifiers. I disagree with the claim that the model is interpretable. However, I appreciate the effort to interpret the model.
CON(s)
The model is not interpretable because 1) it starts with embeddings and they are not interpretable, 2) model is full of non-linearities and decision boundaries are not possible to find. In other words, it is not possible to answer "what would make this model predict some other classifier".
The authors should discuss the existing few-shot learning mechanisms. Especially, "Prototypical Networks for Few-shot Learning" is very relevant. I also think it can be included as a baseline with very minimal modifications.
The writing is not complete. The authors do not even discuss how the prototypes are learned. I am assuming it is done using full gradient-descent over all parameters. However, this is not clearly discussed. Implementation details should be discussed more clearly.
SUMMARY
I believe the manuscript is definitely interesting and has a potential. In the mean time, It is not ready for publication. It needs a through review of few-shot learning. Authors should also discuss can any of the few-shot learning methods be included in the experimental study. If the answer is yes, it should be. If the answer is no, it should be explained clearly.
Although my recommendation is weak-reject, I am happy to bump it up if these issues are addressed.


Recent years have witnessed growing interests in designing efficient neural networks and neural architecture search (NAS). Although remarkable efficiency and accuracy have been achieved, existing expert designed and NAS models neglect the fact that input instances are of varying complexity and thus different amounts of computation are required. Inference with a fixed model that processes all instances through the same transformations would incur computational resources unnecessarily. Customizing the model capacity in an instance-aware manner is required to alleviate such a problem. In this paper, we propose a novel Instanceaware Selective Branching Network-ISBNet to support efficient instance-level inference by selectively bypassing transformation branches of insignificant importance weight. These weights are dynamically determined by a lightweight hypernetwork SelectionNet and recalibrated by gumbel-softmax for sparse branch selection. Extensive experiments show that ISBNet achieves extremely efficient inference in terms of parameter size and FLOPs comparing to existing networks. For example, ISBNet takes only 8.70% parameters and 31.01% FLOPs of the efficient network MobileNetV2 with comparable accuracy on CIFAR-10.

This paper proposes an instance-aware dynamic network, ISBNet, for efficient image classification. The network consists of layers of cell structures with multiple branches within. During the inference, the network uses SelectionNet to compute a "calibration weight matrix", which essentially controls which branches within the cell should be used to compute the output. Similar to previous works in NAS, this paper uses Gumbel Softmax to compute the branch selection probability. The network is trained to minimize a loss function that considers both the accuracy and the inference cost. Training of the network is divided into two stages: First, a high temperature is used to ensure all the branches are sufficiently optimized, and at the second stage, the authors aneal the temperature. During the inference, branches are selected if their probability computed by Gumbel Softmax is larger than a certain threshold.
Overall, the idea of the paper is clearly presented. The methods used in this paper are similar to previous works on neural architecture search (NAS), but this paper can be seen as a meaningful extension to NAS.
My main concern for this paper is the experiment section.
1) The paper claims that "ISBNet takes only 8.70% parameters and 31.01% FLOPs of the efficient network MobileNetV2 with comparable accuracy on CIFAR-10". Mainstream efficient networks, such as MobileNet and ShuffleNet, are not designed for CIFAR-10 datasets, but ImageNet datasets. Their downsampling strategy is very aggressive, which leads to relatively poor accuracy on CIFAR-10 datasets with 32x32 images. Therefore, it is not fair to compare the MobileNetV2 on CIFAR-10 and claim superiority over it. Compared with other networks customized for the CIFAR-10 dataset, such as NASNet-A, DARTS, and so on, the error rate of ISBNet is significantly worse (>1% higher than DARTS, or >33% relative increase in error rate).
2) In table 2, the paper compares ISBNet's performance on the ImageNet dataset with other baselines. However, the baseline models are not up to date. For example, MobileNetV2 achieves a 28% error rate with 300M FLOPs, FBNet achieves a 27% error rate with 249M FLOPs. These results, however, are not cited in this paper. Particularly, MobileNetV2 is compared against on the CIFAR-10 dataset, but not the ImageNet dataset.
3) ISBNet is not the first instance-aware dynamic network. Previous works such as SkipNet, Soft-Conditional computing [1] explored similar ideas. However, in this paper, there is no comparison with previous dynamic networks.
Overall, I would expect a much stronger experiment section for the paper to be published.
[1] https://arxiv.org/abs/1904.04971


Graph convolutional networks (GCNs) are powerful tools for graph-structured data. However, they have been recently shown to be vulnerable to topological attacks. To enhance adversarial robustness, we go beyond spectral graph theory to robust graph theory. By challenging the classical graph Laplacian, we propose a new convolution operator that is provably robust in the spectral domain and is incorporated in the GCN architecture to improve expressivity and interpretability. By extending the original graph to a sequence of graphs, we also propose a robust training paradigm that encourages transferability across graphs that span a range of spatial and spectral characteristics. The proposed approaches are demonstrated in extensive experiments to simultaneously improve performance in both benign and adversarial situations.

This paper proposes a new architecture for graph convolutional network based on graph powering operation which generates a new graph based on the shortest distance between pair of nodes.  Its main motivation is to overcome the dominance of the first eigenvector in the existing GCN architectures based on the graph Laplacian operator. The theoretical evidence for the robustness is provided based on the signal-to-noise (SNR) ratio of the simplified stochastic block model (SBM). Two versions of the algorithms are proposed, namely the robust graph convolutional network (r-GCN) and variable power network (VPN). First, r-GCN is based on augmenting the graphs with graph powering operation. Next, VPN replaces the adjacency matrix of the graph convolutional operator by the newly proposed variable power operator. An additional sparsification scheme is proposed since the graph powering operation densifies the original graph.
Overall, I like how the paper addresses the weakness of the existing graph Laplacian operators (dominance of the first eigenvector) and proposed a new method with theoretical justifications. Experiments were conducted thoroughly and results look great in the presented datasets. However, I also have concerns about the paper that I feel necessary to be resolved.
Most importantly, the concept of "robustness" in GCN seems to be inconsistent throughout the paper.  Namely, the meaning of robustness in the neural network (adversarial robustness) and the SBM literature (spectral robustness) are different. This point is crucial since the paper use the spectral robustness for justification of the method, yet experiments are done on the adversarial attacks. More specifically, adversarial training methods for neural networks, e.g., adversarial attack methods [1] considered in the paper, typically make the loss function (or output of network) more persistent against the small perturbation of inputs. On the other side, the robustness for SBM models, e.g., Theorem 3 in the paper, cares more about the preservation of the original input characteristics. For illustration, an invertible neural network [2] is not necessarily robust to adversarial attacks (the first meaning of robustness) but preserves all the input characteristics (the second meaning of robustness).
I also hope the paper could have done the experiments on more datasets since there exists some evidence on the unreliability of evaluations on citation networks [3].  However, I do not think this point is critical since the paper did a great job of evaluating the robustness in various aspects and they all show consistent improvement.
Minor questions and suggestions:
- The acronyms are slightly confusing to understand at first sight, since they first appear at the equations without any information on what the letters stand for.  Something like a "variable power network (VPN)" would make the paper more pleasant to read.
- In the r-GCN framework, there might be an edge case where the powered graph is almost identical to another graph. Would there be any justification for avoiding this?
- In the r-GCN framework, the terminology distillation is slightly confusing. Was this choice of word used for making a connection to the knowledge distillation [4]? How is the knowledge distilled between graphs?
References
[1] Bojchevski and Günnemann. Adversarial attacks on node embeddings via graph poisoning. ICML 2019
[2] Jacobsen et al., i-RevNet: Deep Invertible Networks. ICLR 2018
[3] Shchur et al., Pitfalls of Graph Neural Network Evaluation, Arxiv 2018
[4] Hinton et al., Distilling the Knowledge in a Neural Network, Arxiv 2015


Many methods have been developed to represent knowledge graph data, which implicitly exploit low-rank latent structure in the data to encode known information and enable unknown facts to be inferred. To predict whether a relationship holds between entities, their embeddings are typically compared in the latent space following a relation-specific mapping. Whilst link prediction has steadily improved, the latent structure, and hence why such models capture semantic information, remains unexplained. We build on recent theoretical interpretation of word embeddings to derive an explicit structure for the representations of relations between entities. From this we explain properties and justify the relative performance of leading knowledge graph representation methods for identifiable relation types, including their often overlooked ability to make independent predictions.

This paper proposes to provide a detailed study on the explainability of link prediction (LP) models by utilizing a recent interpretation of word embeddings. More specifically, the authors categorize the relations in KG into three categories (R, S, C) using the correlation between the semantic relation between two words and the geometric relationship between their embeddings. The authors utilize this categorization to provide a better understanding of LP models’ performance through several experiments.
This paper reads well and the results appear sound. I personally believe that works on better understanding KGC models are a very essential direction which is mostly ignored in this field of study. Moreover, the provided experiments support the authors’ intuition and arguments.
As for the drawbacks, I find the technical novelty of the paper is somewhat limited, as the proposed method consists of a mostly straightforward combination of existing methods. Further, I believe this work needs more experimental results and decisive conclusions identifying future directions to achieve better performance on link prediction. My concerns are as follows:
•    I am wondering about the reason for omitting Max/Avg path for two of the relations in WN18RR? Further, the average of 15.2 for the shortest path between entities with “also_see” relation appears to be a mistake?
•    Was there any specific reason in choosing WN18RR and NELL-995 KGs for the experiments?
•    It would be interesting to see the length of paths between entities for train and test data separately.
•    I suggest providing a statistical significance evaluation for each experiment to better validate the conclusions.
•    I find the provided study in section 4.2 very similar to the triple classification task in KGs. Can you elaborate on the differences and potential advantages of your setting?
•    I am wondering how you identified the “Other True” triples for WN18RR KG in section 4.2 experiments?
On overall, although I find the proposed study very interesting and enlightening, I believe that the paper needs more experimental results and decisive conclusions.


This paper presents a phenomenon in neural networks that we refer to as local elasticity. Roughly speaking, a classifier is said to be locally elastic if its prediction at a feature vector x is not significantly perturbed, after the classifier is updated via stochastic gradient descent at a (labeled) feature vector x that is dissimilar to x in a certain sense. This phenomenon is shown to persist for neural networks with nonlinear activation functions through extensive simulations on real-life and synthetic datasets, whereas this is not observed in linear classifiers. In addition, we offer a geometric interpretation of local elasticity using the neural tangent kernel (Jacot et al., 2018). Building on top of local elasticity, we obtain pairwise similarity measures between feature vectors, which can be used for clustering in conjunction with K-means. The effectiveness of the clustering algorithm on the MNIST and CIFAR-10 datasets in turn corroborates the hypothesis of local elasticity of neural networks on real-life data. Finally, we discuss some implications of local elasticity to shed light on several intriguing aspects of deep neural networks.

[Summary]
This paper proposes and studies the “local elasticity”, a quantitative measure for the ability of neural networks to only locally change its prediction (around x) after a stochastic gradient step at x. The paper verifies experimentally that nonlinear neural nets are locally elastic through showing that an elasticity-motivated similarity score can perform clustering well.
[Pros]
The notion of local elasticity is interesting and has the potential of opening up lots of further directions. The way I understand it is to relate to memorization (as the authors have indeed discussed) --- I think “local elasticity” can be viewed as some sort of “local memorization ability”, in that the NN is able to change its prediction only in a small neighborhood of x---without affecting predictions at other remote x’s---after one SGD step on x. Conceptually this is something not covered by the existing narratives in deep learning theory, yet the phenomenon itself is quite convincing and could provide a new perspective into lots of things.
[Cons]
It feels like the experimental results in the present paper is a rather indirect evidence for the local elasticity -- that the similarity score coming from the elasticity works well for a downstream clustering task. Could there be some more direct evidence about the local elasticity? How would the elasticity compare on different architectures? In the present form the experiments perhaps at most says that the similarity score makes sense, not yet that a fully quantitative characterization of the local elasticity.
I’m also a little bit concerned about the fairness of the clustering experiment, in that the elasticity-motivated clustering algorithm utilizes an auxiliary dataset whereas simple baselines such as K-means and PCA K-means are not able to use that. Is there a way of modifying the K-means and PCA K-means so that they can also use this auxiliary dataset while still giving a sensible algorithm for the primary 2-class clustering task?


We undertake the problem of representation learning for time-series by considering a Group Transform approach. This framework allows us to, first, generalize classical time-frequency transformations such as the Wavelet Transform, and second, to enable the learnability of the representation. While the creation of the Wavelet Transform filter-bank relies on affine transformations of a mother filter, our approach allows for non-linear transformations. This is achieved by sampling a subset of invertible maps on R. The subset considered contains strictly increasing and continuous functions. The transformations induced by such maps enable us to span a larger class of signal representations, from wavelet to chirplet-like filters. We propose a parameterization of such a non-linear map such that its sampling can be optimized for a specific loss and signal. The Learnable Group Transform can thus be cast into a Deep Neural Network. The experiments on diverse time-series datasets demonstrate the expressivity of this framework, which competes with state-of-the-art performances.

A typical Wavelet Transform is built through the dilation and/or rotation of a mother wavelet, which can been viewed as a group action on a mother wavelet. This work proposes to extend this construction beyond the Euclidean group, and to supervisedly learn operators that will be applied on a mother wavelet. Competitive numerical performances are obtained.
Overall, I think that re-thinking the way a Wavelet Transform is designed, is an interesting direction of research, but I think some of the theoretical tools developed in this paper are not dedicated to achieve this purpose. In particular, the group/representation properties seem to not be used, and the authors could simply consider a specific subset of invertible mapping on $\mathbb{R}^2$ which would be applied on the mother wavelet and lead to a Wavelet Transform. In other words, the overall formulation could be simplified.
Pros:
- In general, the numerical experiments are at the level of the state of the art.
- Parametrizing a subset of the group of increasing function and its application to signal processing tools is novel, to my knowledge.
Cons:
- Some very relevant elements in the literature review are missing. Learning or using an underlying group of symmetry that will be combined with a deep neural network is not novel, cf: https://arxiv.org/abs/1601.04920 ; in particular for reducing the number of parameters, filters or samples: https://arxiv.org/abs/1809.06367 ; http://proceedings.mlr.press/v48/cohenc16.pdf ; https://arxiv.org/abs/1809.10200 ; https://arxiv.org/abs/1605.06644 - I think the authors should discuss at least one or two of those papers, if not all.
- The performance on the bird detection task is good but the improvement compared to other work is not clear, given that some supervision in the first layer is incorporated.
- Subsections 2.2 and 2.3 are difficult to parse because the authors introduce a lot of equations or notion that are not useful to understand their algorithm/method. The equation (6) seems wrong to me (one should consider t->s_i(-t) and not t->s_i(t) and b seems missing in the second line).
- Figure 2 is difficult to read because of the illustrative graphics. Maybe a block schema would be easier to parse.
- It seems to me that no-where the group properties are used, such as the stability to composition. In this paper, the authors simply try to parametrize a diffeomorphism to dilate the mother wavelet. From my understanding of 3.3, the subset of function used to approximate $G_{inc}$ do not form a subgroup as well, contrary to the Euclidean case, where for instance discrete rotations in the case of images are a finite group.
- In subsection 4.1(Table 1), a comparison with a wavelet transform followed by a linear operator is compared with the proposed method. I find this result surprising :
LGT/nLGT/cLGT/cnLGT and the WT are some linear methods whereas the STFT is non linear. As the WT should be unitary, if the linear classifier method is reasonably trained, then both methods should lead to the same result, except if the data are poorly conditioned. In which case, this experiment would not be meaningful. I think the authors should comment more this result because it is surprising.
- I slightly disagree with the sentence "in the case of WT, the precision in frequency degrades as the frequency increases". Actually, the heisenberg principle is optimally optimized by wavelets, meaning that the area of the frequency/spatial support on a spectrogram is constant. On the contrary, the STFT has a lack of localisation (and thus the "precision" is not constant along frequencies). Maybe this could be rephrased slightly.
- Given the filter learned in Figure 5, one can wonder if a foveal approach (i.e., Foveal Wavelets) could perform similarly? It would be interesting to display the littlewood-paley plot(i.e., the sum of the modulus of the filters in the Fourier domain) of this representation to understand the nature of this operator in the Fourier domain.
- I think group actions could be considered instead of representations: it would be simpler to understand for a potential reader.
Typos:
- abstract "in order to transform the mother .." > "in order to transform a mother.."
- page 7 "this variation is as not captured as well" > "this variation is not captured as well".


Many real-world data sets are represented as graphs, such as citation links, social media, and biological interaction. The volatile graph structure makes it non-trivial to employ convolutional neural networks (CNN's) for graph data processing. Recently, graph attention network (GAT) has proven a promising framework by combining graph neural networks with attention mechanism, so as to achieve massage passing in graphs with arbitrary structures. However, the attention in GAT is computed mainly based on the similarity between the node content, while the structures of the graph remains largely unemployed (except in masking the attention out of one-hop neighbors). In this paper, we propose an "adaptive structural fingerprint" (ADSF) model to fully exploit both topological details of the graph and content features of the nodes. The key idea is to contextualize each node with a weighted, learnable receptive field encoding rich and diverse local graph structures. By doing this, structural interactions between the nodes can be inferred accurately, thus improving subsequent attention layer as well as the convergence of learning. Furthermore, our model provides a useful platform for different subspaces of node features and various scales of graph structures to "cross-talk" with each other through the learning of multi-head attention, being particularly useful in handling complex real-world data. Encouraging performance is observed on a number of benchmark data sets in node classification.

---- Problem setting and contribution summary ----
The paper considers the problem of graph node classification in a semi-supervised learning setting. When classifying a node, the decision is based on the node’s features, as well as by a weighted combination of its neighbors (where the weights are computed using a learnt attention mechanism). The authors extend upon the recent Graph Attention Networks (GAT) paper by proposing a different way of computing the attention over the neighboring nodes. This new attention mechanism takes into account not just their feature similarity, but also extra structure information, which also enables their method to attend not only over direct neighbors, but also up to k-hop neighbors.
---- Overall opinion ----
While I believe the general idea indeed has merit and empirically shows great promise, I believe the paper in its current state is not ready for publication. However, I believe that a more thorough revision can lead to an a publication with potential impact on the applications side.
---- Pros ----
1. The paper is easy to read.
2. I really appreciated the good visualizations (Figures 2,3,4) that indeed help in understanding the method.
3. Really good empirical results on the 3 datasets that were presented.
---- Major issues ----
1. Motivation:
I believe the paper is not well motivated from an applications perspective. In section 2.1., the authors did a good job explaining the limitations of current approaches on a generic graph structure, but this is only under the main assumption that a node should attend more to neighbors in its denser community than other neighbors that not connected so strongly (Fig 1). The issue that I have with this is that:
      a) Why should we take this assumption for granted? What are some concrete practical node classification problems where it is indeed better if a node attends to its neighbors in this way (as opposed to the GAT approach)?
      b) Even if the above is proven true, suppose node A in Fig. 1(a) attends with equal weights to all its 4 neighbors. That means node C (which is outside its densest community) gets 1/4, while the nodes inside the dense community get a total of ¾. That means node A puts most of its attention to the dense community anyway. In what conditions is it necessary to bias this attention even further?
2. Experiments:
While the reported accuracies for the three datasets look good and also the authors have provided a link to their code (great to see that!), I believe the experimental section is missing an important amount of details for reproducibility purposes and also for explaining how certain parameters have been chosen:
    a) There are no details on the model size and training procedure (hidden units, optimizer, learning rate schedule).
    b) Maybe I am missing this, but I don’t see any reference on what alpha and beta from equation (6) were used in the experiments.
    c) What is the value of k in Table 2? How did you choose it? I believe Figure 5 shows test accuracies for different k, but I hope the authors did not choose k based on the test set performance.
    d) How did you select the structural fingerprint to be 3?
     e) “...optimizing c through the learning process also gives very similar choice” → how did you optimize c exactly?
     f) Fig 5 a) Why does increasing the number of hops to 3 or 4 decrease the performance so much? Shouldn’t the attention weights learn to ignore the further neighbors, if they are not useful?
	Another important question regarding experiments: since the ablation study shows that the optimal neighbor range is actually 2, a natural baseline to compare with would be something similar to GAT, where an attention weight is applied to all neighbors within two hops (basically skip the fingerprint step, and assume s_{ij} is 1 for all neighbors within 2 hops, and 0 otherwise).
	Also regarding experiments, these 3 datasets, although common across many recent graph node classification papers, they are known to be quite limited (small in size and not very representative of real world). Since GAT is your main competitor, why not also show experiments on the PPI dataset they also test?
3. Writing quality:
While the language is clear and easy to follow, there are many grammatical mistakes throughout the paper (e.g. “benefitial”, subject-verb agreement).
---- More minor issues ----
    a) Section 2.1: One could argue that GAT also contains longer range node dependencies through the node embeddings it learns. Since the node embeddings is trained through gradient descent, and at each iteration the embedding of a node changes according to its neighbors, you could say that information does get propagated from the neighbor’s neighbors.
     b) Why do you need a LeakyRelu in Equation (5) ? Also, aren’t e_{ij} non-negative anyway (in which case LeakyRelu doesn’t change anything)?
     c) Why would a Sigmoid be a good choice for alpha and beta in Eq. (6)?
     d) Section 3: A bag of words is typically represented as a binary vector, which is also categorical.
     e) Please use \citet when specifically referring to the authors of a paper as part of your sentence (e.g. “Following Velickovic et. al., 2017 we….” as opposed to “Following (Velickovic et al., 2017), we....”).


Recent Transformer-based models such as Transformer-XL and BERT have achieved huge success on various natural language processing tasks. However, contextualized embeddings at the output layer of these powerful models tend to degenerate and occupy an anisotropic cone in the vector space, which is called the representation degeneration problem. In this paper, we propose a novel spectrum control approach to address this degeneration problem. The core idea of our method is to directly guide the spectra training of the output embedding matrix with a slow-decaying singular value prior distribution through a reparameterization framework. We show that our proposed method encourages isotropy of the learned word representations while maintains the modeling power of these contextual neural models. We further provide a theoretical analysis and insight on the benefit of modeling singular value distribution. We demonstrate that our spectrum control method outperforms the state-of-the-art Transformer-XL modeling for language model, and various Transformer-based models for machine translation, on common benchmark datasets for these tasks.

Summary: This paper deals with the representation degeneration problem in neural language generation, as some prior works have found that the singular value distribution of the (input-output-tied) word embedding matrix decays quickly. The authors proposed an approach that directly penalizes deviations of the SV distribution from the two prior distributions, as well as a few other auxiliary losses on the orthogonality of U and V (which are now learnable). The experiments were conducted on small and large scale language modeling datasets as well as the relatively small IWSLT 2014 De-En MT dataset.
Pros:
+ The paper is well-written with great clarity. The dimensionality of the involved matrices (and their decompositions) are clearly provided, and the approach is clearly described. The authors also did a great job providing the details of their experimental setup.
+ The experiments seem to show consistent improvements over the baseline methods (at least the ones listed by the authors) on a relatively extensive set of tasks (e.g., of both small and large scales, of two different NLP tasks). Via WT2 and WT103, the authors also showed that their method worked on both LSTM and Transformers (which it should, as the SVD on word embedding should be independent of the underlying architecture).
+ I think studying the expressivity of the output embedding matrix layer is a very interesting (and important) topic for NLP. (e.g., While models like BERT are widely used, the actual most frequently re-used module of BERT is its pre-trained word embeddings.)
---------------------------------
I have a few questions/comments on the work as well:
1) One of the things that is not clearly described in the paper is how the proposed spectrum control method was injected into training in practice. In Section 4.3 (when you do the theoretical analysis), you assumed "all the other parameters are fixed and well-optimized". Is that also what you did in training on WT2/WT103 (e.g., first pre-train a model such as Transformer-XL, and then fine-tune its embedding layer using the proposed decomposed method)?
2) How does the runtime and memory cost of your approach compare to the baselines? (e.g., you now need to compute $U^\top U$, which can also be prohibitively large when the vocabulary size is large; for instance, on the 1-Billion word dataset).
3) I remember the base Transformer-XL model did not use the adaptive embedding/softmax (although they set the `--adaptive` flag, they did use `--div_val 1`). How does your method work in the adaptive setting, where the embedding size $d$ of different words could be very different (e.g., depending on the word frequency)?
4) Regarding the theoretical analysis in Section 4.3. Why is the cross-entropy loss function (essentially NLL loss + softmax) bounded (as is required by Theorem 4.1)? Given a fixed ground-truth $y_i$, if the corresponding predicted likelihood is small (e.g., $\rightarrow 0$), won't the CE loss be very large? In that case, the generalization bound in Eq. (4.3) would be vacuous, as B would be very large too. Moreover, I'm also not fully convinced by what the theoretical analysis is trying to convey--- that your method "achieve a trade-off between the training loss and generalization error"--- but isn't that what (any) regularizations are designed for? The proved bound is no different in nature from any generalization bound with a regularizer (e.g., weight decay), and it does not necessarily reflect the usefulness of the proposed approach.
5) In experiments, you only showed the better performance of the exponential and the polynomial singular value decays (cf. beginning of Sec. 5). Could you show both? Which one is better (and on what task), and by how much? If one is to use your method, which decay scheme do you recommend?
6) Why is MLE-CosReg (Gao et al. 2019b) not compared to in the WikiText-103 and the machine translation task? As the MLE-CosReg approach only involves regularizing the cosine similarity via $\text{Sum}(\hat{W}\hat{W}^\top)$, it should computationally be even slightly cheaper than the proposed method (you need to compute $\mathbf{U}^\top \mathbf{U}$, which has the same complexity). They also tested on the larger-scale WMT En-De and De-En dataset (which contains 4.5M sentence pairs). Is there any reason that you chose IWSLT 2014 instead?
---------------------------------
Some issues that didn't impact the score:
7) It'd be useful to add labels to the x- and y-axis of the plots in Figure 1.
8) When implementing the method described in Section 4.2, did you explicitly sort the singular values in each iteration, or just set them to learnable parameters (along with learnable $\mathbf{U}, \mathbf{V}$) without sorting? If you do sort, do you also "sort" the columns of $\mathbf{U}$ and $\mathbf{V}$ (as I would expect a one-to-one mapping from $\sigma_i$ to $\mathbf{U}_i$, for instance)? If you don't sort, how do you make sure that $\sigma_i \geq \sigma_{i+1}$?
9) Why did you regularize $\mathbf{U}$ and $\mathbf{V}$ by both its Frobenius norm and its spectral norm? Does using only one of them compromise the performance?
---------------------------------
Overall, I find this work well-written and well-motivated. The experiments seem to show consistent improvement when using the approach. I vote for weak accept, but I also look forward to the author's response to the questions I raised above.


We train embodied agents to play Visual Hide and Seek where a prey must navigate in a simulated environment in order to avoid capture from a predator. We place a variety of obstacles in the environment for the prey to hide behind, and we only give the agents partial observations of their environment using an egocentric perspective. Although we train the model to play this game from scratch, experiments and visualizations suggest that the agent learns to predict its own visibility in the environment. Furthermore, we quantitatively analyze how agent weaknesses, such as slower speed, effect the learned policy. Our results suggest that, although agent weaknesses make the learning problem more challenging, they also cause more useful features to be learned.

# Review ICLR20, Visual Hide and Seek
This review is for the originally uploaded version of this article. Comments from other reviewers and revisions have deliberately not been taken into account. After publishing this review, this reviewer will participate in the forum discussion and help the authors improve the paper.
## Overall
**Summary**
The authors introduce a new RL environment and task, "Visual Hide and Seek", in which they analyze how the agent's learned visual representations are impacted by its speed, auxiliary rewards, and opponent behavior.
**Overall Opinion**
This paper presents a thorough analysis and great visualizations of agent behaviors and representations under different conditions. I wish more papers would put this much effort into analyzing their agents. I'd highly recommend this paper get accepted since I believe the analysis carried out here and the conclusions reached are quite novel and the paper is overall well-written.
However, at the same time, the work of [Baker et al., 2019][1] was published with significantly more fanfare. I hope their work does not overshadow this one since they are only related in the general task concept.
[1]: https://arxiv.org/abs/1909.07528
Some major issues I had with this work:
- In general, please run more random seeds. Just reporting on a single random seed is not enough, as per [Henderson et al., 2018][2].
- There are some sections of the paper where the order of paragraphs is confusing. You start the introduction by stating what you've done and letting the reader wonder "why?". The explanation is only given in the second paragraph. So I'd suggest rotating the second paragraph upwards before the first. Similarly, at the beginning of section 4, you just mention the results - this should either be shorter (1 sentence, as an overview of the work in this section) or moved to the end of that section.
- You're missing a section about future work and flaws/problems of your work at the very end (the latter if which should be in "Discussion"), which is common to include in ICLR publications.
[2]: https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/viewFile/16669/16677
Here are some minor...
## Specific comments and questions
### Abstract
all good
### Intro
- Fig.1: What is that white thing that the seeker/hider have on the capsule?
### Rel. Work
all good
### Method
- What's the speed (FPS) of that Unity engine? Why didn't you use Mujuco/(Py)Bullet/Gym-Miniworld?
- You need to add some measurements and units: the arena size doesn't have a unit, the size (diameter) of the hiders/seekers is unclear, turning left/right is unclear (how far left/right, after action-repeat)
- You mention any real-valued position to be valid - so the agents can step into obstacles? And how about through obstacles?
- "Affordance Learning" is usually not used for static environment geometry like obstacles (e.g. [Georgia-Tech course on "Human-Robot Interaction"][3])
- Why 4-layer CNNs? Why not 6 or 8 or a ResNet? Would you think the features would be stronger/weaker in an 8-layer CNN?
[3]: https://www.cc.gatech.edu/~athomaz/classes/CS8803-HRI-Spr08/MayaChandan/Site/Affordance_Learning.html
### Experiments
- 4.1 "... learned this play game" -> "... learned this game".
- 4.2 "mid-level features" -> what's that? The activation of the convolutional kernels after the second layer CNN? What's the dimension? And why did you pick the 2nd layer, not any of the other 3?
- Tab.3: This is averaged over how many frames of rollout?
- "... case can moves a lot faster." -> "... case can move a lot faster".
- Fig.2 is very interesting. Well done.
- Fig.3: the font is not consistent with other figures
- Fig.4: remove the blueish background to increase contrast. Increase the font size of the ticks on the left. Make the legend color boxes slightly bigger. Add more space or a visual divider between the different states - especially on the right side it's hard to make out where one stops and the next begins.
- Fig.4/5: This analysis is lovely and we need more of this in DRL.
- 4.4 in the text, you sometimes write "not S" and sometimes "¬S". Please change the "not s"
- Fig.5: (suggestion) Merge/sum the 2 columns (in both A/B merge the left and right plot into one by summing); subtract the random policy values as you did with Fig.4.
- "We summarize representative cases, and put the full results for all combinations in the Appendix" - no you didn't.
- Fig.6: What is going on in the left third of this diagram? What is this colorful mush? If this is by any chance indicating a change over time, do you maybe want to spread a single, very colorful plot of distance over time into multiple less colorful plots? At least add a legend, please. Also, I'd recommend smoothing (moving average or smoothing spline).
- Also maybe add reward over time plots, as is common in DRL, to show that your policies converged after 8 mil. steps.
### Conclusion
All good, save for the missing future work and critical analysis of your work.
### Appendix
all good


Deep approaches to anomaly detection have recently shown promising results over shallow methods on large and complex datasets. Typically anomaly detection is treated as an unsupervised learning problem. In practice however, one may have-in addition to a large set of unlabeled samples-access to a small pool of labeled samples, e.g. a subset verified by some domain expert as being normal or anomalous. Semi-supervised approaches to anomaly detection aim to utilize such labeled samples, but most proposed methods are limited to merely including labeled normal samples. Only a few methods take advantage of labeled anomalies, with existing deep approaches being domain-specific. In this work we present Deep SAD, an end-to-end deep methodology for general semi-supervised anomaly detection. Using an information-theoretic perspective on anomaly detection, we derive a loss motivated by the idea that the entropy of the latent distribution for normal data should be lower than the entropy of the anomalous distribution. We demonstrate in extensive experiments on MNIST, Fashion-MNIST, and CIFAR-10, along with other anomaly detection benchmark datasets, that our method is on par or outperforms shallow, hybrid, and deep competitors, yielding appreciable performance improvements even when provided with only little labeled data.

[Summary]
The paper proposes an abnormal detection (AD) framework under general settings where 1) unlabeled data, 2) labeled positive (normal) data, and 3) labeled negative (abnormal) data are available (with the last two optional), denoted as semi-supervised AD. Starting from the assumption that abnormal data are sampled from background unpredicted distribution, rather than “cluster” assumption, it is argued that conventional discriminative formulation is not applicable. Motivated by the recent deep AD methods (e.g., deep SVDD), the paper proposes to approach semi-supervised AD from the information theoretic perspective where 1) mutual information between raw data and learnt representation should be maximized (infomax principle), 2) entropy of labeled positive data should be minimized (“compactness” constraint), and 3) enrtropy of labeled negative data should be maximized to reflect the uncertainty assumption of anomaly. The solution is implemented by the encoder of a pre-trained autoencoder that is further fine tuned to enforce entropy assumption on all types of training data. Extensive experiments on benchmarks suggests promising results on the proposed framework versus other state-of-the-arts.
[Comments]
The paper is well written and easy to follow (the presentation is especially pleasant to read). The problem is well defined and of interest to the community under fairly general and practical conditions. Despite the fact that the implementation is only marginally tweaked from previous work (deep SVDD), the theoretical motivation, nevertheless, is sound and well justified, and the empirical evaluation is extensive to reveal the behaviors of the proposed method. It would be better if complexity analysis can also be provided for all concerning methods. Overall, the value of the paper is worth circulation in the community.
[Area to improve]
The manuscript could be further improved by exploring the training process more. In the current format, the solution follows the strategy of deep SVDD that learns the model in two separate stages: pre-training the autoencoder, and then fitting the encoder to enforce compactness and entropy minimization/maximization. What if these are implemented in an end-to-end fashion? Will this help to achieve a better result?


In multi-agent systems, complex interacting behaviors arise due to heavy correlations among agents. However, prior works on modeling multi-agent interactions from demonstrations have largely been constrained by assuming the independence among policies and their reward structures. In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents' policies, which is able to recover agents' policies that can regenerate similar interactions. Consequently, we develop a Decentralized Adversarial Imitation Learning algorithm with Correlated policies (CoDAIL), which allows for decentralized training and execution. Various experiments demonstrate that CoDAIL can better regenerate complex interactions close to the demonstrators and outperforms state-of-the-art multi-agent imitation learning methods.

This paper proposes to model interactions in a multi-agent system by considering correlated policies. In order to do so, the work modifies the GAIL framework to derive a learning objective. Similar to GAIL, the discriminator distinguishes between state, action, next state sequences but crucially the actions here are considered for all agents.
The paper is a natural extension of GAIL/MA-GAIL. I have two major points that need to be addressed.
1. The exposition and significance of some of the theoretical results is unclear.
- The non-correlated and correlated eqns in 2nd and 3rd line in eq. 8 are not equivalent in general, yet connected via an equality.
 In particular, Proposition 2 considers an importance weighting procedure to reweight state, action, next state triplets. It is unclear how this resolves the shortcomings of pi_E^{-1} being inaccessible. Prop 2 shifts from pi_E^{-1} to pi^{-1} and hence, the expectations in Prop 2 and Eq. 11 are not equivalent.
- More importantly, how are the importance weights estimated in Eq. 12? The numerator requires pi_E^{-1}, which is not accessible. If the numerator and denominator are estimated separately, it becomes a chicken-and-egg problem since the denominator is itself intended to be an imitating the expert policy appearing in the numerator?
2. Missing related work
There is a huge body of missing work in multi-agent interactions modeling and generative modeling. [1, 2] consider modeling of agent interactions via imitation learning and a principled evaluation framework of generalization in the Markov games setting. By sharing parameters, they are also able to model correlations across agent policies and have strong results on generalization to cooperation/competition with unseen agents with similar policies (which wouldn't have been possible if correlations were not modeled). Similarly, [3, 4] are other similar works which consider modeling of other agent interactions/diverse behaviors via imitation style approaches. Finally, the idea of correcting for the mismatch in state, action, next state triplets in Proposition 2 has been considered for model-based off-policy evaluation in [5]. They proposed a likelihood-free method to estimate importance weights, which seems might be necessary for this task as well (re: qs. on how are importance weights estimated?).
Re:experiments. Results look good and convincing for most parts. I don't see much value of the qualitative evaluation in Figure 1. If the KL divergence is low, we can expect the marginals to be better estimated. Trying out various levels of generalization as proposed in [2] would significantly strengthen the paper.
Typos
sec 2.1 Transition dynamics should have range in R+
Proof of Prop 2. \mu instead of u
References:
[1] Learning Policy Representations in Multiagent Systems. ICML 2018.
[2] Evaluating Generalization in Multiagent Systems using Agent-Interaction Graphs. AAMAS 2018.
[3] Machine Theory of Mind. ICML 2018.
[4] Robust imitation of diverse behaviors. NeurIPS 2017.
[5] Bias Correction of Learned Generative Models using Likelihood-free Importance Weighting. NeurIPS 2019.


We address the challenging problem of efficient deep learning model deployment, where the goal is to design neural network architectures that can fit different hardware platform constraints. Most of the traditional approaches either manually design or use Neural Architecture Search (NAS) to find a specialized neural network and train it from scratch for each case, which is computationally expensive and unscalable. Our key idea is to decouple model training from architecture search to save the cost. To this end, we propose to train a once-for-all network (OFA) that supports diverse architectural settings (depth, width, kernel size, and resolution). Given a deployment scenario, we can then quickly get a specialized sub-network by selecting from the OFA network without additional training. To prevent interference between many sub-networks during training, we also propose a novel progressive shrinking algorithm, which can train a surprisingly large number of sub-networks (> 10 19 ) simultaneously. Extensive experiments on various hardware platforms (CPU, GPU, mCPU, mGPU, FPGA accelerator) show that OFA consistently achieves the same level (or better) ImageNet accuracy than SOTA NAS methods while reducing orders of magnitude GPU hours and CO 2 emission than NAS. In particular, OFA requires 16× fewer GPU hours than ProxylessNAS, 19× fewer GPU hours than FBNet and 1,300× fewer GPU hours than MnasNet under 40 deployment scenarios.

In this papers, the authors learn a Once-for-all net. This starts as a big neural network which is trained normally (albeit with input images of different resolutions). It is then fine-tuned while sampling sub-networks with progressively smaller kernels, then lower depth, then width (while still sampling larger networks occasionally, as it reads). This results in a network from which one can extract sub-networks for various resource constraints (latency, memory etc.) that perform well without a need for retraining.
This paper is well written, and the results are very good. However there are serious problems that need addressing.
The method as described *is not reproducible*. The scheduling of sampling subnetworks is alluded to on page 4, and that's it. It is essential that the authors include their exact subnet sampling schedule e.g. as pseudocode with hyperparameters. There is no point doing good work if other researchers cannot build off it.
On another reproducibility note, as far as I can tell, the original model isn't given. There would be no harm in adding this to the appendix.
Figure 1 is misleading, as we don't find out until later in the paper that Once For All #25 means that each of these points was finetuned for a further 25 epochs (which on ImageNet is non-trivial). This defeats the narrative of the paper (once-for-all plus some fine-tuning isn't exactly once-for-all).
Is there a reason why the progressive shrinking goes resolution->kernel->depth->width? Was this just the permutation that worked best? I would be curious as to why this is.
For elastic width, I wasn't sure why the "channel sorting operation preserves the accuracy of larger sub-networks". Could you please elaborate?
Kudos on adding CO2 emissions in Table 2, I hope this gets reported more often.
In the introduction, the authors talk about iPhones and then the hardware considered is Samsung and Google. A minor note, but it seems inconsistent.
Another minor note, in Table 2, (Strubell et al) should be out of the brackets, as it is part of the sentence.
Given that there are 10^19 subnetworks that can be sampled, it would be nice to see more than 3-4 appear on a plot. This makes it seem like they might have been cherry-picked. Sampling a few 100/1000 subnets and producing some Pareto curves would be both interesting and insightful.
Pros
-------
- Good results
- Well written
- Neat idea
Cons
-------
- Training details are obfuscated. This paper should not be accepted without them.
- Very few subnetworks of the vast quantity that exist are observed.
In conclusion, I am giving this paper a weak reject, as it is currently impossible to reproduce, and as such, is of no use to the community. If the authors remedy this I will gladly raise my score.


Program verification offers a framework for ensuring program correctness and therefore systematically eliminating different classes of bugs. Inferring loop invariants is one of the main challenges behind automated verification of real-world programs which often contain many loops. In this paper, we present Continuous Logic Network (CLN), a novel neural architecture for automatically learning loop invariants directly from program execution traces. Unlike existing neural networks, CLNs can learn precise and explicit representations of formulas in Satisfiability Modulo Theories (SMT) for loop invariants from program execution traces. We develop a new sound and complete semantic mapping for assigning SMT formulas to continuous truth values that allows CLNs to be trained efficiently. We use CLNs to implement a new inference system for loop invariants, CLN2INV, that significantly outperforms existing approaches on the popular Code2Inv dataset. CLN2INV is the first tool to solve all 124 theoretically solvable problems in the Code2Inv dataset. Moreover, CLN2INV takes only 1.1 second on average for each problem, which is 40× faster than existing approaches. We further demonstrate that CLN2INV can even learn 12 significantly more complex loop invariants than the ones required for the Code2Inv dataset.

Summary:
This paper introduces a novel way to find loop invariants for a
program. The loop invariants are expressed as SMT formulas. A
continuous relaxation of an SMT solver is introduces mapping
every SMT formula onto a truth value \in [0, 1]. This relaxation
called a continuous semantic mapping is decided such that every
true formula has a higher value than every false formula. This
allows an invariant to be learned.
Novelty and Significance:
This work is interesting and although authors do seem to be
outside of the community, I firmly believe is appropriate for
ICLR. If the claims made by the paper are true they constitute
a significant contribution to the field of program synthesis
and program analysis.
Technical Quality:
The evaluation was fairly thorough, but the paper can be
strengthened massively with a few small changes and additions.
It might be more helpful if there was a sense of how many problems
none of the systems can do and how complicated of a program can
this system extract a loop invariant from. Why were was it these
particular 12 that work? Are there examples that don't?
I don't know why all this time is spent on t-norms when behavior
on them is fairly similar and the simplest norm works best.
Lots of details are missing in this paper. How much training data
was generated? How long did it take? Does training data need to
be generated for each example? If so is that included in the
runtime for Figure 3?
The paper talks about neural architecture, but all I see is
effectively a curve-fitting task for some template. This feels
different from the code2inv paper where a program can be fed
into the system and the pretrained model emits the invariant.
Clarity:
Not enough of this paper concentrates on the novel aspects of the
approach. Section 5 discusses template generation, but not in
enough detail that I would be able to replicate this work. I
could also not find enough details in the Appendix.
I don't know why vital page space is spent on defining completeness
and soundness.
Possibly related work as relaxations to SAT/SMT solvers do exist in the literature.
Guiding High-Performance SAT Solvers with Unsat-Core Predictions
https://arxiv.org/abs/1903.04671
Learning to Solve SMT Formulas
https://papers.nips.cc/paper/8233-learning-to-solve-smt-formulas.pdf
Notes:
You cite Si et al. for LoopInvGen when you should be citing Pathi
and Millstein in the third paragraph on page 2.


Model-based reinforcement learning (MBRL) is widely seen as having the potential to be significantly more sample efficient than model-free RL. However, research in model-based RL has not been very standardized. It is fairly common for authors to experiment with self-designed environments, and there are several separate lines of research, which are sometimes closed-sourced or not reproducible. Accordingly, it is an open question how these various existing algorithms perform relative to each other. To facilitate research in MBRL, in this paper we gather a wide collection of MBRL algorithms and propose over 18 benchmarking environments specially designed for MBRL. We benchmark these algorithms with unified problem settings, including noisy environments. Beyond cataloguing performance, we explore and unify the underlying algorithmic differences across MBRL algorithms. We characterize three key research challenges for future MBRL research: the dynamics bottleneck, the planning horizon dilemma, and the early-termination dilemma. Finally, to facilitate future research on MBRL, we open-source our benchmark 1 .

This paper performs an empirical comparison of a number of model-based RL (MBRL) algorithms over 18 benchmarking environments. The authors propose a set of common challenges for MBRL algorithms.
I appreciate the effort put into this evaluation and I do think it helps the community gain a better understanding of these types of algorithms. My main issue with the paper is that I don't find the evaluation thorough enough (for instance, no tabular environments are evaluated) and the writing still needs quite a bit of work. I encourage the authors to continue this line of work and improve on what they have for a future submission!
More detailed comments:
- Intro: "model-free algorithms ... high sample complexity limits largely their application to simulated domains." I'm not sure this is a fair criticism. Model-based are also mostly run on simulations, so sample efficiency is not necessarily the cause model-free are only run on simulations. Further, this statement kind of goes against your paper, since all your evaluations are on simulations!
- Intro: "2) Planning horizon dilemma: ... increasing planning horizon... can result in performance drops due to the curse of dimensionality..." and "similar performance gain[sic] are not yet observed in MBRL algorithms, which limits their effectiveness in complex environments" goes against the point you made earlier on in the intro about sample efficiency.
- Preliminaries: "In stochastic settings, it is common to represent the dynamics with a Gaussian distribution", this is only for continuous states. It would be nice if you could evaluate tabular environments as well.
- Sec 4.1: "we modify the reward funciton so that the gradient... exists..." which environments were modified and how did they have to be modified?
- Sec 4.1: You discuss early termination but have not defined what exactly you mean by it.
- Fig 1: 12 curves is still a lot and really hard to make much sense of. A lot of the colors are very similar.
- Sec 4.1: "it takes an impractically long time to train for 1 million time-steps for some of the MBRL algorithms" why?
- Table 1 is a *lot* of numbers and colors and is really hard to make much sense of. There are also so many acronyms on the LHS it's difficult to keep track.
- Table 2: What about memory usage?
- Sec 4.4: "Due to the limited exploration in baseline, the performance is sometimes increased after adding noise that encourages exploration." Why does this noise not help exploration in the baselines?
- Sec 4.5: "This points out that when learning models more data does not result in better performance." This seems like it's closely correlated with the particular form chosen for the model parameterization more than anything.
- Fig 3: y-axis says "Relative performance", relative to what?
- Sec 4.7: "Early termination...is a standard technique... to prevent the agent from visiting unpromising states or damaging states for real robots." I've never seen this used as a justification for this.
- Sec 4.7: "Table 1, indicates that early termination does in fact decrease the performance for MBRL algorithms", and then you say "We argue that to perform efficient learning in complex environments... early termination is almost necessary." Those two statements contradict each other.
Minor comments to improve writing:
- When using citations as nouns, use \citep so you get "success in areas including robotics (Lillicrap et al., 2015)" as opposed to "success in areas including robotics Lillicrap et al. (2015)" (the latter is what you have all throughout the paper).
- Sec 3.1: s/This class of algorithms learn policies/These class of algorithms learn policies/


Recent work has shown that deep generative models can assign higher likelihood to out-of-distribution data sets than to their training data (Nalisnick et al., 2019;Choi et al., 2019). We posit that this phenomenon is caused by a mismatch between the model's typical set and its areas of high probability density. In-distribution inputs should reside in the former but not necessarily in the latter, as previous work has presumed (Bishop, 1994). To determine whether or not inputs reside in the typical set, we propose a statistically principled, easy-to-implement test using the empirical distribution of model likelihoods. The test is model agnostic and widely applicable, only requiring that the likelihood can be computed or closely approximated. We report experiments showing that our procedure can successfully detect the out-of-distribution sets in several of the challenging cases reported by Nalisnick et al. (2019).

Thanks for the authors for your detailed reviews.
My major concerns about the proposed method are whether "the typicality set" could be faithfully applied in the small data regime. The authors point me to the interesting Figure 4, which shows that it basically achieves converged performance when $m = 50$ or smaller numbers for some problems. I think this experiment is a strong support for the proposed method.
However, I don't agree that the M=1 Gaussian case acts as a strong support for the method. As I said, for some other wired distribution, it is difficult to interpret what the M=1 Typicality set becomes.
The authors also clarify the difference between different baselines.
Overall, I will increase my score to "Weak Accept".
##########################
Recent works have shown that out-of-distribution samples can have higher likelihoods than in-distribution samples for some generative models. To explain this phenomenon and to tackle the problem for OOD detection, this paper adopts "typical sets" for identifying in-distribution samples. Specifically, a "typical set" is a set of examples whose expected log likelihood approximate the model's entropy. For a Gaussian distribution, the paper finds that a single point typical set locates exactly in the \sqrt{d} radius, which is usually favored over the high-likelihood origin. Then the paper uses the "typical set" to perform OOD for a batch of examples. Empirically they demonstrate competitive performance over MNIST and natural image tasks.
Typical set seems natural for out-of-distribution detection. An important property is that, if one draws a large number of independent samples from the distribution, it is very likely that these samples belong to the typical set (basically Theorem 2.1).  However,  for small n, this property doesn't hold anymore, which leaves here a questionmark whether "Typical set" can be used for OOD detection in small n regime. As the author argues, for Gaussian distribution when n=1 the typical locations are those \sqrt{d} radius points. But this doesn't justify the "Typical set". If the distribution is some non-Gaussian wired distribution, the typical locations doesn't seem to make sense at all.
Following the previous argument above, the Typical set method requires to perform OOD for a batch of examples. In contrast, the Annulus method can be directly applied to one single test example.
Empirically, the Typically set doesn't demonstrate obvious advantages compared to the baselines. For both MNIST and natural image tasks, it seems that all methods behave similarly. For comparing such big tables, I would recommend adding a column showing the average ranks among all methods. Beyond that, standard OOD tasks usually evaluate methods using AUROC and AUPR (Hendrycks and Gimpel, 2017). Is it possible to also include such metrics ?
Theorem 2.1 is confusing. It is beneficial to define what P is, and verbally state what the theorem conveys.


Adversarial perturbations cause a shift in the salient features of an image, which often results in misclassification. Previous work has suggested that these salient features could be used as a defense, arguing that with saliency tools we could successfully detect adversarial examples. While the idea itself is appealing, we show that prior work which used gradient-based saliency tools is ineffective as an adversarial defense -it fails to beat a simple baseline which uses the same model but with the saliency map removed. To remedy this, we demonstrate that learnt saliency models can capture the shifts in saliency due to adversarial perturbations, while also having a low computational cost. This allows saliency models to be used effectively as a real-time defense. Further, using the learnt saliency model, we propose a novel defense: a CNN that distinguishes between adversarial images and natural images using salient pixels as its input. On MNIST, CIFAR-10, and ASSIRA, our defense improves on using the saliency map alone, and can detect various adversarial attacks. Lastly, we show that even when trained on weak defenses, we can detect adversarial images generated by strong attacks such as C&W and DeepFool.

The paper studies methods for detecting adversarial examples using saliency maps. The authors propose using the method of Dabkowski and Gal (2017) to generate saliency maps and then train a classifier on these maps (or their dot product with the input image) to distinguish natural from adversarial examples. They perform experiments evaluating the white-box and black-box robustness of their detection scheme.
From a technical perspective, the contribution of the paper is rather incremental. The detection of adversarial examples by training a classifier on saliency maps has already been studied in prior work. The only modification proposed in this work is using an (existing) alternative method for producing the saliency maps and utilizing the dot product of maps with images.
From a conceptual perspective, the impact of detecting specific adversarial attacks is not clear. In a realistic setting, an adversary could use a very different attack or even utilize a different set of transformations (e.g. image rotations). Thus, in order to demonstrate the utility of their method in a black-box scenario, the authors would need to evaluate the defense in a variety of different scenarios. At the very least, they should consider generalization to difference attacks (e.g., train against FGSM and BIM, and test against DF).
Moreover, the robustness against white-box adversaries is not sufficiently studied. Firstly, the robustness of the non-adversarially trained detector is suspiciously high. There is little reason to expect that a composition of two neural networks (the saliency map methods and the classifier) would be non-trivially robust. The authors should consider alternative attacks perhaps using more iterations with a smaller step size. Secondly, after adversarial training, only the robustness against the same attack is considered. In order to argue about white-box robustness, the authors would need to evaluate against a variety of diverse adversaries.
Overall, the technical and conceptual contribution of this paper is insufficient for publication at ICLR, even ignoring the concerns about its experimental evaluation.


Partially Observable Markov Decision Processes (POMDPs) are popular and flexible models for real-world decision-making applications that demand the information from past observations to make optimal decisions. Standard reinforcement learning algorithms for solving Markov Decision Processes (MDP) tasks are not applicable, as they cannot infer the unobserved states. In this paper, we propose a novel algorithm for POMDPs, named sequential variational soft Q-learning networks (SVQNs), which formalizes the inference of hidden states and maximum entropy reinforcement learning (MERL) under a unified graphical model and optimizes the two modules jointly. We further design a deep recurrent neural network to reduce the computational complexity of the algorithm. Experimental results show that SVQNs can utilize past information to help decision making for efficient inference, and outperforms other baselines on several challenging tasks. Our ablation study shows that SVQNs have the generalization ability over time and are robust to the disturbance of the observation.

This paper proposes a new sequential model-free Q-learning methodology for POMDPs that relies on variational autoencoders to represent the hidden state. The approach is generic, well-motivated and has  clear applicability in the presence of partial observability. The idea is to create a joint model for optimizing the hidden-state inference and planning jointly. For that reason variational inference is used to optimize the ELBO objective in this particular setting. All this is combined with a recurrent architecture that makes the whole process feasible and efficient.
The work is novel and it comes with the theoretical derivation of a variational lower bound for POMDPs in general. This intuition is exploited to create a VAE based recurrent architecture. One motivation comes from maximal entropy reinforcement learning (MERL), but which has the ad hoc objective of maximizing the policy entropy. On the other hand SVQN optimizes both a variational approximation of the policy and that of the hidden state. Here the rest terms of the ELBO objective can be approximated generatively and some of them are conditioned on the previous state which calls for a recurrent architecture. The other parts are modeled by a VAE.
The paper also explores two different recurrent models in this context: GRU and LSTM are both evaluated. Besides the nice theoretical derivation the paper presents compelling evidence by comparing this approach to competing approaches on four games of the flickering ATARI benchmark and outperforming the baselines significantly. Also both the GRU and LSTM version outperforms the baseline methods on various tasks of the VIZDoom benchmark as well.
In general, I find that this well written paper presents a significant progress in modelling POMDPS in a model-free manner with nice theoretical justification and compelling empirical evidence.


Generative models for singing voice have been mostly concerned with the task of "singing voice synthesis," i.e., to produce singing voice waveforms given musical scores and text lyrics. In this work, we explore a novel yet challenging alternative: singing voice generation without pre-assigned scores and lyrics, in both training and inference time. In particular, we propose three either unconditioned or weakly conditioned singing voice generation schemes. We outline the associated challenges and propose a pipeline to tackle these new tasks. This involves the development of source separation and transcription models for data preparation, adversarial networks for audio generation, and customized metrics for evaluation.

This paper claims to be the first to tackle unconditional singing voice generation. It is noted that previous singing voice generation approaches leverage explicit pitch information (either of an accompaniment via a score or for the voice itself), and/or specified lyrics the voice should sing. The authors first create their own dataset of singing voice data with accompaniments, then use a GAN to generate singing voice waveforms in three different settings:
1) Free singer - only noise as input, completely unconditional singing sampling
2) Accompanied singer - Providing the accompaniment *waveform* (not symbolic data like a score - the model needs to learn how to transcribe to use this information) as a condition for the singing voice
3) Solo singer - The same setting as 1 but the model first generates an accompaniment then, from that, generates singing voice
Firstly, the authors have done a lot of work - first making their own data, then designing their tasks and evaluating them. The motivation is slightly lacking - it is not clear why we are interested in these three task settings i.e. what we will learn from a difference in their performance, and there is a lack of discussion about which setting makes for better singing voice generation. Also, there is no comparison with other methods: whilst score data is not available it could be estimated, then used for existing models, providing a nice baseline e.g. first a score is extracted with a state of the art AMT method, then a state of the art score to singing voice generation method could be used.
There are existing datasets of clean singing voice and accompaniment, for example MIR-1k (unfortunately I think iKala, another dataset, is now unavailable). It is true that this dataset is small in comparison to the training data the authors generate, but it will certainly be cleaner. I would have liked to see an evaluation performed on this data as opposed to another dataset which was the result of source separation (the authors generate a held out test set on Jazz from Jamendo, on which they perform singing voice separation).
I also had questions about the training data - there is very little information about it included other than it is in-house and covers diverse musical genres (page 6 under 4.1), a second set of 4.5 hours of solo piano, and a third set (?) of jazz singers. This was a bit confusing and could do with clarification. At minimum, I would like to know what genre we are restricting ourselves to - is everything Jazz? Are the accompaniments exclusively piano (it's alluded that the answer to this is no, but it's not clear to me)? Is there any difference between the training and test domain.
On page 6, second to last paragraph when discussing the validation set, I would like the sampling method to be specified - it makes a difference whether the same piece of music will be contained within both the training and validation split, or whether the source piece (from which the 10 second clips are extracted) are in separate splits <- I'd recommend that setting.
The data used to train the model will greatly affect my qualitative assessment of the provided audio samples so, without a clear statement on the training data used, I can't really assess this.
However, with respect to the provided audio samples, I'd first note that these are explicitly specified as randomly sampled, and not cherry picked, which is great, thank you. However, whilst I would admit that the domain is different, when the singing samples are compared with the piano generation unconditional samples of MelNet (https://audio-samples.github.io/#section-3), which I would argue is just as hard to make harmonically valid, they are not as harmonically consistent, even when an accompaniment has been provided. However, samples do sound like human voice, and the pitch is relatively good. The words are unintelligible, but this is explicitly specified as out of scope for this paper, and I agree that this is much harder to achieve.
As an aside, MelNet is not cited in this paper and, given the similarity and relevance, I think it probably should be - https://arxiv.org/abs/1906.01083. It was published this year however so it would be a little harsh to expect it to be there. I would invite the authors to rebut this claim if they think the methods are not comparable.
My main criticism is in relation to the evaluation. For Table 2, without a baseline or the raw data (which would have required no further effort) included in the MOS study, it's very difficult to judge success. If the authors think that comparison with raw data is unfair (as it is an embryonic task) they could include a model which has an unfair advantage from the literature - e.g. uses extracted score information.
For Table 1, I appreciate the effort that went into the design of 'Vocalness' and 'Matchness' which are 'Inception Score' type metrics leaning on other learned models to return scores. I would like to see discussion which explains the differences in scores for the different model settings (there is a short sentence at the bottom of page 7, but nothing on vocalness).
In summary - this is a hard problem and the authors are the first to tackle it. The different approaches to solve the problem are not well motivated. However, the models are detailed, and well explained. Code is even provided, but data for training is not. If the authors were able to compare with a baseline (like that I describe above), it would go a long way to convincing me that this was good work. As it stands, Tables 1 and 2, and the provided audio samples have no context, so I cant make a conclusion. If this issue and motivation was addressed I would likely vote to accept the paper.
Things to improve the paper that did not impact the score:
1. p2 "we hardly provide any labelled data" specify whether you do or not (I think it's entirely unsupervised since you extract chord progressions and pitch curves using learned models...)
2. p2 "...may suffer from the artifact" -> the artefacts
3. p2 "for the scenario addressed by the accompanied singer" a bit clumsy, may be worth naming your tasks 1, 2 and 3 such that you can easily refer to them
4. p2 "We investigate using conditional GAN ... to address this issue" - which issue do you mean? If it is the issue specified at the top of the paragraph, i.e. that there are many valid melodies for a given harmony (no single ground truth), I don't think using a GAN is a *solution* to this per se. It is a valid model to use, and the solution would be enough varied data (and evaluation to show you're covering your data space and haven't collapsed to a few modes)
5. p2 "the is no established ways" -> there are no established ways
6. p3 "Discriminators in GAN" -> in the GAN
7. p6 "piano playing audio on our own..." -> piano playing on its own (or even just rephrase the sentence - collect 4.5 hours of audio of solo piano)
8. p7 "We apply source separation to the audios divide them into ..." -> we apply source separation to the audio data then divide each track into 20 second...
9. p7 If your piano transcription model was worse than Hawthorne, why didn't you use it? It would have been fine to say you can't reproduce their model if it is not available, but instead you say that 'according to out observation [it] is strong enough' which comes across quite weakly.
10. p8 "in a quiet environment with proper microphone volume" -> headphone volume?
11. p8 "improv" - I think this sentence trailed off prematurely!


Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes while learning temporal patterns. The node embeddings, which become functions of time under the temporal setting, should capture both static node features and evolving topological structures. Moreover, node and topological features may exhibit temporal patterns that are informative for prediction, of which the temporal node embeddings should also be aware. We propose the temporal graph attention (TGAT) layer to effectively aggregate temporal-topological neighborhood features as well as learning time-feature interactions. For TGAT, we use the self-attention mechanism as the building block and develop the novel functional time encoding technique based on the classical Bochner's theorem from harmonic alaysis. By stacking TGAT layers, the network learns node embeddings as functions of time and can inductively infer embeddings for both new and observed nodes whenever the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to aggregate edge features. We evaluate our method with transductive and inductive tasks under temporal setting with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-theart baselines and prior temporal graph embedding approaches.

Summary: This paper addresses the problem of representation learning for temporal graphs. That is, graphs where the topology can evolve over time. The contribution is a temporal graph attention (TGAT) layer aims to exploit learned temporal dynamics of graph evolution in tasks such as node classification and link prediction. This TGAT layer can work in an inductive manner unlike much prior work which is restricted to the transduction setting. Specifically, a temporal-kernel is introduced to generate time-related features, and incorporated into the self-attention mechanism. The results on some standard and new graph-structured benchmarks show improved performance vs a variety of baselines in both transduction and inductive settings.
Pros:
+ Dynamic graphs are an important but challenging data structure for many problems. Improved methods in this area are welcome.
+ Dealing with the inductive setting is an important advantage.
+ Clear performance improvements on prior state of the art is visible in both transductive+inductive settings and node+edge related tasks.
Cons+Questions:
1. Technical significance: Some theory is presented to underpin the approach, but in practice it seems to involve concatenating or adding temporal kernels element-wise to the features already used by GAT. In terms of implementation the concatenation in Eq 6 seems to be the only major change to GAT. I’m not sure if this is a major advance.
2. Insight. The presented method apparently improves on prior work by learning something about temporal evolution and exploiting it in graph-prediction tasks. But it's currently rather black-box. It would be better if some insight could be extracted about *what* this actually learns. What kind of temporal trends exist in the data that this method has learned? And how are they exploited in by the prediction tasks?
3. Writing. The English is rather flaky throughout. One particular recurring frustration is the use of the term “architect” which seems wrong. Probably “architecture” is the correct alternative.
4. Clarity of explanation. The paper is rather hard to follow and ambiguous. A few specific things that are not explained so well:
4.1. Eq 1+2 is not a sufficiently clear and self-contained recap of prior work.
4.2. Symbol d_T used at the start of Sec 3.1 seems to be used without prior definition making it hard to connect to previous Eq1+2.
4.3 The claim made about alternative approaches (Pg4) “Reparameterization is only applicable to local-scale distribution family, which is not rich enough”. Seems both too vague and unjustified.
4.4 The relationship between $t_i$ and the neighbours of the target node in Eq. 6 is not very clear.