Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge has been addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces. Applying these techniques naively to the multi-agent setting results in agents exploring independently, without any coordination among themselves. We argue that learning in cooperative multi-agent settings can be accelerated and improved if agents coordinate with respect to what they have explored. In this paper we propose an approach for learning how to dynamically select between different types of intrinsic rewards which consider not just what an individual agent has explored, but all agents, such that the agents can coordinate their exploration and maximize extrinsic returns. Concretely, we formulate the approach as a hierarchical policy where a high-level controller selects among sets of policies trained on different types of intrinsic rewards and the low-level controllers learn the action policies of all agents under these specific rewards. We demonstrate the effectiveness of the proposed approach in a multi-agent gridworld domain with sparse rewards, and then show that our method scales up to more complex settings by evaluating on the VizDoom (Kempka et al., 2016) platform.

Overall I like the approach in the paper. It proposes a nice 2 pronged method for exploiting exploration via intrinsic rewards for multi-agent systems. The parts that a bit lacking with the current version of the paper in this are the evaluation tasks are few and a bit simple and I think there needs to be more discussion on the "coverage" of the intrinsic reward types. Are the ones proposed motivated by the tasks in the paper or are they sufficient for tasks in general?  Last using a more recent novelty metric could allow the method to work on more interesting/complex tasks.
More detailed feedback:
- It would be good to include more learning curves in the main text for the paper.
- The fact that applying intrinsic motivation to multi-agent simulations seems like a natural idea would be to convert the problem to a "single" agent problem to compare against the "normal" application of intrinsic rewards. This might be another baseline to consider for comparison.
- It says that all agents share the same replay buffer. Does this also imply that every agent is performing the same task there are just many agents? This does not make the problem very multi-agent with different goals. Would it affect the algorithm significantly to work on an environment where the agents have various types of goals?
- As is noted in the text, this method appears to work well in the centralized training scheme that many have adopted recently. However, It makes me wonder if there is a way to employ these exploration schemes in a non-centralized training form. The ability to ask other agents in the world about there preferences and novelty of states appears to be a strong assumption, especially in a multi-agent robotics problem.
- While the authors note that the intrinsic rewards used in this work are not comprehensive it would be good to note how comprehensive they are. Are there a few that were left out on purpose. Do the authours believe this set is sufficient. This statement makes it seem like the authors just tried a few options and found one that worked. It would be good to expand on this discussion more.
- More detail for Figure 1 would be helpful to understand the overall network design. While that figure it helpful maybe it would be good to include a version that goes into detail for the 2 agent environment. Then a more compressed n agent version can also be shown.
- The paper describes a policy selector that is a type of high-level policy for HRL. This design seems rather unique in that this part of the policy can optimizing for which intrinsic reward to toggle based on the extrinsic rewards observed. I like it. It is noted that entropy is important for this design. Can this be analyzed in an empirical way? Is this true for most environments/tasks?
- Task 2 seems a bit contrived. Is there another instance of this type of task elsewhere in another paper? It would be better to use more standard tasks if they are available.
- Before section 6.1 the paper is discussing rewards the are received. It would be good to more explicit about where these rewards are coming from. I think it is meant that these rewards are the extrinsic rewards but it does not say.
- As noted just before section 6.1 it seems for the collection of tasks 1-3 it is already obvious what types of intrinsic rewards should be used. It would be good to include more tasks where this decision is less obvious.
- Why are there "black holes" in the environment? Also if an agent steps into a black hole they are crushed never to be seen again. What you describe sounds more like a wormhole where one end is non-stationary... Also, can the agents detect the presence of a black hole in some way?
- It appears the novel metric is count based. While this can work in practice it seems a rather simple metric. Is it possible to use something more like ICM or RND that was referenced in the paper? Especially for the VizDoom environment?
- In table 2 where are some of the numbers bold? It would be good to include this information in the caption for the table.
- I am not sure if the discussion on the behaviours the intrinsic reward functions result in are very surprising. Maybe there is a more interesting behaviour that results from the combination of two intrinsic rewards?


Stemming from neuroscience, Spiking neural networks (SNNs), a brain-inspired neural network that is a versatile solution to fault-tolerant and energy efficient information processing pertains to the "event-driven" characteristic as the analogy of the behavior of biological neurons. However, they are inferior to artificial neural networks (ANNs) in real complicated tasks and only had it been achieved good results in rather simple applications. When ANNs usually being questioned about it expensive processing costs and lack of essential biological plausibility, the temporal characteristic of RNN-based architecture makes it suitable to incorporate SNN inside as imitating the transition of membrane potential through time, and a brain-inspired Recurrent Leaky Integrate-and-Fire (RLIF) model has been put forward to overcome a series of challenges, such as discrete binary output and dynamical trait. The experiment results show that our recurrent architecture has an ultra anti-interference ability and strictly follows the guideline of SNN that spike output through it is discrete. Furthermore, this architecture achieves a good result on neuromorphic datasets and can be extended to tasks like text summarization and video understanding.

####
A. Summarize what the paper claims to do/contribute. Be positive and generous.
####
The paper translates the Leaky Integrate and Fire model of neural computation via spike trains into a discrete-time RNN core similar to LSTM. The architecture would be readily amenable to the modern deep learning toolkit if not for the non-differentiability of the hard decision to spike or not. The hard decision is made by thresholding. The paper adopts a simple approximation of backpropagating a "gradient" of 1.0 through the operation if the threshold is within a neighbourhood [thresh - a, thresh + a], and otherwise 0.0, so the system can be trained by backpropagation.
The architecture is tested on a few "neuromorphic" video classification datasets including MNIST-DVS and CIFAR-DVS. Experiments are also run on a text summarization task.
####
B. Clearly state your decision (accept or reject) with one or two key reasons for this choice.
####
The reviewer thinks the paper should be rejected in its current state.
The proposed architecture is a straightforward change to a standard LSTM core. Thus it should be compared head-to-head to LSTM on standard datasets for these models (e.g. classic synthetic tasks, language modeling, speech recognition, machine translation, etc) with everything else held constant (hidden size, learning rate, sequence length, etc etc).
It also doesn't really carry over any of the benefits of Spiking Neural Nets even though it is inspired by Leaky Integrate and Fire because it operates in discrete time like a normal RNN, just with an extra binary output produced by spiking. It's unclear that a spiking inductive bias is actually useful, even though event-driven computation could in theory allow much less computation, the proposed method does not have that property.
So the paper doesn't really provide evidence to back up their claim that the proposed model combines the complimentary advantages of Deep Learning and Spiking Neural Nets.
####
C. Provide supporting arguments for the reasons for the decision.
While the proposed method is in-spirit inspired by the leaky integrate and fire model, it is operated/trained in discrete time which does not allow it to achieve the benefits of continuous time integrate-and-fire models which allow for less computation and time-discretization-invariance.
The conversion of the spiking model to the deep learning framework is rather crude, as the differentiable approximation to the non-differentiable threshold operation is biased and not well-motivated either empirically, intuitively, or theoretically (i.e. there are no comparisons to alternative choices).
There are new techniques for marrying continuous-time models and deep learning which seem more promising to investigate to this end (e.g. Neural ODE).
So in summary, the method doesn't have the computational benefits of a biologically plausible spiking algorithms and is not well-tested against competing deep learning methods, making it hard to verify the motivation of pushing toward a performant yet biologically plausible algorithm.
####
####
D. Provide additional feedback with the aim to improve the paper. Make it clear that these points are here to help, and not necessarily part of your decision assessment.
####
There are many grammatical and word-choice mistakes which make the paper hard to read.
Mainly, from a practical perspective, the paper would be much-improved by showing what benefit the spiking inductive bias confers over a standard LSTM on standard tasks in the deep learning community.
The method/landscape should be developed and studied in further detail until claims can be made about combining the strengths of spiking and deep-learning models.


We show how neural models can be used to realize piece-wise constant functions such as decision trees. Our approach builds on ReLU networks that are piece-wise linear and hence their associated gradients with respect to the inputs are locally constant. We formally establish the equivalence between the classes of locally constant networks and decision trees. Moreover, we highlight several advantageous properties of locally constant networks, including how they realize decision trees with parameter sharing across branching / leaves. Indeed, only M neurons suffice to implicitly model an oblique decision tree with 2 M leaf nodes. The neural representation also enables us to adopt many tools developed for deep networks (e.g., DropConnect (Wan et al., 2013)) while implicitly training decision trees. We demonstrate that our method outperforms alternative techniques for training oblique decision trees in the context of molecular property classification and regression tasks.

*Summary*
This paper leverages the piecewise linearity of predictions in ReLU neural networks to encode and learn piecewise constant predictors akin to oblique decision trees (trees with splits made on linear combinations of features instead of axis-aligned splits). The core observation is that the Jacobian of a ReLU network is piecewise constant w.r.t to the input. This Jacobian is chosen to encode the hard splits of a decision tree. The paper establishes an exact equivalence between decision trees and a slightly modified form of the locally constant networks (LCN). The LCN used for experiments is slightly relaxed to allow for training, including "annealing" from a the softplus nonlinearity to ReLU during training, adding one or more output layers to perform the final prediction, and training with connection dropout. Experiments show LCN models outperform existing methods for oblique decision trees, but ensembles are often matched or outperformed by random forests.
*Rating*
Perhaps the greatest attribute of decision trees is utter simplicity. (The second best attribute the out-of-the-box competitive accuracy of tree ensembles on a wide variety of problems.) An argument to be made for this paper is that it leverages the machinery of learning DNNs to learn more powerful, oblique tree-like models. The counterpoint is that despite the added complication, it's still often beaten by ensembles of CART trees. Overall, the idea is clever, the presentation could be improved slightly, and the experiments raise existential questions for this kind of work. My current rating is weak reject.
(1) It's difficult to know how LCNs should be compared to traditional decision trees, with accuracy, number of parameters, prediction speed, and training time/parallelism as viable components. The paper focuses almost exclusively on accuracy, while cross-validating over model sizes and other hyperparameters. This is a reasonable choice, though a discussion of model size and prediction speed would be welcome. I do have two significant questions about the experiments:
(2) It seems unfair that LCN has access to one or more hidden layers between the splits and the final output, denoted g_\phi. Would competing decision tree models improve with such a layer learned and appended to the final tree? Would LCN suffer from using a tabular representation like the others?
(3) Despite the assertion that these are datasets that necessitate tree-like predictors, the LLN method outperforms LCN and the trees on 4/5 datasets and is competitive with ensemble methods. While not explicitly stated, am I correct that LLN is essentially a traditional ReLU-network? If high accuracy is the goal, then why should I go to the trouble of training LCN when a traditional DNN is better. And if a tree is needed, then LCNs should be evaluated on more than just accuracy.
(4) LCNs seem to present a less bulky alternative to e.g. Deep Neural Decision Trees (https://arxiv.org/abs/1806.06988), but that work should be cited and discussed
(5) The proof sketch in Section 3.6 of the equivalence between the "standard architecture" and decision trees is difficult to understand and not convincing. (On second reading I noticed the subtle vector "\mathbf 0" indicating that all entries of "\grad_x a^i_1" are zero. Some further exposition and enumeration of steps would clear up confusion.)
(6) Overall the presentation is reasonable, other than the notes below. I did find myself searching back over the (dense) notation section and following sections looking for definitions of variables and terms used later. Consider better formatting (e.g. more definitions in standalone equations), strategic pruning of some material to make it less dense, and repeating some definitions in line (e.g. see below for "p7:... remind the reader").
*Notes*
(Spelling typos throughout; most are noted below)
p3: clarify in 3.1/3.3 that L is the number of outputs
p4: "interpred"
p5: "aother"
p5: Theorem 2 proof: note that the T/T' notation is capturing left/right splits
p5: "netwoworks"
p5: "Remark 5 is important for learning shallow...": should "shallow" be "narrow" instead?
p7: in the first paragraph, remind the reader of the definitions of õ^M and J_x ã^M
p7: "Here we provide a sketch of [the] proof"
p7: "unconstraint" should be "unconstrained"
p7: "...can construct a [sufficiently expressive] network g_\theta"
p7: "simlify"
p9: Table 2: instead of "2nd row", ..., use "1st section", ...; also consider noting which methods are introduced in this paper
p9: Figure 2: text is too small


Amharic language is an official language of the federal government of the Federal Democratic Republic of Ethiopia. Accordingly, there is a bulk of handwritten Amharic documents available in libraries, information centres, museums, and offices. Digitization of these documents enables to harness already available language technologies to local information needs and developments. Converting these documents will have a lot of advantages including (i) to preserve and transfer history of the country (ii) to save storage space (ii) proper handling of documents (iv) enhance retrieval of information through internet and other applications. Handwritten Amharic character recognition system becomes a challenging task due to inconsistency of a writer, variability in writing styles of different writers, relatively large number of characters of the script, high interclass similarity, structural complexity and degradation of documents due to different reasons. In order to recognize handwritten Amharic character a novel method based on deep neural networks is used which has recently shown exceptional performance in various pattern recognition and machine learning applications, but has not been endeavoured for Ethiopic script. The CNN model is trained and tested our database that contains 132,500 datasets of handwritten Amharic characters. Common machine learning methods usually apply a combination of feature extractor and trainable classifier. The use of CNN leads to significant improvements across different machinelearning classification algorithms. Our proposed CNN model is giving an accuracy of 91.83% on training data and 90.47% on validation data.

This paper considers the problem of character recognition applied to handwritten Amharic text. The authors collect a dataset of handwritten Amharic characters and apply a CNN model for character recognition. The task is contextualized within an extensive description of related work both in Amharic and other languages. The dataset is novel and would be of interest to the character recognition community, and I would encourage the authors to present it in its own right along with technical details about its collection. Unfortunately, due to methodological issues (detailed below), I do not think that the machine learning results in this paper are ready for publication at ICLR. The machine learning techniques applied are of limited novelty, however, the dataset is certainly a novel contribution.
Methodological issues:
The paper describes dividing the dataset in an 80-20 train-validation split. To the best of my understanding, there is no held out test set, and so we cannot know the performance of the model on unseen data. It appears that validation accuracy was taken into account when selecting hyperparameters, and so, validation accuracy also does not represent the model's performance on unseen data.
I would recommend that the authors introduce a test set into their dataset split, or designate the validation part of the dataset as a test part use cross validation for hyperparameter tuning.
The authors present the train and validation numbers on their dataset, but it is difficult to know the impact of the result without comparing to a baseline of some kind. It is challenging to compare to prior work since, according to the authors, prior work on Amharic character recognition has been focused on printed text.  However, a simple non-neural baseline would be illuminating.
Recommendations:
1. The authors should report results on an unseen test dataset, rather than train and validation sets.
2. The authors spend a lot of time motivating the use of deep learning in the introduction, and later on describing convolutional neural networks, ReLU, fully connected layers, etc. in great details. I believe that this space in the paper could be better utilized describing the things that are unique to the work presented.
3. It is really important to be able to see (a) the references and (b) the example characters, as it helps readers to situate the work and to understand the specific challenges addressed. I urge the authors to prioritize these technical details in submissions of future versions of this work.
4. Please consider separating out the description of related work into its own section. While it is useful to describe state of the art systems applied to related datasets, it is not necessary to go into great technical detail, especially if the methods applied are quite different from those attempted in this paper.
5. Based on the challenging nature of Amharic character recognition, it would be extremely helpful to see F1 numbers or a confusion matrix. While Figure 3 shows that different characters can be written similarly, it would be great to provide a quantitative measure of this phenomenon.
6. For a figure such as Figure 3, please provide an indication (e.g. the character number from 0-265, or a printed version of the character) of what the drawn character is supposed to look like, to help readers who cannot read Amharic script.
Questions about dataset
1. Please clarify what you mean by "the data collected are of two types". Does this refer to the train/validation split?
2. It would be great to know more about how many individuals were selected (and the choices made about their demographics) for writing the example characters, and whether there were any interesting variations observed in the dataset based on attributes highlighted in the paper (e.g. age range)
3. Please explain the meaning of 'Form A' and 'Form B' in Figure 2.
4. In the text, features such as a mark of palatalization are noted. Are these addressed in the data collection? Please give further details about how the dataset addresses the special features of the Amharic script.
5. At the top of page 6, it says that the data was labelled. Could you say more about the annotators (multiple?), and report inter annotator agreement? Or, did the writers produce each character based on a specific prompt, such that the labels are known to be correct?
6. Section 3 mentions data augmentation. Could you describe this in more detail?
7. Are the images in Figure 3 from your dataset? Please show some examples from the new dataset you have collected!


Domain adaptation focuses on the reuse of supervised learning models in a new context. Prominent applications can be found in robotics, image processing or web mining. In these areas, learning scenarios change by nature, but often remain related and motivate the reuse of existing supervised models. While the majority of symmetric and asymmetric domain adaptation algorithms utilize all available source and target domain data, we show that efficient domain adaptation requires only a substantially smaller subset from both domains. This makes it more suitable for real-world scenarios where target domain data is rare. The presented approach finds a target subspace representation for source and target data to address domain differences by orthogonal basis transfer. By employing a low-rank approximation, the approach remains low in computational time. The presented idea is evaluated in typical domain adaptation tasks with standard benchmark data.

In this work, the authors improve the work (Raab & Schleif, 2018) by (1) reducing the computational complexity, (2) neglecting the sample size requirement, and (3) achieving a low-rank projection through Nystrom approximation. More specifically, the feature dimensionality is reduced by using only s biggest eigenvalues and eigenvectors, and the sample size is coordinated through Nystrom approximation. Class-wise sampling is used for the source, and uniform sampling is used for the target. Experimental studies on three datasets have been done.
This paper should be rejected because (1) the paper lacks important latest references on domain adaptation, (2) the paper misuses the notations that makes the paper is not easy to follow, (3) the algorithm is not well justified either by theory or experiments, and (4) the presentation should be further polished. Here are some detailed comments:
(1)	A lot of recent deep domain adaptation methods are missing. These deep works achieve the state-of-the-art results on many transfer benchmark tasks. Without comparison with them (the paper does not mention any deep works), it is very unconvincing to conclude the paper makes new contributions to the transfer community.
(2)	In line 2 of page 5, the authors claim that BT assumes S_Z ~ S_X, which is not true. The key idea of BT is to construct new source data using target basis and source eigenvalues. Similarly, the authors make the same claim in section 4.1 for NBT, which is also not valid.
(3)	The notations in eq. (12) and (13) are very misleading. Eq. (12) follows the notations of the first line in page 5, but in the first line below eq. (12), why R_X \in R^{d \times s}, S_X^2 \in R^{s \times s}? I understand that X_s is the low dimensional X, then X_s = XR_s with R_s is the dimensionality reduction matrix whose size is d \times s (using biggest s eigenvectors is fine). With this, eq. (13) is incorrect as X should be L_X*S_X*R_X^T, but not L_s*S_s*R_s^T. Moreover, it is also unclear why X is decomposed into product of two matrices in this work, is there any benefit of doing so for transfer purpose?
(4)	In section 4.1, A_Z and A_X have exactly same form with X and Z, i.e., L_Z*S_Z*R_Z^T and L_X*S_X*R_X^T, please clarify. How eq. (14) X_s = \tide{L_X}*S_X come from? Is it the same as eq. (13)?
(5)	The title highlights low-rank, but it is not very clear how low-rank matters in the proposed method. I do not find contents stating the low-rank property of the proposed algorithm in the main technical sections.
(6)	Regarding section 4.2, what is the benefit of using class-wise sampling for the source? Have you tried to use uniform sampling for both the source and target domains?
(7)	The bound in eq.(16) is not very meaningful as s << n, m, d. Moreover, I am also not convinced by the claim this reduces the distribution differences, please give more theoretical justifications.
(8)	Regarding the experimental studies, why not use accuracy as many existing works do? The baselines are all subspace-based papers, and are out-of-the-date. The latest subspace papers, e.g., JGSA and MEDA, should be included. Moreover, deep methods are completely missing, which makes the empirical evaluation much less convincing. The improvements of NBT to BT are very marginal, 0.6.
(9)	Some typos and unclear points (please further polish the paper):
(a)	The last sentence of para 2, X and Z should be data, not features/
(b)	Para 3, it is unclear what are the implicit alignment and explicit alignment of domain distributions.
(c)	The first sentence in section 2, it should be homogeneous.
(d)	Page 5 above eq. (15), it should be S_Z ~ S_X.
(e)	Section 4.2 the second line, it should be “in the data matrix”
(f)	The second last line of page 6, it should be inequality (16).
(g)	Some references miss page information.


Lead bias is a common phenomenon in news summarization, where early parts of an article often contain the most salient information. While many algorithms exploit this fact in summary generation, it has a detrimental effect on teaching the model to discriminate and extract important information. We propose that the lead bias can be leveraged in a simple and effective way in our favor to pretrain abstractive news summarization models on large-scale unlabeled corpus: predicting the leading sentences using the rest of an article. Via careful data cleaning and filtering, our transformer-based pretrained model without any finetuning achieves remarkable results over various news summarization tasks. With further finetuning, our model outperforms many competitive baseline models. For example, the pretrained model without finetuning achieves state-of-the-art results on DUC-2003  and DUC-2004  datasets. The finetuned model obtains 3.2% higher ROUGE-1, 1.6% higher ROUGE-2 and 2.1% higher ROUGE-L scores than the best baseline model on XSum dataset. Human evaluations further show the effectiveness of our method.

This paper proposed an interesting idea on how we can leverage the lead bias in summarization datasets to pretrain abstractive news summarization models on large-scale unlabelled corpus in simple and effective way.
For pre-training, they collected three years of online news articles data. Then, they take the top 3 sentences of the article as summary and the rest of the article as input document. For better choosing such article-summary pairs, they employ effective data cleaning and filtering process. Overall, they collected 21.4M articles for the pretraining.

Overall, the pretrained model does decent on three summarization datasets without any fine-tuning. After fine-tuning the respective datasets, the gains seem significant. Especially on the XSum dataset, the improvements are remarkable.
I believe that the idea is interesting but the experiments are incomplete and more investigation is required to make this paper stronger. Therefore I suggest to reject this paper.
Arguments:
1) The important experiments that are missing in this paper are evaluating the proposed method on better human written summarization datasets -- DUC. The real world summarizations resemble more like the ones in the DUC dataset and it would be interesting to see if the transfer results of the pretrained model on the DUC datasets. The important question is to understand whether the pretrained model which took advantage of lead-bias could achieve good summaries on real summarization samples. This would also answer whether the pretrained just took advantage of the lead-bias issue of many large summarization datasets or does it really learn good summarization model.
2) This paper has good idea but mainly missing ablation studies. For example, how does the proposed model do compared with GPT-2 in the fine-tuning setting, and how do these two models perform on the DUC datasets.
3) During the dataset filtering/collection, a check on the quality of the filtering process by doing a small human study would have been a great addition. Also, instead of showing the output examples (which can go in the supplementary), human study comparing the quality of the pretrained model with fine-tuning and a baseline (can be from previous work) would have been better.
Other minor questions
1) “we only keep articles with 10-150 words in the top three sentences and 150-1200 words in the rest” -- is there any reason on fixing to these numbers? How did you make this decision ?
2) Even though the performance gains look visibly significant, I would suggest to report the statistical significance scores.


In this paper, we propose a framework that leverages semi-supervised models to improve unsupervised clustering performance. To leverage semi-supervised models, we first need to automatically generate labels, called pseudo-labels. We find that prior approaches for generating pseudo-labels hurt clustering performance because of their low accuracy. Instead, we use an ensemble of deep networks to construct a similarity graph, from which we extract high accuracy pseudo-labels. The approach of finding high quality pseudo-labels using ensembles and training the semi-supervised model is iterated, yielding continued improvement. We show that our approach outperforms state of the art clustering results for multiple image and text datasets. For example, we achieve 54.6% accuracy for CIFAR-10 and 43.9% for 20news, outperforming state of the art by 8-12% in absolute terms.

This paper proposes a method for unsupervised clustering. Similarly to others unsupervised learning (UL) papers like "Deep Clustering for Unsupervised Learning of Visual Features" by Caron et al., they propose an algorithm alternating between a labelling phase and a training phase. Though, it has interesting differences. For example, unlike the Caron et al. paper, not all the samples get assigned a labels but only the most confident ones. These samples are determined by the pruning of a graph whose edges are determined by the votes of an ensemble of clustering models. Then, these pseudo labels are used within a supervised loss which act as a regularizer for the retraining of the clustering models.
Novelties /contributions/good points:
* Votes from the clustering models to create a graph
* Using a graph to identify the most important samples for pseudo labelling
* Modification of the ladder network to be used as clustering algorithm
* Good amount of experiments and good results
Weaknesses:
* The whole experiment leading to Table 1 in page 2 is unclear for me. I have trouble understanding the experiment settings. Could you please rephrase it. About initial/ final clustering for example and the rest as well. The whole thing puzzles me whereas the experiments section at the end is much more clear.
* Lack of motivation about why using the Ladder method rather than another one. Other recent methods have better results in semi-supervised learning.
* Algorithm 1 seems quite ad-hoc. Do more principled algos exist to solve this problem ? You could write about it and at least explain why it would not be feasible here. The sentence "The intuition is that most of the neighbours of that node will also be connected with each other" is unmotivated: no empirical proof for this ?
* Related work section is too light. It is an important section and should really not be hidden or neglected.
* In the experiments, you could add the "Deep Clustering for Unsupervised Learning of Visual Features"  as baseline as well even if they use it for unsupervised learning as they do clustering as well.
* In the experiments, you use the features extracted from ResNet-50 but what about finetuning this network rather than adding something on top or even better starting from scratch. Because here CIFAR-10 benefits greatly from the ImageNet features. I know that you should reproduce the settings from other papers but it might be good to go a bit beyond. Especially, if the settings of previous papers are a bit faulty.
* Regarding, the impact of number of models in section D of the appendix, there is no saturation at 10 models. So how many models are necessary for saturation of the performance ?
* Minor point: several times, you write "psuedo".
Conclusion: the algorithm is novel and represents a nice contribution. Though, there are a lot of weaknesses that could be solved. So, I am putting "Weak accept" for the moment but it could change towards a negative rating depending on the rebuttal.


Published as a conference paper at ICLR 2020 REGIONAL BASED QUERY IN GRAPH ACTIVE LEARN- ING
Graph convolution networks (GCN) have emerged as a leading method to classify nodes and graphs. These GCN have been combined with active learning (AL) methods when a small chosen set of tagged examples can be used. Most AL-GCN use the sample class uncertainty as selection criteria, and not the graph. In contrast, representative sampling uses the graph, but not the prediction. We propose to combine the two and query nodes based on the uncertainty of the graph around them. We here propose two novel methods to select optimal nodes in AL-GCN that explicitly use the graph information to query for optimal nodes. The first method named regional uncertainty is an extension of the classical entropy measure, but instead of sampling nodes with high entropy, we propose to sample nodes surrounded by nodes of different classes, or nodes with high ambiguity. The second method called Adaptive Page-Rank is an extension of the page-rank algorithm, where nodes that have a low probability of being reached by random walks from tagged nodes are selected. We show that the latter is optimal when the fraction of tagged nodes is low, and when this fraction grows to one over the average degree, the regional uncertainty performs better than all existing methods. While we have tested these methods on graphs, such methods can be extended to any classification problem, where a distance can be defined between the input samples.

The paper studies active learning in graph representations. To decide which nodes in a graph to label during the active labeling, the paper proposes two approaches. First it argues that one should consider region-based measures rather than single nodes. Second, it proposes to adapt the page rank algorithm (APR) to determine which nodes are far away from labeled nodes.
Overall it remains unclear *how* to select the right strategy (before seeing the results for a dataset) i.e. which of the proposed approaches or variants should one select for a new dataset.
Strength:
-	One of the ideas of the paper, using region entropy over single node entropy makes sense to me.
-	The paper evaluates on 6 datasets and compares different variants as well to related work on 2 datasets.
Weaknesses:
1.	The paper contains several confusing and contradicting statements or claims which are not supported by the experimental results:
For example:
1.1.	“APR outperforms all other methods at low sampling fractions”.  This is supported neither in Table 1 nor Table 2, where APR is frequently not highest performing
1.2.	 “We have here shown that the accuracy of AL when uncertainty is computed regionally is much higher than when either local uncertainty or representative nodes are used”, this is not the case on CiteSeer in Table 1
1.2.1.	Also e.g. “Region Margin” is worse than random on 5% Email-EU; or “Region Margin AE” on 3% SubeljCora (Table 2) [It is unclear how to select with or without AE]
1.3.	“We outperform all existing methods in the Cora dataset, and get very similar results to the best accuracy obtained by Chang et al methods:”
1.3.1.	The difference to Cai et al. on Cora is very small (improvement by only 0.002), while on Citeseer the performance is comparatively bigger (Cai et al. is by 0.016 better)
1.3.2.	It should be “Cai et al”
2.	Clarity: I found the paper rather difficult to understand and follow:
Some specifics:
2.1.	The introduction could be more concisely discussing the motivation, the main idea of the paper, as well as contributions.
2.2.	Figure 1: according to the caption, APR should point to node 15, but in the figure it points to node 14. From the example it makes much more sense to label node 14 to me.
2.3.	Page 6 mentions twice the “ratio between APR and PR”, is this is this used/evaluated in the results?
2.4.	The decision what is bold and what is not is not consistent throughout the table 2.
2.5.	“Thus, hybrid techniques, combining several approaches, outperform using only one approach have been proposed.” It is not clear what this refers to and where the hybrid techniques have been evaluated.
Minor:
The paper contains many minor writing issues, e.g.
-	missing spaces, e.g. “distribution,and” (page 2)
-	Table 1: incomplete sentence: “∗∗ scores for smaller budget, since it was the”
-	Table 2: unclear: “accuracy without content”
The paper’s incorrect claims (weakness 1) are highly concerning and strongly suggest rejecting the paper. Furthermore, the clarity of the paper should be improved to follow the author arguments and make the paper easier to read.


We propose an inductive matrix completion model without using side information. By factorizing the (rating) matrix into the product of low-dimensional latent embeddings of rows (users) and columns (items), a majority of existing matrix completion methods are transductive, since the learned embeddings cannot generalize to unseen rows/columns or to new matrices. To make matrix completion inductive, most previous works use content (side information), such as user's age or movie's genre, to make predictions. However, high-quality content is not always available, and can be hard to extract. Under the extreme setting where not any side information is available other than the matrix to complete, can we still learn an inductive matrix completion model? In this paper, we propose an Inductive Graphbased Matrix Completion (IGMC) model to address this problem. IGMC trains a graph neural network (GNN) based purely on local subgraphs around (user, item) pairs generated from the rating matrix and maps these subgraphs to their corresponding ratings. It achieves highly competitive performance with state-of-the-art transductive baselines. In addition, since IGMC is inductive, it can generalize to users/items unseen during the training (given that their ratings exist), and can even transfer to new tasks. Our transfer learning experiments show that a model trained out of the MovieLens dataset can be directly used to predict Douban movie ratings and works surprisingly well. Our work demonstrates that: 1) it is possible to train inductive matrix completion models without using side information while achieving similar or better performances than state-of-the-art transductive methods; 2) local graph patterns around a (user, item) pair are effective predictors of the rating this user gives to the item; and 3) Long-range dependencies might not be necessary for modeling recommender systems.

This paper presents a method for inductive matrix completion that does not rely on side information to make predictions. The approach is as follows: select the sub-graph from an h-hop neighbourhood of any target user-item pair (u,v) and relabel it according to its distance from (u, v). The authors then treat rating prediction as graph classification problem given the selected subgraph and use a graph convolution architecture to make the prediction. They achieve strong empirical performance in both the transductive and inductive settings.
The paper presents an interesting approach to the inductive learning problem, with a number of interesting ideas and impressive empirical results. However, I have voted to reject this paper because I found the experimental section did a poor job of showing how the various ideas contribute to overall performance which made it difficult to evaluate the method beyond the absolute performance on benchmark datasets.
The key idea of the paper is interesting - samples h-hop enclosing subgraphs and label the resulting nodes by their hop-length and type (i.e. users at hop length h get a different label to items at length h).  In their experiments they find that a model with only a 1-hop neighbourhood to performs so well, which suggests that for recommender systems, supporting long chains of dependencies is unnecessary. This is a useful property of the data that the method depends on, because without it, h-hop neighbourhoods with larger h would quickly include the whole graph on most datasets (most real-world graphs have small diameter).
They process these subgraphs with relational graph convolutions, but make two modifications: first they build their final representations by concatenating all the message passing steps (eq. 2) and both user and item representations (eq. 3), and second they have adjacent rating regularization which constrains weights between adjacent ratings. Eq.2 in particular is unusual because it  amounts to a skip connection from every layer (somewhat analogous to a DenseNet [Huang et al. 2015]). The authors say both lead to improved performance - but don’t evaluate the approaches in the experimental section. What is the contribution of eq 2, 3 and 6 relative to more standard approaches (i.e. no skip connections, sum pooling, no regularization)?
Finally - I found the positioning of the work of the work misleading: from the abstract and related worked we're told that this is a "seemingly impossible problem" and prior work has relied on side information to deal with the inductive case. From this description, we might infer that this is the first approach that deals with the no-side information case. But the clear baseline is Hartford et al. 2018 which also doesn't use side information for inductive matrix completion; and yet it is not mentioned in either the introduction or the related work sections. This omission is unfortunate because both because it's misleading, and more importantly it misses the opportunity to discuss the relative merits of the approaches (beyond performance, where this paper provides a clear improvement). For example,
 - How do the two methods compare with respect to computational complexity? This approach seems more expensive because a different subgraph is selected for each prediction (i.e. 1 forward pass per test set point rather than a single forward pass to compute the full test set), but this isn't discussed.
 - Can you give any intuition about where the differences in performance are coming from?
------
Increased rating to Weak Accept following the rebuttal below.


Parameter-transfer is a well-known and versatile approach for meta-learning, with applications including few-shot learning, federated learning, and reinforcement learning. However, parameter-transfer algorithms often require sharing models that have been trained on the samples from specific tasks, thus leaving the task-owners susceptible to breaches of privacy. We conduct the first formal study of privacy in this setting and formalize the notion of task-global differential privacy as a practical relaxation of more commonly studied threat models. We then propose a new differentially private algorithm for gradient-based parameter transfer that not only satisfies this privacy requirement but also retains provable transfer learning guarantees in convex settings. Empirically, we apply our analysis to the problems of federated learning with personalization and few-shot classification, showing that allowing the relaxation to task-global privacy from the more commonly studied notion of local privacy leads to dramatically increased performance in recurrent neural language modeling and image classification.

This paper considers the problem of achieving formal privacy guarantees in the context of parameter-sharing meta learning. This is a problem that has been studied recently, although generally not under the exact record-level task-privacy model studied in this paper (what they call global task privacy). The problem is well-motivated: since in meta learning we want to leverage information from similar tasks to increate data efficiency, there may be privacy concerns for each of the task owners about both other task owners and the aggregator (meta-learner).
Their approach is conceptually simple; they use standard private stochastic gradient descent within each task to provide task level privacy. In combination with post-processing and composition guarantees this gives privacy for the overall mechanism. They are able to show theoretical guarantees via the standard accuracy guarantees of private SGD and no-regret
guarantees of OCO. They show a bound on the expected transfer risk:
O(V/sqrt(m) + 1/Tsqrt(m)) + o(1/sqrt(m)) which is close to the non-private bound from Denevi (2019).
They evaluate the empirical performance of these models via a transfer learning setting  where they are training a deep RNN for next word prediction on two large corpi, and the tasks correspond to individual articles. They show that the private variate of Reptile is competitive with the non-private variant in these settings.
A few comments
-	It is odd to just fix epsilon = 9.2 instead of showing a Pareto curve. Why this particular value?
-	Simpler (convex) experiments to illustrate the theoretical guarantees would improve the paper
Overall I like the motivation and the theory results are solid, if not a bit obvious. However, due to the lack of novelty in any of the applied techniques, and the fact that the experiments could be expanded, I recommend a weak accept.


Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation. Our contributions in this paper are twofold. First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a single aggregated explanation. The aggregation is more robust and aligns better with the neural network than any single explanation method. Secondly, we propose a new approach to evaluating explanation methods that circumvents the need for manual evaluation and is not reliant on the alignment of neural networks and humans decision processes.

The paper has two main messages: 1- Averaging over the explanation (saliency map in the case of image data) of different methods results in a smaller error than an expected error of a single explanation method. 2- Introducing a new saliency map evaluation method by seeking to mitigate the effect of high spatial correlation in image data through grouping pixels into coherent segments. The paper then reports experimental results of the methods introduced in the first message being superior to existing saliency map methods using the second message (and an additional saliency map evaluation method in the literature). They also seek to magnify the capability of the 2nd message's evaluation method by showing its better capability at distinguishing between a random explanation and an explanation method with a signal in it.
I vote for rejecting this paper for two main reasons: the contributions are not enough for this veneue, and the paper's introduced methods are not backed by convincing motivations. The first message of the paper is trivial and cannot be considered as a novel contribution: the ''proof'' is basically the error of the mean is smaller than the mean of the errors. Additionally, this could have been useful if the case was that there was a need for playing the safe-card: that is, all of the existing methods have equal saliency map error and averaging will decrease the risk. Not only authors do not provide any evidence but also both the experimental results of the paper itself (results in Table 2 and Fig 4 are disproving this assumption) and the existing literature disprove it. Even considering this assumption to be correct, the contribution is minimal to the field and benefits of averaging saliency maps have been known since the SmoothGrad paper. The second contribution is an extension of existing evaluation methods (e.g. SDC) where instead of removing (replacing by mean) individual pixels, the first segment the image and remove the segments. The method, apart from being very similar to what is already there in the literature, is not introduced in a well-motivated manner. The authors claim that their evaluation method is able to circumvent the problem with removing individual pixels (which is the removed information of one pixel is mitigated by the spatial correlations in the image and therefore will not result in a proportional loss of prediction power) by removing ''features'' instead. Their definition of a feature, though, are segments generated by simple segmentation methods. There is a long line of literature showing the incorrectness of this assumption; i.e. a group of coherent nearby pixels does not necessarily constitute a feature seen by the network and does not necessarily remove the mentioned problem of the high correlation of pixels. This method does not remove "the interdependency of inputs" for the saliency evalatuion metric. Even assuming the correctness of this assumption, the contribution over what already exists in the literature is not enough for this venue.
A few suggestions:
* The authors talk about a ''true explanation''. This concept needs to be discussed more clearly and extensively. What does it mean to be a true evaluation? It is also important to prove that the introduced evaluation metric of IROF would assign perfect score for a given true explanation.
* The mentioned problem of pixel correlations that IROF seeks to mitigate is also existing in other modalities of data and the authors do not talk about how IROF could potentially be extended.
* The qualitative results in the text and the appendix do not show an advantage. It would be more crips if the authors could run simple tests on human subjects following the methods in the previous literature.
* There are many many grammatical and spelling errors in the paper. The font size for Figures is very small and unreadable unless by zooming in.
* Many of the introduced heuristics are not backed by evidence or arguments. One example is normalizing individual saliency maps between 0-1 which can naturally be harmful; e.g. aggregating a noisy low-variance method with almost equal importance everywhere (plus additive noise) and a high-variance one which does a good job at distinguishing important pixels - AGG-VAR will not mitigate this issue.
One question:
The authors introduced aggregation as a method for a ''better explanation''. It has been known that another problem with saliency maps is robustness: one can generate adversarial examples against saliency maps. It would be an interesting question to see whether aggregation would improve robustness rather than how good the map itself is.


Graph Neural Networks (GNNs) are based on repeated aggregations of information across nodes' neighbors in a graph. However, because common neighbors are shared between different nodes, this leads to repeated and inefficient computations. We propose Hierarchically Aggregated computation Graphs (HAGs), a new GNN graph representation that explicitly avoids redundancy by managing intermediate aggregation results hierarchically, and eliminating repeated computations and unnecessary data transfers in GNN training and inference. We introduce an accurate cost function to quantitatively evaluate the runtime performance of different HAGs and use a novel search algorithm to find optimized HAGs. Experiments show that the HAG representation significantly outperforms the standard GNN graph representation by increasing the end-to-end training throughput by up to 2.8× and reducing the aggregations and data transfers in GNN training by up to 6.3× and 5.6×. Meanwhile, HAGs improve runtime performance by preserving GNN computation, and maintain the original model accuracy for arbitrary GNNs.

This paper aims to propose a speeding-up strategy to reduce the training time for existing GNN models by reducing the redundant neighbor pairs. The idea is simple and clear. The paper is well-written. However, major concerns are:
1. This strategy is only for equal contribution models (e.g., GCN, GraphSAGE), not for methods which consider distinct contribution weights for individual node neighbor (e.g. GAT). However, in my opinion, for one target node, different neighbors should be assigned with different contributions instead of equal contributions.
2. What kind of graphs can the proposed model be applied to? This paper seems to only consider unweighted undirected graphs. How about directed and weighted graphs? Even for an unweighted undirected graph, the symmetric information may also be redundant for further elimination. Then can HAG reduce this symmetric redundancy?
3. There is no effectiveness evaluation comparing the original GNN models with the versions with HAG. The authors claim that, with the HAG process, the efficiency could be improved without losing accuracy. But there are no experimental results verifying that the effectiveness of the HAG-versions which could obtain comparable performance with the original GNN models for some downstream applications (e.g., node classification).
--------------------------------------------------Update------------------------------------------------
Thanks very much for the authors' feedback. The revised version has clarified some of my concerns. However, the equal-contribution (in Comment 1) is still a big one that the authors should pay attention. I increase my score to 3.


We propose Flow Gaussian Mixture Model (FlowGMM), a general-purpose method for semi-supervised learning based on a simple and principled probabilistic framework. We approximate the joint distribution of the labeled and unlabeled data with a flexible mixture model implemented as a Gaussian mixture transformed by a normalizing flow. We train the model by maximizing the exact joint likelihood of the labeled and unlabeled data. We evaluate FlowGMM on a wide range of semi-supervised classification problems across different data types: AG-News and Yahoo Answers text data, MNIST, SVHN and CIFAR-10 image classification problems as well as tabular UCI datasets. FlowGMM achieves promising results on image classification problems and outperforms the competing methods on other types of data. FlowGMM learns an interpretable latent representation space and allows hyper-parameter free feature visualization at real time rates. Finally, we show that FlowGMM can be calibrated to produce meaningful uncertainty estimates for its predictions.

The paper describes a normalising flow with the prior distribution represented by a Gaussian mixture model (GMM). The method, FlowGMM, maps each class of the dataset to a Gaussian distribution in the latent space by optimising the joint likelihood of both labelled and unlabelled data, thus making the method useful for semi-supervised problems. Predictions are made using the maximum a posteriori estimate of the class label. To make the method robust to small perturbations of the inputs, the authors introduce a novel consistency regularisation term to the total loss function, which maximises the likelihood of predicting the same class after a perturbation.
The authors further examine the learnt latent space by considering two simple, synthetic datasets that can be easily visualised, showing that the latent space behaves in a way one would intuitively expect.
The method is evaluated on both tabular and image data, showing promising results in terms of accuracy (presumably, see below). As the model is found to be overconfident in its predictions, the authors introduce a calibration scheme and empirically verify that it improves the uncertainty estimates. Lastly, the authors introduce a feature visualisation scheme and use it to illustrate the effect of perturbing the activations of the invertible transformations.
I generally like the proposed method, which seems useful and intuitive. I am particularly happy with the discussion on uncertainty calibration, where the authors suggest an elegant addition to the model to increase the variance of the mixture components. I do, however, have significant concerns about the novelty of the paper as well as its structure and clarity, as detailed below. I do, therefore, not recommend it for acceptance.
The paper reads well, although I feel that it lacks some details and explanations. For example, in table 2, it is never mentioned what "FlowGMM Sup" refers to and if it is different from "FlowGMM Supervised". It is also not clear what "(All labels)" refers to - does it mean that labels were provided for the entire dataset or that the models were trained only on the small subset with labels? Or something else? Which performance metric is used in the tables? The accuracy, presumably, but this is never specifically stated. Similarly, the number of datapoints and ratio of labelled to unlabelled data for the synthetic datasets are not reported. They are not crucial to know but should be included for completeness.
While the first half of the paper is informative and well-structured, the second half appears a bit less so. From experimentally verifying that the method works, the paper goes on to discuss uncertainty calibration, examine the latent space representations, and visualising the effect of feature perturbations. While I greatly appreciate the focus on interpreting the trained model, I think it appears somewhat chaotic, as if the authors tried to squeeze too much into the paper. For example, the feature visualisation technique is quite neat, but it works for any flow and is not really used for anything in the paper. I would suggest saving it for a dedicated paper.
I am not convinced by the novelty of this paper. The authors list two contributions: 1) the model itself, 2) an empirical analysis with much focus on the interpretability of the model. While the model is, to my knowledge, indeed novel, the analysis is quite standard, and the interpretability even appears to be oversold. GMMs are nice and intuitive, but not novel in any way, yet the authors seem to be describing the properties of GMMs as specific to their method.
In particular, the authors go to great lengths to show that the latent space representations cluster around the means of the mixture components and that the decision boundary lies in low-density regions of the latent space. I do not see why these properties should be so surprising since the method directly optimises the likelihood of the data under the mixture distribution. That this is also empirically observed is, of course, reassuring, but these observations are better suited for the appendix, in particular given that the paper went over the recommended page limit.
I think that much of the claimed second contribution follows directly from the GMM aspect of the model. Instead of claiming the standard GMM properties as contributions, I think the proposed consistency loss term should be highlighted as a contribution on its own. I find it elegant, and I guess it would be particularly useful for NLP tasks where sentences can be phrased in different ways but still mean the same.
Instead of discussing the latent space, I would have preferred to see extra evaluations of the method, like convergence rates of both FlowGMM and FlowGMM-cons compared to the competing models. Furthermore, a major limitation of the model is that knowledge of the correct number of classes in the data - even in the unsupervised setting. The authors hint at extensions to mitigate this in the discussion (using a Chinese Restaurant Process GMM or by adding extra Gaussians to the mixture during training), but these should have been investigated in the current paper.
In conclusion, I think that the paper lacks novelty and that it spends far too much space on "trivial" properties of the model instead of addressing shortcomings, like the prior specification of the number of classes, which the authors even point out in the discussion.
Minor comments:
- p 4, bottom: "each with 1 hidden layers" -> "each with 1 hidden layer"
- p 5, middle: "FlowGMM is able to leveraged" -> "FlowGMM is able to leverage"
- p 5, bottom: "Table 5.1" -> "Table 1"


Although few-shot learning and one-class classification (OCC), i.e. learning a binary classifier with data from only one class, have been separately well studied, their intersection remains rather unexplored. Our work addresses the few-shot OCC problem and presents a meta-learning approach that requires only few data examples from only one class to adapt to unseen tasks. The proposed method builds upon the model-agnostic meta-learning (MAML) algorithm ) and learns a model initialization particularly suited for learning few-shot OCC tasks. This is done by explicitly optimizing for a parameter initialization which only requires a few gradient steps with one-class minibatches to yield a performance increase on class-balanced test data. We provide a theoretical analysis that explains why our approach works in the few-shot OCC scenario, while other meta-learning algorithms, including MAML, fail. Empirical results on six datasets from the image and time-series domains show that our method substantially outperforms both, classical OCC and few-shot classification approaches, and demonstrate the ability to quickly learn unseen tasks from only few normal class samples. Moreover, we successfully learn anomaly detectors for a real-world application on sensor readings recorded during industrial manufacturing of workpieces with a CNC milling machine using a few examples from the normal class.

This paper tackles an interesting problem, one-class classification or anomaly detection, using a meta-learning approach. The main contribution is to introduce a parameter such that the inner-loop of the meta-learning algorithm better reflects the imbalance which occurs during meta-testing. Results are shown comparing a few simple baselines to both MAML and the modified variant, on a few datasets such as image-based ones (MNIST, miniImageNet), a synthetic dataset, and a real-world time-series example from CNC milling machines.
Overall, the paper presents an interesting problem and awareness that meta-learning might be general enough to solve it well, but provides no real novelty in the approach. The datasets and comparison to other state of art methods (including both other anomaly detection methods and out of distribution methods) is lacking. I suggest the authors perform more rigorous experimentation and focus the paper to be a paper about an understudied problem with rigorous experiments/findings, or improve their method beond the small modification made. Due to these weaknesses, I vote for rejection at this time. Detailed comments are below.
Strengths

  - The problem is interesting and under-studied in the context of deep learning and transferable methods from similar ML problems (e.g. few-shot learning)
  - The method is simple and adapts a state of art in few-shot learning (meta-learning, and specifically MAML)
Weaknesses
  - While I enjoyed reading the paper since it tackles an under-explored problem, it is hard to justify publishing the method/approach at a top machine learning conference. Changing the balance in meta-learning is a relatively obvious modification that one would do to better reflect the problem; I don't think it results in general scientific/ML principles that can be used elsewhere.
  - The relationship to out-of-distribution detection (which some of the experiments, e.g. Multi-task MNIST and miniImagenet essentially test) is not discussed or compared to. How are anomalies defined and is it really different than just being out-of-distribution?
  - The datasets are limited. The MNIST dataset seems to choose a fixed two specific categories for meta-validation and meta-testing, as opposed to doing cross-validation. Results on just one meta-testing seems limited in this case with just one class. In terms of time-series, anomaly detection has been studied for a long time; is there a reason that the authors create a new synthetic dataset? For the milling example, how were anomalies provoked?
  - The baselines do not represent any state of art anomaly detection (e.g. density based, isolation forests, etc.) nor out of distribution detection; the latter especially would likely do extremely well for the simple image examples.
  - There is no analysis of what the difference is in representation (initialization) learning due to the differences between the OCC and FS setup. What are the characteristics of the improved initialization?
One minor comment not reflecting the decision:
  - Exposition: Define the one-class classification problem; it's not common so it would be good to define in the abstract, or mention anomaly detection which is a better-known term.


Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute clusters. As current approaches are limited by network bandwidth, we propose the use of communication compression in the decentralized training context. We show that CHOCO-SGD achieves linear speedup in the number of workers for arbitrary high compression ratios on general non-convex functions, and non-IID training data. We demonstrate the practical performance of the algorithm in two key scenarios: the training of deep learning models (i) over decentralized user devices, connected by a peer-to-peer network and (ii) in a datacenter.

The authors present an algorithm CHOCO-SGD to make use of communication compression in a decentralized setting. This is an interesting problem and the results are promising. Firstly they prove the convergence rate of the algorithm on non-convex smooth functions, which shows a nearly linear speedup.
Second, on the practical part, there have 3 main results:
	1. They compare CHOCO-SGD under various compression schemes with the baseline. The results show the algorithm generally outperforms the baseline.
	2. They implement it over a realistic peer-to-peer social network and show a great communication performance under such a network with limited bandwidth.
	3. In a datacenter setting, they compare the algorithm with all-reduce, which is a centralized communication method. The results show a strong training reduction for CHOCO-SGD.
Also, the paper is mostly nicely written.
However, there have several issues:
	1. In the introduction, they introduce their experiments with the order from "datacenter experiment" to "peer-to-peer experiment", which is different from the actual presenting order.
	2. In the description of Algorithm 1, the representation of initial values should be x{(-1/2)}_{i} instead of x{(0)}_{i} since line 2 using the term x^{t-1/2}_{i} with the range of t from 0 to T-1.
	3. About "datacenter setting" experiment, it seems not an apple to apple comparison between CHOCO-SGD and all-reduce method since CHOCO-SGD stands for the decentralized algorithm with compression and all-reduce stands for a centralized algorithm without compression. It's better to compare with at least one centralized algorithm with a compression scheme (like QSGD[1], signSGD[2], DGC[3]).
	4. Although they compare with the baseline (DCD and ECD) on Cifar-10 dataset,  it's worth to compare with them on the ImageNet since the result may be different under large-scale training.
Overall, this could be a great paper if fixing the issues above.
[1] D. Alistarh, D. Grubic, J. Z. Li, R. Tomioka, and M. Vojnovic. QSGD: Communication-efﬁcient SGD via gradient quantization and encoding. In Proc. Advances in Neural Information Processing Systems (NIPS), 2017.
[2] Bernstein J, Zhao J, Azizzadenesheli K, Anandkumar A. signSGD with majority vote is communication efficient and fault tolerant. arXiv. 2018 Oct 11.
[3] Lin Y, Han S, Mao H, Wang Y, Dally WJ. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887. 2017 Dec 5.


In geometric computer vision applications, multi-image feature matching gives more accurate and robust solutions compared to simple two-image matching. In this work, we formulate multi-image matching as a graph embedding problem, then use a Graph Neural Network to learn an appropriate embedding function for aligning image features. We use cycle consistency to train our network in an unsupervised fashion, since ground truth correspondence can be difficult or expensive to acquire. Geometric consistency losses are added to aid training, though unlike optimization based methods no geometric information is necessary at inference time. To the best of our knowledge, no other works have used graph neural networks for multi-image feature matching. Our experiments show that our method is competitive with other optimization based approaches.

The paper presents graph neural network approach for learning multi-view feature similarity.
The input data is local feature descriptors (SIFT), keypoint location, orientation and scale. The objective is to learn embedding such that features corresponding to the same 3d location will have similar embeddings, while different - far away. Such embedding distance matrix is called "feature universe" in the paper.
Instead of ground truth correspondence matrix, authors use smth called "noisy adjacency matrix", although it was not clear to me, what does it mean precisely. Loss is also augmented with epipolar distance reprojection error.
Training and testing is performed on Rome16k dataset.
Overall I find paper hard to follow and weak on experiment side. Comments and questions:
 1) There is no proper (or any) traditional baseline, which is: match SIFT features, apply cross-consistency and SNN ratio thresholds, run RANSAC, throw away all the inconsistent things. See COLMAP, Bundler or any other Visual SfM/MVS pipeline. Moreover, Hartman et.al. method is cited, but not compared to, because it needs "3d reconstruction" for supervision. Here is my second objection.

 2) Paper states that the method is unsupervised. Yet, it is based on known scene geometry (R and t) , which is typically obtained via 3d reconstruction pipeline. Once you do it,  you actually have ground truth correspondences, a lot of them. I don`t understand, why not use them.

 3) Experimental validation is also weird. Method was trained on Rome16k and tested on it as well. No other datasets were used besides Graffity sequence (see below). The metric is L1 norm on adjecency matrix instead of some.

 4) Regarding "feature universe". It is clear, that one cannot fit (or can?) the full n_features x n_3d_points matrix into GPU memory, as it should be super huge matrix. No details were given on how such problems are tackled.
 5) The paper, unfortunately, has a number of side claims, which are false and actually irrelevant to the paper core. Just to list a few: "deep learning has revolutionized how image features are computed (Yi et al., 2016)." It has not. SIFT is still quite the gold standard:  (To Learn or Not to Learn: Visual Localization from Essential Matrices, https://arxiv.org/abs/1908.01293, https://image-matching-workshop.github.io), besides that cited LIFT method was not used in practice because of being super slow.
 - "More fundamentally, deep neural networks need large amounts of labeled data to train. In the case of multi-image feature matching, one would need hand-labeled point correspondences between images, which can be difficult and expensive to obtain.".
  Nobody hand-labels correspondences. Instead, one uses runs 3d reconstruction pipeline with densification like COLMAP to obtain dense depth map, what where one can get multiview correspondences (e.g., MegaDepth: Learning Single-View Depth Prediction from Internet Photos https://arxiv.org/abs/1804.00607)
  - "Typically putative correspondences are matched probabilistically, meaning a feature in one image matches to many features
in another. The ambiguity in the matches could come from repeated structures in the scene, insuffi-
ciently informative low-level feature descriptors, or just an error in the matching algorithm. Filtering
out these noisy matches is our primary learning goal.". Typically, one-to-many matches are just thrown away (e.g from Bundler SfM paper "Modeling the World from Internet Photo Collections", Sec.4.2: "If a track contains more than one keypoint in the same image, it is deemed inconsistent.
We keep consistent tracks containing at least two keypoints for the next phase of the reconstruction procedure").
 6) Citations are sometimes weird. E.g. part of OxfordAffine dataset (http://www.robots.ox.ac.uk/~vgg/research/affine/)  is referred as Graffity without any reference at all to the dataset itself (???), but with references to two irrelevant works which are testing on it. Why benchmark sycle consistency of such a small dataset of a flat surfaces? Then one could use Fountain sequence, at least, which has some non-planar structures on it.

 Minor Comments:

  - Table 1 is hardly readable because of scientific notation used.
****
After rebuttal update.
I am increasing my score a bit, but still think that paper is not is the shape for publishing. Method itself might be good, but evaluation is still bad, and what is worse - authors haven`t even tried to improve it.
>At the end of the review period it will be revealed that we have 20 years of publication history in structure from motion and visual odometry.
This, unfortunately, does not help the current paper.
>We do in fact test on data not in our training set: we never trained on Graffiti.
Graffity is still only 6 (six!!!) images, which are related by homography. One could show results, at least for HSequences (118 * 6) images or add other dataset. Since, it is not training, it could be done quite fast.
> However, the focus of our paper is a novel feature representation, the formulation using a GNN framework, and the self-supervised losses. Overall, our paper is a novel approach to learning feature representations, a topic of great interest to the ICLR community, rather than a new structure from motion system that has to prove its superior performance over the current state of the art.
I completely agree with it and would be happy to accept such paper to any kind of workshop.


Saliency methods attempt to explain a deep net's decision by assigning a score to each feature/pixel in the input, often doing this credit-assignment via the gradient of the output with respect to input. Recently Adebayo et al. (2018) questioned the validity of many of these methods since they do not pass simple sanity checks, which test whether the scores shift/vanish when layers of the trained net are randomized, or when the net is retrained using random labels for inputs. We propose a simple fix to existing saliency methods that helps them pass sanity checks, which we call competition for pixels. This involves computing saliency maps for all possible labels in the classification task, and using a simple competition among them to identify and remove less relevant pixels from the map. Section 3 provides a theoretical justification to support our method and indicates when it can be applied, and its performance is empirically demonstrated on several popular methods.

I Summary
The paper directly answers two sanity checks for saliency maps proposed by Adebayo et al (2018): 1. randomizing the weights of a model to prove that the input's resulting saliency map is different from a trained model' saliency map.  2. randomizing the inputs' labels to make the same proof. The authors propose a "competitive version of saliency method" which uses the saliency scores of every pixel for each labels and zero out: positive scored pixels that would not be maximal for the predicted class and negative scored pixels that are not minimal for the worst predicted label.
Overall the method solves the aforementioned sanity checks, the authors claim it also generates more refined saliency maps.
II Comments
1. Content
The paper can be hard to read, due to multiple writing mistakes, abrupt phrasing, not well-articulated sentences. However, the idea is easy to understand and interesting but the contribution does not seem strong enough in its actual state.
My main concern is that the method seems to be designed only to answer the sanity checks: the resulting saliency maps can hardly be seen as more informative as other existing methods (eg figure 1). Quantitative measures (ROAR & KAR, Hooker et al. 2018) or surveys to show that the newly obtained saliency maps are more refined or help to best localize regions of interest would be a big bonus.
2. Writing
The paper comports numerous typos, those do not impact the score of the review except if the sentence is not understandable. Please see the following points as support to improve the clarity of the paper.
- Abstract last sentence: "Some theoretical justification is provided" -> "Some" is vague and makes your claim less credible -> "theoretical justifications are given in the last paragraph to support our method..."
- Intro
   paragraph 2 first sentence lack some words, l 2 product -> a product
   "See paper XX et al" -> "As in XX et al, we can see that" or "As stated in XX et al", "See" is too familiar, formalizing the phrasing gives more credibility to your work
- Related work
  "To give an example, does the map change a lot if we blank out or modify a portion of the image that humans find insignificant Kim et al. (2019)? " This is not very well articulated, "a lot" is vague and a little familiar, "significantly" could be used here. Moreover, the citation is a little abrupt "as we can see in XX" would work better
Little typo on etc..
"fare best" -> far better? The wording is still vague, it would help to add a quantitative measure
that's -> that is
- Section 3
"This figure highlights" -> which figure? (I think you just missed citing the fig here)
- Section 4
First sentence: Why is it a good idea? The claim is a little abrupt and could be detailed a little more
"destroy the saliency map" -> destroy is a very strong word
"These random variables are complicated." -> This statement seems a little out of place and abrupt
"some constants" -> "constants" (too vague otherwise as before)
- Subsection 4.1
"randomly sampling a few such methods" I believe there is a typo?
"See figure 3" is abrupt as a sentence itself "as you can see in figure 3 bla bla"
Figure 4 The image is small and hard to see on printed paper (same for the images in the appendix, they could be stacked over multiple lines instead of just one horizontal row)
Definition 2 punctuation at the end
- Section 5
"The available code for these maps is slow, and computing even gradient for all 1000 ImageNet labels can be rather slow." What is the aim of this sentence?
- subsection 5.3
lables -> labels
III Conclusion
The method itself is interesting, it would be interesting to see more qualitative results on the obtained saliency map itself: Does it produce more information? Is it more meaningful etc. Because as of now, it only seems to answer the two aforementioned sanity checks.  As for the writing, it is not always clear and can impede the understanding of the paper. I would be glad to change my review if those points are addressed.


Bitcoin is a virtual coinage system that enables users to trade virtually free of a central trusted authority. All transactions on the Bitcoin blockchain are publicly available for viewing, yet as Bitcoin is built mainly for security its original structure does not allow for direct analysis of address transactions. Existing analysis methods of the Bitcoin blockchain can be complicated, computationally expensive or inaccurate. We propose a computationally efficient model to analyze bitcoin blockchain addresses and allow for their use with existing machine learning algorithms. We compare our approach against Multi Level Sequence Learners (MLSLs), one of the best performing models on bitcoin address data.

In this paper, the authors propose a new method for generating vector embeddings. The studied problem is important and the topic is related to area of Bitcoin.
Empirical studies on some dataset (no description about the dataset) show some results on some evaluation metrics (no clear description about the metrics). For the methods 1-MLSL, 2-MLSL and 3-MLSL, it is seems that they are better than the proposed one (i.e., A2V) on some metrics, which is then inconsistent with the claims in the paper.
My major concern is that the paper is a bit too short and is lack of some necessary information, for example:
1 The authors are encouraged to provide sufficient background introduction, so that the reader can have a big picture of the problem and area.
2 The authors are encouraged to provide a detailed discussion and justification about the motivation, as well as the challenges and intuitions about the proposed method in the Introduction Section.
3 The authors are encouraged to show a detailed derivation about the technical details, in particular its difference compared with the major baseline, i.e., MLSL.
4 The authors are encouraged to follow the typical writing about the experiments in a paper, e.g., description about the datasets, evaluation metrics, baseline methods, parameter configurations and results analysis, etc.
Based on the above comments, I have to make a reject recommendation.


Human observers can learn to recognize new categories of objects from a handful of examples, yet doing so with machine perception remains an open challenge. We hypothesize that data-efficient recognition is enabled by representations which make the variability in natural signals more predictable, as suggested by recent perceptual evidence. We therefore revisit and improve Contrastive Predictive Coding, a recently-proposed unsupervised learning framework, and arrive at a representation which enables generalization from small amounts of labeled data. When provided with only 1% of ImageNet labels (i.e. 13 per class), this model retains a strong classification performance, 73% Top-5 accuracy, outperforming supervised networks by 28% (a 65% relative improvement) and state-of-the-art semisupervised methods by 14%. We also find this representation to serve as a useful substrate for object detection on the PASCAL-VOC 2007 dataset, approaching the performance of representations trained with a fully annotated ImageNet dataset.

The authors augment contrastive predictive coding (CPC), a recent representation learning technique organized around making local representations maximally useful for predicting other nearby representations, and evaluates their augmented architecture in several image classification problems. Although the modifications to CPC aren't particularly original, the authors show first that these yield a significant improvement in linear classification accuracy. They then use this improved model to obtain impressive performance in classification within semi-supervised and transfer learning settings, giving strong support for the use of such methods within image processing applications.
Pros:
Owing to its generality (CPC assumes only a weak spatial prior in the input data), and cheap computational cost relative to earlier generative approaches, CPC is already a promising unsupervised representation learning technique. The paper gives more evidence of this usefulness for image data, yielding leading performance on several different image classification benchmarks.
The authors also make the observation that linear separability, the standard benchmark for evaluating unsupervised representations, correlates poorly with efficient prediction in the presence of limited labeled data. This observation should be of interest in the broader community, and points to the need for more diverse metrics for unsupervised representations.
Cons:
The improvements given in the paper are quite useful within their stated domain (image data), but aren't directly applicable to other types of input data. Although the authors make a point of emphasizing the relevance of CPC for other problem domains, they don't currently provide any suggestions for how this current work could be generalized to handle these other cases. In this sense, I think it is a bit deceptive to refer to their model as "CPC v2", as the majority of their changes have no bearing on the intrinsic CPC algorithm itself.
I am sure that some of the methods used here could lead to improvements in the use of CPC for other types of data, but the authors currently don't provide any insight on this issue. In line with that, I think their work would be improved by some commentary on this, in particular by any concrete suggestions they have about how similar augmentations to CPC could be carried out in text, audio, and/or video data.
Verdict:
Owing to the reasons given above, I recommend acceptance.
Minor suggestions:
Please use a different color scheme for your figures that is still meaningful if the paper is printed in greyscale.


Efficient deep neural network (DNN) inference on mobile or embedded devices typically involves quantization of the network parameters and activations. In particular, mixed precision networks achieve better performance than networks with homogeneous bitwidth for the same size constraint. Since choosing the optimal bitwidths is not straight forward, training methods, which can learn them, are desirable. Differentiable quantization with straight-through gradients allows to learn the quantizer's parameters using gradient methods. We show that a suited parametrization of the quantizer is the key to achieve a stable training and a good final performance. Specifically, we propose to parametrize the quantizer with the step size and dynamic range. The bitwidth can then be inferred from them. Other parametrizations, which explicitly use the bitwidth, consistently perform worse. We confirm our findings with experiments on CIFAR-10 and ImageNet and we obtain mixed precision DNNs with learned quantization parameters, achieving state-of-the-art performance.

This paper considers the problem of training mixed-precision models.
Since quantization involves non-differentiable operations, this paper discusses how to use the straight-through estimator to estimate the gradients,  and how different parameterizations of the quantized DNN affect the optimization process. The authors conclude that using the parameterization wrt the stepsize d and quantization range q_max has the best performance.
In the discussion for the three parameterization choices in section 2.1.
It is not clear how the range for U2 is obtained. Given d an integer, the gradient wrt b is also bounded. In this case, why is case U3 better than U2? In Table 1, it is shown that U2 also has good performance for uniform quantization.
Indeed, the gradient of any of the three parameters (stepsize, bitwidth and quantization range) can be derived by using chain rule given the gradients of the other two. It is not clear to me why some of them can be unbounded while others do not. In addition, It is not clear to me why having different gradient scales is a big problem. Adaptive learning rate methods like Adam should be able to help deal with the different scale of the gradients for three parameters. Can the authors compare the three parameterizations using Adam and see if similar empirical results can still be observed.
At the end of Section 2.1, the authors said that "similar considerations can be made for power-of-two quantization".  However, from table 1, these three parameterizations indeed have quite different performances for uniform and power-of-two quantization.  E.g., for uniform quantization, U2 and U3 perform significantly better than U1, while for power-of-two quantization, U1 and U3 perform significantly better than U2. Can the authors elaborate more on the difference?
Is the proposed differential quantization method used for both weight and activation? If so, how are the gradients w.r.t. the weights propagated through the quantized activations?
---------- post-rebuttal comment -------------
I thank the authors for their detailed response. It has solved most of my concerns and I accordingly raised my score.
---------------------------------------------------------


Existing works in deep Multi-Agent Reinforcement Learning (MARL) mainly focus on coordinating cooperative agents to complete certain tasks jointly. However, in many cases of the real world, agents are self-interested such as employees in a company and clubs in a league. Therefore, the leader, i.e., the manager of the company or the league, needs to provide bonuses to followers for efficient coordination, which we call expensive coordination. The main difficulties of expensive coordination are that i) the leader has to consider the long-term effect and predict the followers' behaviors when assigning bonuses and ii) the complex interactions between followers make the training process hard to converge, especially when the leader's policy changes with time. In this work, we address this problem through an event-based deep RL approach. Our main contributions are threefold.
(1) We model the leader's decision-making process as a semi-Markov Decision Process and propose a novel multi-agent event-based policy gradient to learn the leader's long-term policy. (2) We exploit the leader-follower consistency scheme to design a follower-aware module and a follower-specific attention module to predict the followers' behaviors and make accurate response to their behaviors. (3) We propose an action abstraction-based policy gradient algorithm to reduce the followers' decision space and thus accelerate the training process of followers. Experiments in resource collections, navigation, and the predator-prey game reveal that our approach outperforms the state-of-the-art methods dramatically.

This paper proposes an interesting event-based policy gradient algorithm for single-leader multi-follower Stackelberg Markov Games (SMG). The authors claimed 3 contributions on their work: model the leader's decision-making process as a semi-MDP and proposed an event-based policy gradient algorithm for the leader to consider long-term effect; exploit a leader-follower consistency scheme with a follower-aware module and a follower-specific attention module to better predict the follower's behaviors; and propose an action-abstraction policy gradient algorithm in order to make SMG easier to converge.
The proposed model is interesting and the authors have done sufficient empirical studies to show the success of their method, as well as some necessary theoretical derivations. My major concern about this work is whether the task that this paper follows is widely studied in the reinforcement learning community, as the authors mentioned that only [1] can be used as baseline methods since the other methods can not be applied in their problem. But in general I think this is an interesting paper. Some detailed comments and suggestions:
1. This paper proposed 3 major contributions: event-based policy gradient, leader-follower consistency scheme and action abstraction policy gradient. The authors show the empirical performance comparison for the case with and without each of these techniques in the experiments. It can be seen that all of the 3 techniques help the proposed method perform better. In addition, the proposed method performs better compared to the state-of-the-art method (in [1]). Hence the authors have shown the success of their algorithm empirically. One thing that can be done to improve this paper is to show a joint comparison for the 2 ^ 3 = 8 cases of with / without any of the 3 techniques so that we can gain a better understanding about the effect of these 3 techniques.
2. The authors provide some necessary theoretical derivations to compute the policy gradients. But there are some typos in the equations. For example, the numerator of the equation of the case e_i^k\not\in A_T in I(e_i^k) at the bottom of page 4 is incorrect. In addition, the equation on the second last line of page 3 should be P_\gamma(s_{t+1}, \omega_t, a_t | s_t, \omega_{t-1}), not P_\gamma(s_{t+1}, \omega_t | s_t, \omega_{t-1}, a_t).  Please check the details and solve these typos.
3. It will be better if the authors can provide more theoretical analysis or quantitative explanation about the proposed method. For example, why a dense EBPG better than a sparse one? Why the equations in the attention mechanism appear in their form, not some similar forms (e.g. scale the values from the function A before putting them into the softmax function)? Intuitively the proposed techniques make sense, but it will be great if more insights are provided.
4. As I mentioned, I am concerned if the proposed problem is widely studied in the reinforcement learning community. Also, since the authors mentioned in the appendix that two of the tasks are original from [2], but we can not apply the method in [2] as a baseline method. It is possible to modify the method in [2] to fit with the setting and compare with them?
5. It will be better if the authors can rearrange the paper for a little bit, since there are too many important details (e.g. experiment settings, assumptions in the theorems) are in appendix.
Questions:
1. Can the authors try to modify the method in [2] and compare with it (details in my comment 4)?
Reference:
[1] Tianmin Shu and Yuandong Tian. M3RL: Mind-aware multi-agent management reinforcement learning. In ICLR, 2019.
[2] Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. In NeurIPS, pp. 6379–6390, 2017.