Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts. We propose AutoGrow to automate depth discovery in DNNs: starting from a shallow seed architecture, AutoGrow grows new layers if the growth improves the accuracy; otherwise, stops growing and thus discovers the depth. We propose robust growing and stopping policies to generalize to different network architectures and datasets. Our experiments show that by applying the same policy to different network architectures, AutoGrow can always discover near-optimal depth on various datasets of MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and ImageNet. For example, in terms of accuracy-computation trade-off, AutoGrow discovers a better depth combination in ResNets than human experts. Our AutoGrow is efficient. It discovers depth within similar time of training a single DNN.

This paper's contribution is a method for automatically growing the depth of a neural network during training. It compares several heuristics that may be used to successfully achieve this goal and identifies a set of choices that work well together on multiple datasets.
The paper focuses on CNNs that conform to a popular design pattern where the network is organized into a series of sub-networks, each consisting of a series of sub-modules (sometimes called blocks) operating at the same resolution. To be precise, the proposed method aims to learn the length of each series of sub-modules. A main contribution of the paper is the demonstration that it is not necessary to train a network until convergence before adding new sub-modules as proposed in past work. Instead, it is better to grow the network after training for a short while.
My current decision for this paper is a weak rejection due to the points below. However, I am open to revising my opinion if these points are addressed satisfactorily.
- The growing strategy identified in the paper as a superior alternative seems to be already known and used, at least in the speech recognition community. Seide et al. (2011) called it Discriminative Pre-training, and showed that it outperforms greedy layer-wise pretraining and DBN pre-training. Zeyer et al. (2017) reported that a similar method also enables the training of very deep LSTM networks which is otherwise notoriously hard. In general, the existence of prior work with the same ideas does not preclude acceptance, but the existence of this work needs to be clearly stated early on and the additional value of the current study sufficiently clarified.
- I find it strange that the final networks found by the proposed method usually have the same/similar number of sub-modules per sub-network (Tables 4,5,6) on multiple datasets. The only exceptions appear to be Basic4ResNet/CIFAR100 in Table 6 and about 50% of ImageNet results in Table 7. This regularity suggests that either A) the proposed algorithm prefers to set same number of sub-modules per sub-network due to its design, or B) datasets except ImageNet have an inherent shared property that produces this result. Since option A suggests a bias in the algorithm, this peculiarity of the results needs to be investigated or explained further.
- Figure 3 constitutes the main evidence that Autogrow finds approximately optimal depths as compared to manual searching, but it is not clear how the plot for baselines is obtained. For any given parameter budget, there are multiple baseline networks possible since the sub-networks can have different number of sub-modules (see previous point). This does not appear to be accounted for in Figure 3. Further, when dealing with CNNs, it would be more useful to have computation budget on the x-axis instead of the parameter budget. This would better account for the difference between increasing depth in an earlier sub-network vs. a later one.
- The reported results appear to be for single trials throughout the paper. This does not seem sufficient especially for results in Tables 2 and 3 where many differences are rather small, and so drawing conclusions from these tables would be unscientific.
References:
Seide, Frank, et al. "Feature engineering in context-dependent deep neural networks for conversational speech transcription." 2011 IEEE Workshop on Automatic Speech Recognition & Understanding. IEEE, 2011. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FeatureEngineeringInCD-DNN-ASRU2011-pub.pdf
Zeyer, Albert, et al. "A comprehensive study of deep bidirectional LSTM RNNs for acoustic modeling in speech recognition." 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2017. https://arxiv.org/abs/1606.06871
Update after rebuttal
-----------------------------
I'm sympathetic to the unfortunate situation that the authors are in, since the underlying growing strategy has already been covered by prior work. As I mentioned earlier, a sufficient rewrite of the paper can clearly state what has been done already so as not to take credit from the earlier authors. A revised version of the paper has not been uploaded; I suggest that the authors do so for the future.
I agree that the focus of this paper is learning the 'optimal' depth by using the growing strategy. But I am not convinced that the technique indeed finds optimal depths based on the regularity of the sub-network depths mentioned in my review. The rebuttal suggests reasons for the obtained regularity, but does not prove that these regular structures obtained are indeed optimal and not an artifact of the algorithm itself. The baselines are also using the same regular architectures, which distorts the overall picture because it is possible that a non-regular architecture provides a better trade-off.
While my rating doesn't change, I do think that the work is in an interesting direction. My final suggestions for the future are:
- Investigate where non-regular architectures (unequal sub-network depths) are in the trade-off between accuracy, flops and parameters.
- Investigate whether the proposed algorithm can be modified to easily find non-regular architectures if they can yield equally good performance as regular ones at similar or lower cost.


We introduce the notion of property signatures, a representation for programs and program specifications meant for consumption by machine learning algorithms. Given a function with input type τ in and output type τ out , a property is a function of type: (τ in , τ out ) → Bool that (informally) describes some simple property of the function under consideration. For instance, if τ in and τ out are both lists of the same type, one property might ask 'is the input list the same length as the output list?'. If we have a list of such properties, we can evaluate them all for our function to get a list of outputs that we will call the property signature. Crucially, we can 'guess' the property signature for a function given only a set of input/output pairs meant to specify that function. We discuss several potential applications of property signatures and show experimentally that they can be used to improve over a baseline synthesizer so that it emits twice as many programs in less than one-tenth of the time.

This paper proposed the concept of "property signatures" , which are learned to represent programs. The property signatures are essentially some key attributes that one may summarize from a given set of input-output pairs, which the target function has. Then a program can be generated by evaluating these property signatures vectors (which is simply a bag-of-word representation with only 0 or 1 as each element). Much discussions have been given to discuss why and how these properties may be useful and very little real experiments are conducted quantitatively compared with existing works. Although this paper is quite interesting, I think this paper is in its very early stage and there are a lot of serious concerns I have for using this approach to synthesize the real complex programs.
1) First of all, the notion of property signatures are easy to understand and is very natural. Just like human beings, when we write a program, we first think about the possible attributes of this program may have given a set of input-output pairs for both correctness and completeness. However, this is also the hard part of this idea. Potentially it could have an exponential number of possible properties as the program goes more complex and complex. It will quickly become computationally intractable problem.
2) When I read the middle of paper, I would eager to know how authors can effectively find a good set of properties of a target program from a given input-output pairs. However, when I eventually reached the Section 4, I was kindly disappointed since I did not see any effective and principle way to get them. All I saw are "randomly sample and generate". This may be Ok for a very simple program given a set of simple input-output pairs. But it is definitely not feasible for any complex function, not to mention project. I think this is the key for the proposed idea since how to construct a good set of property signatures is crucial to treat them as the inputs for any program synthesis task later.
3) There are very little baselines to compare against even though authors listed "substantial prior work on program synthesis". I understand the existing works may have their limitation in both what they can do and how well they can do. But it is still important to compare with directly on the same set of benchmarks. Otherwise, it is hard to be convincing that this approach is indeed superior compared to existing ones.


Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models. In this work, we propose a novel adversarial training algorithm, FreeLB, that promotes higher robustness and invariance in the embedding space, by adding adversarial perturbations to word embeddings and minimizing the resultant adversarial risk inside different regions around input samples. To validate the effectiveness of the proposed approach, we apply it to Transformerbased models for natural language understanding and commonsense reasoning tasks. Experiments on the GLUE benchmark show that when applied only to the finetuning stage, it is able to improve the overall test scores of BERT-base model from 78.3 to 79.4, and RoBERTa-large model from 88.5 to 88.8. In addition, the proposed approach achieves state-of-the-art single-model test accuracies of 85.44% and 67.75% on ARC-Easy and ARC-Challenge. Experiments on CommonsenseQA benchmark further demonstrate that FreeLB can be generalized and boost the performance of RoBERTa-large model on other tasks as well.

In this paper, the authors present a new adversarial training algorithm and apply it to the fintuning stage large scale language models BERT and RoBERTa. They find that with FreeLB applied to finetuning, both BERT and RoBERTa see small boosts in performance on GLUE, ARC, and CommonsenseQA. The gains they see on GLUE are quite small (0.3 on the GLUE test score for RoBERTa) but the gains are more substantial on ARC and CommonsenseQA. The paper also presents some ablation studies on the use of the same dropout mask across each ascent step of FreeLB, empirically seeing gains by using the same mask. They also present some analysis on robustness in the embedding space, showing that FreeLB leads to greater robustness than other adversarial training methods
This paper is clearly presented and the algorithm shows gains over other methods. I would recommend that the authors try testing their method on SuperGLUE because it's possible they're hitting ceiling issues with GLUE, suppressing any gains the algorithm may yield.
Questions,
-  In tables 4 and 5, why are only results on RTE, CoLA, and MRPC presented? If this is because there was not noticeable difference on the other GLUE datasets, please mention it in the text.
- I realize that this method is meant to increase robustness in the embedding space, but did you do any error analysis on the models? Did they make different types of errors than models fine-tuned the vanilla way?
Couple typos,
- Section 2.2, line 1: many -> much
- Section 4.2, GLUE paragraph: 88 -> 88.8


While variational dropout approaches have been shown to be effective for network sparsification, they are still suboptimal in the sense that they set the dropout rate for each neuron without consideration of the input data. With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss. To overcome this limitation, we propose adaptive variational dropout whose probabilities are drawn from sparsity-inducing beta-Bernoulli prior. It allows each neuron to be evolved either to be generic or specific for certain inputs, or dropped altogether. Such input-adaptive sparsityinducing dropout allows the resulting network to tolerate larger degree of sparsity without losing its expressive power by removing redundancies among features. We validate our dependent variational beta-Bernoulli dropout on multiple public datasets, on which it obtains significantly more compact networks than baseline methods, with consistent accuracy improvements over the base networks.

The paper proposes Variational Beta-Bernoulli Dropout to sparsify the parameters of the network. The authors also proposed an input-dependent dropout using input-dependent Beta-Bernoulli priors. The paper presents necessary theoretical details as well as the experimental comparison with the other dropout methods on MNIST and CIFAR 10/100 datasets.
The paper is well-written overall and easy to follow. The paper provides a thorough background on previous work; however the motivation for having an input-dependent dropout method is relatively weak.
I believe the paper presents main idea and necessary details thoroughly, though I have some confusions:
- In sections 4.1 and 4.2 (more specifically formula 7, 8 and 18), is there a global z (i.e. only one z), or there is a separate z for each layer? I implicitly inferred it's per layer but it's not clear from the paper.
- The paper explains well how the two proposed dropouts can be learned during training, but it's not clear about the inference, specially in variational dependent Beta-Bernoulli dropout. In section 5, experimental details, it is said: "In the testing phase, we prune the neurons/filters whose expected dropout mask probability E_q[pi_k] are smaller than a fixed threshold 10^-3." This is not clear to me what it means, and also the related footnote which says different thresholds were tried but the difference is insignificant! More elaboration is helpful.
- In the experimental results, the authors reported median and standard-deviation. Is there any reason the authors didn't report mean and standard-deviation instead? Also, it is not clear if xFLOP and Memory report the best, mean or median of 3 runs.
- In Table 1, what does column Neurons represent? Is it the model parameters at inference time? and if yes, is it the smallest number of parameters in 3 runs? In Table 2, how large the network is in terms of model parameters?
- In the experimental results, some models are missing that are mentioned in the paper as previous work which seem to be good to use for comparison, such as Sparse Variational Dropout (Molchanov 2017) and Structured Sparsity Learning (Wen et al 2016).
- I encourage the authors to expand more on analyzing the experimental results! In Table 1 and Table 2, there are 2 lines for BB and DBB, which is helpful to explain what each line represents. Looking at the results of all tables, and comparing different dropouts w.r.t :1) Error, 2) xFLOPs and 3) Memory, I cannot draw a clear conclusion from the results. For example, in Table 2, CIFAR-10, comparing first line of BB to SBP and second line to VIB, the difference is not significant. More analysis about the results can be helpful!


We present an approach based on multilingual sentence embeddings to automatically extract parallel sentences from the content of Wikipedia articles in 85 languages, including several dialects or low-resource languages. We do not limit the extraction process to alignments with English, but systematically consider all possible language pairs. In total, we are able to extract 135M parallel sentences for 1620 different language pairs, out of which only 34M are aligned with English. This corpus of parallel sentences is freely available.

The paper presents a multi-lingual multi-way pseudo-parallel text corpus automatically extracted from Wikipedia.
The authors use a variety of pre-existing techniques applied at large scale with substantial engineering effort to extract a large number of sentence pairs in 1620 language pairs from 85 languages.
In the proposed method 1) raw sentences are extracted from a Wikipedia dump, 2) LASER sentence embeddings and language IDs are computed for each sentence, 3) for each language pair candidate sentence pairs are extracted using a FAISS approximate K-nearest neighbor index on the cosine distance between sentence embeddings, 4) sentence similarity scores are computed between the candidate pairs using the "max margin" criterion of Artetxe & Schwenk, 2018 and finally 5) sentence pairs are selected according to a language-pair-agnostic threshold on the similarity scores.
The extraction method is symmetric w.r.t. language directions for each language pair.
Structural metadata of Wikipedia, such as cross-lingual document alignments, is deliberately not exploited (some discussion is provided but I would have preferred an empirical comparison of local vs global extraction).
The similarity threshold is determined by evaluating training corpora extracted at different thresholds on a machine translation task on De->En, De->Fr, Cs->De and Cs->Fr translation directions, evaluated on WMT newstest2014, and manually selecting the threshold based on BLEU scores. The paper also reports that combining the automatically extracted corpora with Europarl results in strong BLEU improvements over training only on Europarl. BLEU scores on TED test sets obtained using only the automatically extracted corpus are also reported. The corpus has been released.
Overall the methodology presented in the paper is strong and the corpus is likely going to become a valuable tool to build machine translation systems and other multi-lingual applications. However, I am concerned that ICLR 2020 may not be the appropriate venue for this paper, as in my understanding dataset release papers are not explicitly solicited in the Call for Papers https://iclr.cc/Conferences/2020/CallForPapers . The corpus generation method is based on existing techniques, and to the extent that the engineering effort is innovative, it might not necessarily transfer well to data sources other than Wikipedia, thus limiting its broad scientific value. Therefore I suggest to submit the paper to a different venue.


We present a Deep Neural Network with Spike Assisted Feature Extraction (SAFE-DNN) to improve robustness of classification under stochastic perturbation of inputs. The proposed network augments a DNN with unsupervised learning of lowlevel features using spiking neuron network (SNN) with Spike-Time-DependentPlasticity (STDP). The complete network learns to ignore local perturbation while performing global feature detection and classification. The experimental results on CIFAR-10 and ImageNet subset demonstrate improved noise robustness for multiple DNN architectures without sacrificing accuracy on clean images.

The paper develops a method to augment deep neural networks with Spike-Time-Dependent-Plasticity (STDP) aiming at improving noise robustness of the network learned features. In the hybrid network, learned feature is the concatenation of traditionally supervised-ly learned features and those from an auxiliary module trained locally and unsupervised-ly by STDP. The new network demonstrates improved noise robustness via improved classification accuracy on Cifar10 and ImageNet subset when input data have noise, on different network architectures.
The paper, however, fails to address the many works in the literature about adversarial perturbations ('attack') and adversarial training ('defense'), starting by (Szegedy et al., 2013). The different types of attacks affect the efficiency of defense due to the game-theoretical nature of the adversarial perturbation problem. If the attack is blind to the classification model, e.g., Gaussian attack by adding Gaussian noise, then image restoration techniques like denoising could provide an effective 'defense'. Thus model-specific attacks are of more application interest than model-blind ones. The current manuscript did not address the specific noise type being used to perturb the image. It is unlikely that the local learning techniques proposed in the paper can work on many kinds of perturbations especially the 'attacks' which is model specific.
The proposed methodology is a feature concatenation of local (low-level) features of image data and deep features. Given the current state of the manuscript, the level of methodological novelty and the scope of input perturbations that can be made robust against both appear to be limited.
References:
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.


Advancements in deep generative models have made it possible to synthesize images, videos and audio signals that are hard to distinguish from natural signals, creating opportunities for potential abuse of these capabilities. This motivates the problem of tracking the provenance of signals, i.e., being able to determine the original source of a signal. Watermarking the signal at the time of signal creation is a potential solution, but current techniques are brittle and watermark detection mechanisms can easily be bypassed by doing some post-processing (cropping images, shifting pitch in the audio etc.). In this paper, we introduce ReSWAT (Resilient Signal Watermarking via Adversarial Training), a framework for learning transformation-resilient watermark detectors that are able to detect a watermark even after a signal has been through several post-processing transformations. Our detection method can be applied to domains with continuous data representations such as images, videos or sound signals. Experiments on watermarking image and audio signals show that our method can reliably detect the provenance of a synthetic signal, even if the signal has been through several post-processing transformations, and improve upon related work in this setting. Furthermore, we show that for specific kinds of transformations (perturbations bounded in the 2 norm), we can even get formal guarantees on the ability of our model to detect the watermark. We provide qualitative examples of watermarked image and audio samples in the anonymous code submission link.

The paper proposes a watermark design algorithm that is based on an adversarial training paradigm where a watermark detector and a watermark generator are jointly trained to maximize the likelihood of watermark detection while bounding the manipulation of individual pixels.
I have several concerns about the clarity of the paper, its applicability to the motivating problem, and the completeness of the results that motivate my "reject" recommendation.
The description does not make it clear how the detector and watermark mechanism are jointly obtained in (1) - intuitively, a detector should know the type of watermark to be observed in order to pose a feasible detection model. The discussion focuses on "fixed detector" and "fixed watermark" cases only, but it is not clear how an iterative optimization would be initialized, and no discussion of how to set up the problem appears in the manuscript.
The motivation for the paper is also unclear. The authors discuss the need to be able to detect deep fakes, which is a relevant and meritorious problem. However, it seems that the application of the proposed approach in this setting would require deep fake creators to add watermarks to their creations, which would be a naive assumption. Watermarks are used to demonstrate provenance, but deep fake creators would not want to have such provenance verified.
The watermark transferability premise seems to rely on the random initialization of the underlying deep learning watermark detector. One could posit that if a third party is interested in creating false positives, they could train the deep learning network using a database of watermarked and non-watermarked images. Another concern here is that the formulation provided in (3) would potentially lead to low transferability for the specific classifiers provided there, but one does not know which classifier an adversary would construct - or why an adversary would construct an adversary that does not have knowledge of the specific watermark being used, or of samples of the watermark. It would have been interesting to see if it is possible to construct a good watermark detector this way (given enough training data being available). It would also be good to have some intuition as to why this robustness formulation also improves "certified robustness".
Finally, the performance comparison only considers a 10-year old watermarking algorithm; some discussion as to why this is sufficient should have been included.
Minor comments
The norm infinity subindex is missing from the minimization conditions in (1).
Figure 3 caption: watermark's -> watermarks
It is not clear why no numerical results were given for Section 4.2.4.


Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing nearoptimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience rollouts for adaptation. To achieve single episode transfer in a family of environments with related dynamics, we propose a general algorithm that optimizes a probe and an inference model to rapidly estimate underlying latent variables of test dynamics, which are then immediately used as input to a universal control policy. This modular approach enables integration of state-of-the-art algorithms for variational inference or RL. Moreover, our approach does not require access to rewards at test time, allowing it to perform in settings where existing adaptive approaches cannot. In diverse experimental domains with a single episode test constraint, our method significantly outperforms existing adaptive approaches and shows favorable performance against baselines for robust transfer.

Summary
This paper addresses the problem of transfer in RL. After an agent is given an opportunity to train from a distribution of environments, we want an agent to perform well on the test environment. This paper specifically focuses on the setting where the state space, action space, reward space, and discount factor are the same across all environments, while the transition dynamics may differ. An environment's transition dynamics is assumed to depend on a hidden parameter that is not observed by the agent, in contrast to some previous work which assumes observability.
The idea of the main algorithm is as follows. During the training phase, a series of environments is presented. The agent is aware of the demarcations between the environments, but not of the identity (i.e., hidden parameter) of the environments themselves. A "probing" policy is run for a specified number of time steps. From the trajectory generated, a VAE is used to estimate the hidden parameter governing the transition dynamics. For the remaining time in the environment instance, a master policy, conditioned on the estimated parameter, is trained on the reward (any usual RL algorithm suffices; DDQN is used in the experiments).
The novelties of the approach are the problem setting (no access to optimal policies, immediate deployment at test-time), the joint training of a probing policy and VAE model, and a universal policy conditioned on the estimated hidden parameters. The experiments were conducted on domains that were, individually, both continuous and discontinuous with respect to the hidden parameters. The paper concludes that the proposed method improves upon baselines in terms of speed of reward attainment and computation time. Ablations were run for some of the components of the proposed algorithm.

Decision: Weak Reject

The two key reasons for the Weak Reject were the following.
1. Statistically insignificant results (3 runs)
2. Questions about baselines used (see question 6 below)
The proposed algorithm is promising and attempts to directly address limitations in previous literature. The paper thoroughly contextualizes and motivates the current approach in light of previous work in the literature and discusses possible advantages: simplicity and generality; the ability to deploy immediately at test-time instead of having to train another policy (from learning a universal policy); assuming that the hidden parameters are unobservable, which is more realistic; not assuming access to optimal value functions/policies; the algorithm is model-free, so one can avoid the computational and memory overhead (when compared to BNN) of learning a model and avoid compounding model error. I also appreciated the thorough discussion of the key algorithmic choices and their trade-offs in section 3. The presentation of ideas is clear throughout.

However, weak experiments keep this paper at a weak reject. The paper claims that the proposed algorithm "significantly outperforms" the baselines (abstract); however, more runs are needed to substantiate these claims. Only 3 runs were made for the test episode, which is not enough to be able to make empirical claims with non-trivial confidence (see https://arxiv.org/abs/1709.06560).  As well, since baseline methods are adapted from the original papers, more thorough hyperparameter sweeps should be performed. I wasn’t sure what “coarse coordinate search” was (pg. 6), but in general it would be good to write down the hyperparameters swept over.
Some additional questions about the experiments:
1. Did you try oracle with lower dimensional hidden parameter embedding? (understanding more why oracle does possibly worse in HIV and Acrobot)
2. There is the claim on page 2 that the proposed algorithm is "orders of magnitude faster than best model-based method". What is the best model-free method?
3. Is the universal policy of SEPT optimized during test time?
4. Did you manage to replicate the results of DPT in Yao (2018)? Were architectures the same?
5. What is the significance of not requiring rewards at test-time?
6. Why were BNN, MAML, and EPOpt the baselines chosen? What about the other works mentioned in the related work section, like Tirinzoni (2018) or Paul (2019)? Did you try using an LSTM as a baseline to directly learn the policy? One could imagine treating the existence of a hidden parameter as inducing a partial observability problem.
7. What is a "coarse coordinate search" over hyperparameters? What hyperparameters were tried?
8. Did you try regular DQN? DDQN w/o prioritized replay?
9. Why were the cumulative reward curves not as significant for Acrobot and HIV as compared to 2D navigation?
10. Why different probe length settings for 2D navigation vs. Acrobot and HIV? How did these hyperparameters affect the time to solve?
11. Why do 2D-room and Acrobot have terminal rewards of 1000 and 10?
I would be willing to raise the score if the key experimental problems I noted were addressed.

Minor comments/questions that did not impact the score
1. Should include the table of computational time in the main body if possible since computational efficiency is one of the core claims
2. Move DDQN comment in 3.2 to the experiments section
3. How does the probing policy approach relate to work in active perception? (https://link.springer.com/article/10.1007/s10514-017-9666-5)
EDIT: After the comments below which addressed my concerns, I have raised the score for this paper from a weak reject to an accept.


Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many vision tasks especially when dealing with multiscale inputs. We study, in this paper, a scaling-translationequivariant (ST -equivariant) CNN with joint convolutions across the space and the scaling group, which is shown to be both sufficient and necessary to achieve STequivariant representations. To reduce the model complexity and computational burden, we decompose the convolutional filters under two pre-fixed separable bases and truncate the expansion to low-frequency components. A further benefit of the truncated filter expansion is the improved deformation robustness of the equivariant representation. Numerical experiments demonstrate that the proposed scaling-translation-equivariant networks with decomposed convolutional filters (ScDCFNet) achieves significantly improved performance in multiscale image classification and better interpretability than regular CNNs at a reduced model size.

*Paper summary*
The authors propose a CNN architecture, that is theoretically equivariant to isotropic scalings and translations. For this they add an extra scale-dimension to activation tensors, along with the existing two spatial dimensions. In practice they implement this with scale-steerable filters, which are discretised and truncated in both the spatial and scale-dimensions. They also provide a deformation robustness analysis.
*Paper decision*
Thank you for writing a very interesting paper indeed. I have to admit I am somewhat on the fence about this paper. I think it contains many nice ideas, but the experimental section is somewhat lacking in terms of comparisons or insights which I can gain, and the theory has some missing elements too (which I shall discuss below). For that reason, I am recommending a weak reject, but would very easily upgrade this if the authors provide strong rebuttal to my comments below.
*Supporting arguments*
-Experiments: The experiments are quite light, although I must admit that many other works in the area of equivariance are also light on experimentation and if there is enough theory, that is not such a great issue. The main issues I have are 1) the choice of experiments, 2) the comparisons against an insufficient number of baselines, and 3) the ablation studies.
1) The choice of experiments: I think looking at scaled-MNIST is not particularly useful as an experiment nowadays, unless used as a toy experiment. A larger dataset with real-life scale variations would have been better. Furthermore, I’m not sure what the image reconstruction task is supposed to tell the reader, that the ScDCFNet is able to generalise to new scales?
2) There is a lot of concurrent work on multiscale architecture. To name a few:
Multigrid Neural Architectures, Ke et al., 2017
Feature Pyramid Networks for Object Detection, Lin et al., 2017
Multi-Scale Dense Networks for Resource Efficient Image Classification, Huang et al., 2018
Deep Scale-spaces, Worrall and Welling, 2019
I believe these works should at least be cited, but ideally compared against.
3) I would have liked to have seen some numerical results for the verification of scale equivariance. In the first layer the equivariance improbably going to be close to perfect because there is no truncation of the scale-dimension in the network, but after this layer I predict the equivariance error increases due to truncations effects. This would be a similar effect to other works, such as “Deep Scale-spaces” (Worrall and Welling, 2019).
-Theory: I think the theory is very interesting and a meaningful contribution in its own right. The authors treat scale-translation in continuous space as a group action on signals. This motivates the convolution presented in Theorem 1. This is a group convolution as per Cohen and Welling, (2016) modified to continuous space for a non-compact group. (Actually, this should be mentioned in the text as a matter of good scholarship). This is nice, since the group convolution has not been used in for a regular representation of a non-compact group (other than translation) as far as I can tell. What is perhaps not clear for me is how the theory breaks down in practice, since the implementation requires discretisation in space AND scale, which is not discussed much and furthermore, filters are restricted in spatial AND scale dimensions, leading to truncation errors in the equivariance. This last perspective was not discussed, and I feel it rather should be.
The theory goes further into deformation stability, which is a fresh perspective in the equivariance literature, so I am happy for its inclusion. Perhaps more motivation for why you think this is necessary would be warmly welcomed.
*Smaller questions/notes for the authors*
- Technically this is scale-translation equivariance, you even write this in the method section of your paper, why is it not in the title? The reason I mention this Is because there are scale equivariant networks, which are not translation equivariant in the literature, see “Warped Convolutions: Efficient Invariance to Spatial Transformations” (Henriques and Vedaldi, 2019).
- Please make the link between Theorem 1 and the group convolution of Cohen and Welling (2016)
- Last paragraph of page 1: A steerable-in-scale filter does exist, see “Deformable kernels for early vision” (Perona, 1991)
- Please use numbering for all display-mode equations.
- This scheme works perfectly in the continuous-image setting, but how about for discretized images? In that case it cannot be scale-translation equivariant because the scaling-action is no longer part of a group.
- In equations 6 and 7, what is the specific motivation for using the laplacian eigen-decompositions as a basis? Is it for steerability with respect to the scale-translation action? Otherwise, surely any basis will do?
- Remark 2: If you are considering a truncation of the scale-axis, surely you can still use an L2 norm when quantifying the robustness of your representation?
- Bandlimiting of the filters: I would consider citing “Structured Receptive Fields in CNNs”, (Jacobsen et al., 2016)
- Pooling: A useful citation here would be “Making Convolutional Networks Shift-Invariant Again” (Zhang, 2019). They precise low pass filter before pooling.
- Experiments:
—I’m not clear on the reason to include reasons with and without batch normalization. This is quite unconventional
— Did you ever use scale augmentation? What is the effect of training on one scale and then testing on another?


Adversarial training is the standard to train models robust against adversarial examples. However, especially for complex datasets, adversarial training incurs a significant loss in accuracy and is known to generalize poorly to stronger attacks, e.g., larger perturbations or other threat models. In this paper, we introduce confidence-calibrated adversarial training (CCAT) where the key idea is to enforce that the confidence on adversarial examples decays with their distance to the attacked examples. We show that CCAT preserves better the accuracy of normal training while robustness against adversarial examples is achieved via confidence thresholding, i.e., detecting adversarial examples based on their confidence. Most importantly, in strong contrast to adversarial training, the robustness of CCAT generalizes to larger perturbations and other threat models, not encountered during training. For evaluation, we extend the commonly used robust test error to our detection setting, present an adaptive attack with backtracking and allow the attacker to select, per test example, the worst-case adversarial example from multiple black-and white-box attacks. We present experimental results using L ∞ , L 2 , L 1 and L 0 attacks on MNIST, SVHN and Cifar10.

*Edit after rebuttal*
See the comment below for a response to the author's rebuttal. I have updated my score to a weak accept.
Summary
========
This paper proposes a more granular adversarial training scheme, where the model is trained to have decaying confidence as the size of the adversarial perturbations increase.
The idea is quite natural and well motivated and presented. The paper suffers in its theoretical and evaluation sections though, which were both fairly confusing to me. I lean towards rejection for this paper but would consider changing my score if the presentation was improved.
Comments
=========
The observation that adversarial training enforces high confidence in the entirety of the epsilon ball is interesting, and the motivation for the proposed scheme is sound.
Yet, while the intuition of CCAT for addressing the accuracy-robustness trade-off makes sense to me, the purported beneficial effect for perturbations outside the given epsilon ball are not as clear.
The authors note in Section 3.1 that adversarial training gives no indications on how the model should behave outside the given epsilon ball, or on other threat models. While this is true, it is not clear how CCAT supposedly remedies this. By making the confidence decay within the epsilon ball, the training also doesn't explicitly enforce any behavior outside this ball or on other threat models. So if CCAT indeed provides benefits in those settings, it isn't clear why those gains should be expected.
The exposition of the technique in Section 3.2. is very clear, and Figure 1 nicely illustrates the effect of CCAT on the class confidences during an attack.
The theoretical toy example in Section 3.3. is a bit confusing, and I'm not convinced that the proof of Proposition 1 is correct. First, a classification problem parametrized by the class imbalance is a little weird in this setting. I have a few questions about the proof:
- You seem to be assuming that the model's confidence is calculated using a softmax. This assumption should be stated.
- The expressions for \hat{p}(y=2|x=x) and \hat{p}(y=1|x=x) are confusing to me. Why are you taking a log over the softmax? These probabilities are now negative... Also, in the second step, an equality of the form e^x / (e^y + e^x) = 1+e^(y-x) is used. This is incorrect, it should be 1/(1+e^(y-x)). In the end, the values log(1+e^a) and log(1+e^(-b)) are not in [0, 1] so they cannot be probabilities.
- From there on, it isn't clear what the rest of the proof is really saying
The experimental section is quite thorough and the results seem convincing. The considered adaptive attack, that optimizes (4) seems natural given that the defense thresholds on the model's confidence. I wonder how this is optimized though: Do you try a targeted attack for every possible target other than the true class?
I found the conflation of evaluation metrics quite confusing. It seems the authors know this and have done an effort to explain their choices, but I feel like this could still be improved further. For example, some of the metrics (e.g., AUC) should be maximized, while others (Err or RErr) should be minimized. This makes the reading of the result tables quite difficult.I would also suggest dropping one digit of precision in the Tables for better readability.
Overall, I have some mixed feelings about this paper. The idea is simple and well explained. The motivation, theoretical analysis, and reporting of results are somewhat confusing though. Some extra work on the presentation could definitely help this paper.
Some additional references
========================
- In the introduction, when mentioning the trade-off between robustness and accuracy, you should cite "Robustness May Be at Odds with Accuracy" by Tsipras et al. rather than Schmidt et al. (that paper shows that robustness may require more training data).
- Also in the introduction, you could consider referencing the following papers on robustness to different threat models:
    * Engstrom et al., "Exploring the Landscape of Spatial Robustness"
    * Tramer and Boneh, "Adversarial Training and Robustness for Multiple Perturbations"
- In the related work, you reference a number of works for adversarial training but not the original idea in the paper by Szegedy et al.
- In Section 3.1., the observation that the epsilon-ball around training examples can cross class boundaries is further analyzed in "Excessive Invariance Causes Adversarial Vulnerability" and "Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness" by Jacobsen et al.
Typos
=====
p.6 enumerator -> numerator


Despite having excellent performances for a wide variety of tasks, modern neural networks are unable to provide a prediction with a reliable confidence estimate which would allow to detect misclassifications. This limitation is at the heart of what is known as an adversarial example, where the network provides a wrong prediction associated with a strong confidence to a slightly modified image. Moreover, this overconfidence issue has also been observed for out-of-distribution data. We show through several experiments that the softmax activation, usually placed as the last layer of modern neural networks, is partly responsible for this behaviour. We give qualitative insights about its impact on the MNIST dataset, showing that relevant information present in the logits is lost once the softmax function is applied. The same observation is made through quantitative analysis, as we show that two out-of-distribution and adversarial example detectors obtain competitive results when using logit values as inputs, but provide considerably lower performances if they use softmax probabilities instead: from 98.0% average AUROC to 56.8% in some settings. These results provide evidence that the softmax activation hinders the detection of adversarial and out-of-distribution examples, as it masks a significant part of the relevant information present in the logits.

This paper suggests that the logits cary more information than the maximum softmax probability for OOD detection. They suggest this with scatterplots and develop techniques to support this claim.
They use logits for as features for OOD detection with by using a kernel density estimator. They also use a NN trained with logits features, but this assumes we can peak at the test distribution, so I ignore this entirely in this evaluation.
Unfortunately their KDE density estimator does not perform better than the maximum softmax probability for OOD detection on CIFAR-10 (74.3 vs 91.7).
They do not show results on CIFAR-100, but since the dimensionality of the logits would increase by an order of magnitude, one would expect kernel density estimation to perform much worse. The authors should include an evaluation on CIFAR-100 for completeness.
Small comments:
Table 3 is a comment about featurization. Does this hold when taking the log of the softmax probabilities (not the same as logits)? If not, then this isn't much a count against the softmax per se. Even then, this is a comment on using softmax information for KDE, not using the maximum softmax probability itself for OOD detection.
The full results for Table 2 are needed. Perhaps place this in an appendix.
They repeat that the logits contain more information than the maximum softmax probability, but so does the raw input. The challenge is not introducing more noise/variance when introducing more information.
I was confused at the experimental description. The information should be contained in one location. They train one of their CIFAR networks for 30 epochs, which isn't enough training time. Consequently, I suspect that those results are not worth drawing implications from since the accuracy is presumably low.


Does attention matter and, if so, when and how? Our study on both inductive and transductive learning suggests that datasets have a strong influence on the effects of attention in graph neural networks. Independent of learning setting, task and attention variant, attention mostly degenerate to simple averaging for all three citation networks, whereas they behave strikingly different in the protein-protein interaction networks and molecular graphs: nodes attend to different neighbors per head and get more focused in deeper layers. Consequently, attention distributions become telltale features of the datasets themselves. We further explore the possibility of transferring attention for graph sparsification and show that, when applicable, attention-based sparsification retains enough information to obtain good performance while reducing computational and storage costs. Finally, we point out several possible directions for further study and transfer of attention.

This paper presents an empirical study of the attention mechanism in the graph attention networks (GAT).  The study reveals that the attention patterns largely depend on the dataset, on some datasets they are sharp, but on others the attention patterns are almost uniform and not so different from the uniform aggregation weights in GNNs that does not have attention.  The authors further tried to utilize these findings and attempted to do attention-based graph sparsification, and showed that they can get a similar level of performance with only a fraction of the edges in the original graph if they do the sparsification based on the attention weights.
Given the popularity of the GAT model in the graph neural networks literature and the effectiveness of the attention mechanism in many deep learning architectures, the empirical study presented in this paper focused on attention is valuable.  The experiments are clearly motivated and executed, which I appreciate.
As this is an empirical paper, one (maybe) problem with it is that the findings presented aren’t that surprising in hindsight - of course the attention patterns should be data dependent, and doing attention-based graph sparsification seems like an obvious thing that should work.  The results on dataset-dependent attention patterns may have told us more about the datasets rather than the GAT model.
There are a few presentation issues that need clarification:
- sec 5.1: it is not clear from the text what \alpha^{static} is.  As mentioned earlier there are multiple possible static attention weights (GCN vs GraphSAGE).
- sec 5.1: I found it strange to normalize the discrepancy score by 2|V|.  The number of terms in the sum should be 2|E| where |E| is the number of edges as each edge is counted twice.  Normalizing by 2|V| does not guarantee the score is in [0, 1] as claimed in the paper.
- sec 5.1: “Besides, these attention do not get concentrated on self loops based on relatively stable values.” -- I don’t see why having stable attention values can show there’s no concentration on self-loops.
- Figure 5 left: looks like the curves can’t reach the right end, this means 1 <= k <= 8 is probably not a good range.
- Table 5 is a bit confusing.  From the text my understanding is that a GAT is trained first, and then do sparsification, and then train another GraphSAGE on the sparsified graph to do prediction.  It’s not immediately clear why the GAT performance in Table 5 is so bad, while the GraphSAGE performance is just way better.  After reading this a few times I realized a much smaller GAT is used (with much smaller hidden size) while the GraphSAGE model is always using a large hidden size.  I think this part needs some improvement.
Overall I liked the empirical study and think the community can benefit from this paper.


The effectiveness of Convolutional Neural Networks (CNNs) has been substantially attributed to their built-in property of translation equivariance. However, CNNs do not have embedded mechanisms to handle other types of transformations. In this work, we pay attention to scale changes, which regularly appear in various tasks due to the changing distances between the objects and the camera. First, we introduce the general theory for building scale-equivariant convolutional networks with steerable filters. We develop scale-convolution and generalize other common blocks to be scale-equivariant. We demonstrate the computational efficiency and numerical stability of the proposed method. We compare the proposed models to the previously developed methods for scale equivariance and local scale invariance. We demonstrate state-of-the-art results on the MNIST-scale dataset and on the STL-10 dataset in the supervised learning setting.

This paper proposes a framework (SESN) for learning deep networks that possess scale equivariance in addition to translation invariance. The formulation is based on group convolution on the scale-translation group. Filters are represented as the coefficients of a set of continuous basis functions, which are sampled (once) at a discrete set of scales. The theoretical formulatioin is clear and interesting. The approach is evaluated in terms of image classification accuracy. The set of baselines is quite exhaustive, including recent papers and papers that are not widely-known.
The most significant improvement for the STL-10 dataset was obtained by the SESN-B variant. This is interesting, because it applies the same operation independently at multiple scales and periodically performs global pooling over scale.
The effectiveness of the approach was demonstrated in the low-data regime, where the inductive bias of scale equivariance is more likely to help.
Overall I found the paper to be thought-provoking and well-executed. There are a number of questions that I would still like to see investigated, but nevertheless I feel that this paper already represents a worthwhile contribution.
Most important issues and questions:
(1.1) The SESN-B architecture resembles quite closely the SI-ConvNet architecture of Kanazawa et al. (except that that paper resized the images instead of the filters). While your approach may be more computationally efficient, it's not clear what leads to the improvement in accuracy here? Can you explain the difference?
(1.2) I would have preferred to see the approach demonstrated on a task which possesses scale equivariance, such as semantic segmentation.
(1.3) To argue in favour of the continuous basis, it would have been more convincing to compare against directly learning the filters at the highest resolution and obtaining the other filters by downsampling. This would not represent a runtime cost during inference.
(1.4a) It seems that SESN-C should contain SESN-A as a special case. However, SESN-C is worse than SESN-A. Do you have any idea whether this is due to optimization difficulty or over-fitting? Could you compare the training objectives?
(1.4b) It is stated that the scale equivariance of SESN-C is worse than SESN-A and -B. However, it should still be scale equivariant, except for boundary effects in scale? What is the parameter N_S in this experiment compared to the number of scales S? And the same question for the plot on the right in Figure 2.
(1.4c) How does SESN-C have the same number of parameters as SESN-A and SESN-B? I thought that more parameters would be required to compute interscale interaction. Was the number of channels reduced?
Issues with clarity:
(2.1) The explanation of equation 10 is not clear. In particular, the diagonal structure in Figure 1 is not stated anywhere in the text, it is simply explained as an expansion from [C_out, C_in, S, V, V] to [C_out S, C_in S, V, V].
(2.2) It's not immediately apparent how multiple applications of convHH are used to provide interscale interaction. I assume it is achieved by shifting f or psi in the scale dimension for each application of convHH, or equivalently by modifying the base-scale in the basis?
(2.3) The explanation of "scalar" and "vector" variants in the experimental section was not perfectly clear. It is stated that "all the layers have now scalar input instead of vector input." However, I understood that the max-reduction was only over the scale dimension, not the channel dimension, so that the inputs are still vectors? This is confusing as a reader.
(2.4) The expansion of the filters to a diagonal structure is described in the "implementation" section. However, it seems that this would entail wasteful multiplications by zero. Nevertheless, SESN is shown to be highly efficient in the appendix. Do you avoid these pointless operations in the actual implementation?
(2.5) It was not immediately obvious how $\psi_{\sigma}(s, t)$ was related to $\psi_{\sigma}(x)$.
Other details:
(3.1) In Figure 2, how many layers does the network have which was used to construct the middle plot?
(3.2) It would have been useful to include a study of the effect of the range and resolution of the scale space.


We investigate the combination of actor-critic reinforcement learning algorithms with uniform large-scale experience replay and propose solutions for two challenges: (a) efficient actor-critic learning with experience replay (b) stability of off-policy learning where agents learn from other agents behaviour. We employ those insights to accelerate hyper-parameter sweeps in which all participating agents run concurrently and share their experience via a common replay module. To this end we analyze the bias-variance tradeoffs in V-trace, a form of importance sampling for actor-critic methods. Based on our analysis, we then argue for mixing experience sampled from replay with on-policy experience, and propose a new trust region scheme that scales effectively to data distributions where V-trace becomes unstable. We provide extensive empirical validation of the proposed solution. We further show the benefits of this setup by demonstrating state-of-the-art data efficiency on Atari among agents trained up until 200M environment frames.

This paper aims to improve the efficiency of the actor-critic method. The authors first analyzed the cause of instability in the prior work, from the perspective of bias and variance. Two remedies were then presented: (i) mixing the experience replay with online learning; (ii) proposing a trust region scheme to select the behavior policies. The authors finally tested the proposed method on Atari games, and showed the better results, compared with the state-of-the-art methods.
In my opinion, the empirical results are impressive, and the authors also provided some insights for the motivation. Given the results on Atari games, this paper could be a great contribution on the actor critic methods. The propositions are presented to support relevant claims, while their significance seems a bit limited, and some further clarification is necessary. The authors also need to address a few confusing statements and missing details.
1. In Proposition 3, the authors claimed that mixing with on-policy data can reduce the bias. I checked the proof but did not find anything relevant. Also, what is the amount of bias reduced?
2. In Equation (1), could you provide a formal definition for "V"?
3. The authors claimed at the beginning of Section 4 that the trust region method was proposed to mitigate the bias and variance problem of V-trace. However, I did not see how this is reflected in Propositions 4 and 5. Is this statement only based on empirical results?
4. It was mentioned right below Equation (4) that "Observe how this inner expectation ... matches the on-policy return...". Could you provide a formal proof?
5. What are the hyperparameters for the 9 agents used in Figure 1? Also, how did you choose "b" in trust region?
6. A few notation issues / typo:
(1) it's -> its
(2) In Equation (5), should "z \in M_{\beta, \pi} (s_t)" be "\mu_z \in M_{\beta, \pi} (s_t)"?
(3) At the 2nd line of Page 7, should the content for the indicator function be "\beta (\pi, \mu, s_t) < b"?


Neural network models are resource hungry. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary quantization provides a more flexible model and often beats binary quantization in terms of accuracy, but doubles memory and increases computation cost. Mixed quantization depth models, on another hand, allows a trade-off between accuracy and memory footprint. In such models, quantization depth is often chosen manually (which is a tiring task), or is tuned using a separate optimization routine (which requires training a quantized network multiple times). Here, we propose Smart Ternary Quantization (STQ) in which we modify the quantization depth directly through an adaptive regularization function, so that we train a model only once. This method jumps between binary and ternary quantization while training. We show its application on image classification.

The paper discusses a generalization to low bit quantization and combines the approaches of binary and ternary quantization methods. Past methods such as Binary Connect and Binary Weights Network have shown that you can train a network efficiently with 1-bit quantization, and methods such as Ternary Weights Network demonstrate 2-bit quantization with weights taking one of {-1, 0, 1} * mu, with mu being a scale computed per weight tensor. The authors generalize these two methods so that the choice of binary vs ternary weights can be made per layer automatically during training. The primary contribution to make that work is by incorporating a generic regularizer with addition hyper-parameters to trade-off between the binary weight regularizer and ternary weight regularizer. In addition to that, the regularization also includes a prior to make the layers prefer binary weights by default. This is done by adding a cost that penalizes the choice of ternary weights for each layer.
Overall, the paper is well written and explained, with supporting experiments to show on MNIST and on CIFAR10 that this method performs quite competitively compared to an all-binary or all-ternary weights network. Low-bit quantization is an important research area and this paper makes a strong contribution by studying mixed-precision low-bit quantization. The mathematical explanations of the generalized regularizer are well justified. For instance, the regularization constant lambda is normalized for each layer by the total number of weights to evenly weight all layers.
Although the experiments cover MNIST and CIFAR10, it's not clear how mixed-precision low-bit methods perform on models more prevalent in the real-world. Supporting the experiments with ResNet variants on ImageNet would help clarify that further.  The paper very well explains the fundamentals of 1-bit and 2-bit methods, and the contribution of this paper (generalization of these two methods) is a somewhat natural extension without significant novelty. Moreover, in addition to the accuracy, it would also be better to understand how the run-time performance of such models (on existing software and hardware implementations) compares to pure binary and ternary networks, as knowledge of that would reveal insights into systems optimizations to be made in future work in this field. Given all of this, my rating is a weak accept.
Pros:
- The problem and the fundamentals (prior work) are very well laid out.
- The regularization component that generalizes the two types of quantization is sound.
- Experiments on CIFAR10 strongly show improved accuracy and higher compression ratio compared to ternary weights network.
Cons:
- Lacking more realistic experiments on larger datasets such as ImageNet and models like ResNets.
- The runtime performance of how STQ compares to 1-bit and 2-bit quantization variants isn't shown.
- The regularizer introduces more hyper-parameters to tweak and it's not clear how sensitive these are to the choice of the architecture. Different values of gamma are used in the two experiments, and further analysis on how the performance varies for various values of lambda and gamma would shed further insight.
Minor comments:
- In equation (3), the term under argmin should be mu and not alpha.
- Section 3, line above equation (9) reads "very per layer" instead of "vary per layer"


We present an approach to Lagrangian fluid simulation with a new type of convolutional network. Our networks process sets of moving particles, which describe fluids in space and time. Unlike previous approaches, we do not build an explicit graph structure to connect the particles but use spatial convolutions as the main differentiable operation that relates particles to their neighbors. To this end we present a simple, novel, and effective extension of N-D convolutions to the continuous domain. We show that our network architecture can simulate different materials, generalizes to arbitrary collision geometries, and can be used for inverse problems. In addition, we demonstrate that our continuous convolutions outperform prior formulations in terms of accuracy and speed.

This paper applies 3D convolutions to the problem of Lagrangian fluid simulation. The primary difficulty in this is that, unlike for Eulerian fluid simulation, which represents the fluid as a grid and adapts nicely to 3D convolutions, in Lagrangian simulations the fluid is represented as an unordered set of particles. It is not straight-forward to apply 3D convolutions on such a data structure, however this paper proposes a method to apply the same regular-grid kernels used in grid-based convolutions to the particle structure. To do this, several points around the kernel are evaluated by first using trilinear interpolation between the particles to get feature values at those points and then convolving those values with the kernel weights. This results in a new particle in the next layer up with those features. In the paper, this method is used to train the weights of the network to reproduce fluid dynamics generated by a simulator. The results show that the proposed method was able to model fluid dynamics over 2 timesteps more accurately than other methods and can do so quickly.
While I have some reservations about this paper (detailed below), on the whole I think it is a quality contribution and should be accepted. This paper contributes a novel method for performing 3D convolutions on unordered particle sets, and it shows that the learned fluid dynamics generalize to novel situations. One major hurdle to applying modern convolutional learning techniques to Lagrangian methods is the mismatch between the layout of the data (unordered particles) and the layout of the kernels (regular grid). This paper presents a novel way of bridging that divide, and it shows that the proposed method actually works by applying it to the problem of fluid dynamics and successfully learning it. However, one major concern I had was that it seems all of the training data was generated in box-like environments, which could easily lead to overfitting. This was alleviated by the results showing that although the network was trained only in boxes, it generalized to environments with channels and waterfalls (as seen in the video). This is a powerful result and shows that this method really did learn fluid dynamics and not just a shortcut that only works in boxes.
I do think this paper can be improved in a few aspects however. The biggest issue is that the quantitative analysis of the core functionality (reproducing fluid physics) is lacking. The paper only reports results for error after at most 2 timesteps, which is not nearly long enough to determine if the output is accurate. Furthermore, the results are only reported for the box scenes, not the generalization scenes mentioned above. Qualitatively, from the videos, it is clear that the output does at least somewhat model fluid dynamics, but it would be much better to have hard numbers to back that up. I suspect the authors discovered that Lagrangian systems are sufficiently chaotic that after only a few timesteps the particle positions have diverged significantly. This is not a bug but a feature of such systems. In Lagrangian fluids, the particles are but an approximation of the fluid, and unlike Eulerian systems*, multiple different sets of particles can approximate the same fluid. This makes particle position only useful as a measure of error if the trained model can perfectly reproduce the fluid dynamics. But of course it can't (trained networks aren't ever perfect in practice), and so small errors quickly compound into large particle position disparities. So even though the trained network models the fluid well overall, the particles end up in completely different locations. Instead a better error metric would be something like measuring the difference between the surface of the fluids, or the velocities or densities at various locations. These are agnostic to the particular particle positions, but still measure how well two different sets of particles represent the same fluid. Using a metric like this, it would be nice to see error graphs over time for both the box and generalization scenes.
A couple other smaller points. The chaotic divergence behavior of DPI-Nets seems inconsistent with that paper. Is this possibly a bug in the way it was implemented here? Additionally, the paper states that the convolutions of SPNets were "specifically designed to implement the position-based fluids algorithm" but that it was used in the paper with a much larger number of channels. If it was designed only to work for that one algorithm, how were the number of channels increased? That is unclear. Also, the average error for SPNets is not shown in Table 1 and it is not stated why.
*Assuming same grid shape, size, and position.


Sentiment classification is an active research area with several applications including analysis of political opinions, classifying comments, movie reviews, news reviews and product reviews. To employ rule based sentiment classification, we require sentiment lexicons. However, manual construction of sentiment lexicon is time consuming and costly for resource-limited languages. To bypass manual development time and costs, we tried to build Amharic Sentiment Lexicons relying on corpus based approach. The intention of this approach is to handle sentiment terms specific to Amharic language from Amharic Corpus. Small set of seed terms are manually prepared from three parts of speech such as noun, adjective and verb. We developed algorithms for constructing Amharic sentiment lexicons automatically from Amharic news corpus. Corpus based approach is proposed relying on the word co-occurrence distributional embedding including frequency based embedding (i.e. Positive Point-wise Mutual Information PPMI). Using PPMI with threshold value of 100 and 200, we got corpus based Amharic Sentiment lexicons of size 1811 and 3794 respectively by expanding 519 seeds. Finally, the lexicon generated in corpus based approach is evaluated.

This paper proposes a domain-specific corpus-based approach for generating semantic lexicons for the low-resource Amharic language. Manual construction of lexicons is especially hard and expensive for low-resource languages. More importantly, the paper points out that existing dictionaries and lexicons do not capture cultural connotations and language specific features, which is rather important for tasks like sentiment classification. Instead, this work proposes to automatically generate a semantic lexicon using distributional semantics from a corpus.
The proposed approach starts from a seed list of sentiment words in 3 pre-determined POS classes. This is followed by deriving a PPMI matrix from the word-context co-occurence matrix (context size= 2). Now given a word, the cosine distance is computed from centroid of each seed classes and words which are similar than a given threshold, are added to the original list. This process is repeated for a pre-specified number of iterations. They apply the generated lexicons to subjectivity detection and sentiment classification task in a small annotated Amharic corpus of facebook posts
Strengths:
1. NLP tasks on low-resource language is very challenging. This paper proposes a efficient, unsupervised way of gathering semantic lexicons that perform reasonably well on a downstream task.
Weaknesses:
1. Related work: The paper needs to cite and mention a much more broader literature. Currently, it mentions just 3 related work, and contrasts itself with one. But given there is so much work on distributional semantics (in English and other languages), they deserve mentioning. Few example of ones are 1. The distributional inclusion hypotheses and lexical entailment (Geffet and Dagan 2005), Distributed representations of words and phrases and their compositionality (Mikolov et al., 2013), Improving hypernymy detection with an integrated path-based and distributional method (Shwartz et al., 2017), Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection (Chang et al., 2018). However, this is by no means a complete list and you should refer to these papers to get a list of other relevant work.
2. Novelty of work: The novelty of the work is rather limited and the paper should try more low-dimensional embedding based approaches which has been proven to be very effective for a wide variety of tasks. In section 3, the paper mentions there are primarily two kinds of approaches (count based vs embedding based) — The paper should motivate why it chose one over another.
3. Baselines: The paper would benefit from having some comparison with learned baselines trained from some distantly supervised data from, for example the SWN and SOCAL lexicons.
4. Organization of the paper: The writing and the organization of the paper needs to be better. For example, equation 3, 4, 5 are rather simple and can be condensed to one equation (no need to show averaging of the seed embeddings). Also in equation 3, would \vec{w} would be \vec{x}?. The results and discussions did not have either and over-all there are many grammatical errors.
5. I am not sure what the subjectivity detection task is in Table 3. Is it a standard task -- if not, the paper should define it first.
Overall, the paper is a nice effort but in its current form it is not ready for ICLR and I hope the comments will help make the paper better for upcoming NLP workshops and conferences.


While there are convergence guarantees for temporal difference (TD) learning when using linear function approximators, the situation for nonlinear models is far less understood, and divergent examples are known. Here we take a first step towards extending theoretical convergence guarantees to TD learning with nonlinear function approximation. More precisely, we consider the expected learning dynamics of the TD(0) algorithm for value estimation. As the step-size converges to zero, these dynamics are defined by a nonlinear ODE which depends on the geometry of the space of function approximators, the structure of the underlying Markov chain, and their interaction. We find a set of function approximators that includes ReLU networks and has geometry amenable to TD learning regardless of environment, so that the solution performs about as well as linear TD in the worst case. Then, we show how environments that are more reversible induce dynamics that are better for TD learning and prove global convergence to the true value function for well-conditioned function approximators. Finally, we generalize a divergent counterexample to a family of divergent problems to demonstrate how the interaction between approximator and environment can go wrong and to motivate the assumptions needed to prove convergence.

####
A. Summarize what the paper claims to do/contribute. Be positive and generous.
####
The paper characterises the convergence of Temporal Difference learning for on-policy value estimation with nonlinear function approximators. It looks at a few classes of functions which include Deep ReLU networks, with and without residual connections. It looks at how a few aspects of function approximators affect their convergence properties, and how these are intertwined with a property of the environment.
This topic is important for improving the theoretical understanding of Deep RL. The paper provides a series of mathematical results which will be interesting for the community as they are well-motivated, intuitively explained, and of clear practical relevance.  The paper works within a simplified setup that is however NOT toy. It makes reasonable simplifying assumptions to avoid confounding issues with exploration, off-policyness of data, and stochastic optimization. Within a continuous-time MRP framework they show that:
1. When a particular class of function approximators known as homogenous functions (e.g. Deep ReLU networks) is used, the error on the state value function can be bounded. Tweaking the class of function approximators by making them like residual networks (residual-homogenous) obtains a bound similar to known bounds for linear function approximators.
2. It is known that TD(0) converges with linear function approximators and arbitrary environments. It is also known that TD(0) converges with arbitrary function approximators when the environment is fully reversible. This paper shows that there is in fact a tradeoff between how well the function approximator is conditioned and how reversible the environment is (for particular definitions of well-conditionedness and the "extent" to which an environment is reversible). This nicely links up known theory and is especially relevant for practitioners who would like to apply neural net function approximators in arbitrary environments.
3. They show that using n-step returns instead of TD(0) returns can have a similar effect to the environment being reversible. That's a nice insight.
4. There is a classic counterexample for TD(0) with a nonlinear function approximator diverging. The paper makes this example more general which more clearly demonstrates how/why convergence fails beyond the single pathological example whose relevance to real neural nets was hard to determine. It makes it clear that a necessary condition for convergent TD learning with function approximators is dodging this more general class of divergent example.
5. The theory is supported by (toy) experiments, whose empirical results suggest the theory can be made more general (e.g. to include more classes of function approximators).
####
B. Clearly state your decision (accept or reject) with one or two key reasons for this choice.
####
This paper should be accepted because the results are interesting, relevant, novel (as far as the reviewer understands), well-explained, and as far as the reviewer can tell correct (though I have not scrutinized the proofs in the appendix).
The paper is interesting and easy to read (even for someone without background in proving the convergence of RL algorithms).
####
C. Provide supporting arguments for the reasons for the decision.
####
Whether or not (and under what conditions) TD(0) converges is an important object of study for the Deep RL community, which is well-represented at ICLR. The results shown here should be more widely known.
Good intuitions given for the mathematical results in addition to proofs (e.g. why homogeneity prevents divergence).
This work contributes to the understanding of Deep RL and could eventually lead to actionable theory which lets us design more robust RL systems (with insights about the coupling between learning algorithm, function approximator, and environment).
####
D. Provide additional feedback with the aim to improve the paper. Make it clear that these points are here to help, and not necessarily part of your decision assessment.
####
D0. My main critique is that the results are somewhat disparate. How does homogeneity or residual homogeneity relate to the conditioning number of the neural tangent kernel? Can these all be connected up better?
D1. Be more clear in Remark 1 on page 4 that the homogenous property applies to *deep* ReLU networks. Otherwise the reader may assume the proof only applies to single-layer neural networks until they read the more detailed exposition in the appendix.
D2. Is it an issue for homogeneity if there is a point where the derivative does not exist (e.g. at x=0 for ReLU).
D3. Can the paper make it more clear to a Deep RL audience in the intro or discussion what the holy grail of this research direction would be?
D4. As tanh or multiplicative activations (e.g. those found in LSTM/attention networks) are not homogenous, can the authors speculate about whether or not they would have similar convergence properties to homogeneous activations? What are the obstacles to a similar proof for this class of networks?.


Self-supervised neural machine translation (SS-NMT) learns how to extract/select suitable training data from comparable -rather than parallel-corpora and how to translate, in a way that the two tasks support each other in a virtuous circle. SS-NMT has been shown to be competitive with state-of-the-art unsupervised NMT. In this study we provide an in-depth analysis of the sampling choices the SS-NMT model takes during training. We show that, without it having been told to do so, the model selects samples of increasing (i) complexity and (ii) task-relevance in combination with (iii) a denoising curriculum. We observe that the dynamics of the mutual-supervision of both system internal representation types is vital for the extraction and hence translation performance. We show that in terms of the human Gunning-Fog Readability index (GF), SS-NMT starts by extracting and learning from Wikipedia data suitable for high school (GF=10-11) and quickly moves towards content suitable for first year undergraduate students (GF=13).

This paper studies how to extract/select suitable training data from comparable —rather than parallel— corpora. The idea sounds reasonable.
My major concern is about the evaluation: it didn't compare with any existing work. Actually there quite a few papers  on mining parallel sentences from comparable corpora such as Wikipedia, as shown below. Seems the authors are not aware of those works and didn't review and compare with them. Without such comparisons, it is difficult to judge the effectiveness of the proposed method and the quality of this work.
[1] Finding similar sentences across multiple languages in Wikipedia, Proceedings of the Workshop on NEW TEXT Wikis and blogs and other dynamic text sources. 2006.
[2] Method for building sentence-aligned corpus from wikipedia, 2008 AAAI Workshop on Wikipedia and Artificial Intelligence (WikiAI08). 2008.
[3] Extracting parallel sentences from comparable corpora using document-level alignment, Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 2010.
[4] "Improving machine translation performance by exploiting non-parallel corpora." Computational Linguistics2006.
[5] https://www.aclweb.org/anthology/W04-3208.pdf
[6] https://openreview.net/pdf?id=ryza73R9tQ
Minor issues:
	1. "For each language pair, a shared byte-pair encoding (BPE) (Sennrich et al., 2016) of 100k merge operations is applied." Most papers on neural machine translation don't use such a large BPE size, which is likely to lead to better performance. It would be better to use the same setting as previous work for fair comparisons.
	2. "In the case of SS-NMT, both tasks —data extraction and learning NMT— enable and enhance each other, such that this mutual supervision leads to a self-induced curriculum, which is the subject to our analysis." Similar idea, mutual boosting between data selection and model training, has been explored in the following paper, although not for machine translation. What's the difference between these two papers?
Learning to Teach, ICLR 2018.


We propose R2D2 layers, a new neural block for training efficient NLP models. Our proposed method is characterized by a dynamic weight diffusion mechanism which learns to reuse and reduce parameters in the conventional transformation layer, commonly found in popular Transformer/LSTMs models. Our method is inspired by recent Quaternion methods which share parameters via the Hamilton product. This can be interpreted as a neural and learned approximation of the Hamilton product which imbues our method with increased flexibility and expressiveness, i.e., we are no longer restricted by the 4D nature of Quaternion weight sharing. We conduct extensive experiments in the NLP domain, showing that R2D2 (i) enables a parameter savings of up to 2× to 16× with minimal degradation of performance and (ii) outperforms other parameter savings alternative such as low-rank factorization and Quaternion methods.

Paper Summary:
This paper proposes to train smaller models by decomposing the weights of fully connected networks as the product of smaller matrices, along with a reordering/transpose of the outcome. The experiments shows that models with less parameters yield comparable performance with their larger counterparts.
Review Summary:
The method is technically sound and the paper reads well. Experiments demonstrate the efficacy of the method, although some ablations are missing (see below). The paper is however not clear on the ultimate objective of the method (speed/accuracy/generalisation?) and does not compare with alternatives.
Detailed Review:
The introduction does not make clear if your motivation to make model smaller is training speed, inference speed, memory usage, generalization accuracy. Please clarify.
The explanation of the method, i.e. Section 2.2.1, is not clear, in particular for the mapping \psi. I feel it would cleared if somewhere in the paper there was an equation with the element-wise correspondence, i.e. H_{?,?} = \sum_k A_i,k S_k,j
In that section, you should introduce that n is a hyperparameter before using it as well.
In that section, you could also discuss parameter initialization, and whether this model can use weight decay over H or A/S. it is also not clear to me if you control the norm ratio between A and S given the weight magnitude is over parameterized.
The experimental section lack a validation/ablation study to help the reader understand the interplay between the number of blocks and the number of latent dimensions. It will also be good to show learning curves to compare training speed of different parameterization.
Also no training errors are reported, does your method can be seen as a regularizer, i.e. is training objective closer to valid objective when n grows? Did you have to change other regularization parameters like dropout.
To me the main weakness of the paper lies in the lack of comparison with alternatives. Replacing fully connected layers with alternative has a rich literature that the authors ignore.
I feel it is necessary to compare the approach with
(i) block diagonal approaches, popular since ResNext for convolutions but equally applicable to linear layers. https://arxiv.org/abs/1611.05431
(ii) other form of structured sparsity. https://arxiv.org/abs/1902.09574 (survey). https://arxiv.org/abs/1812.08301 (squantizer) https://arxiv.org/abs/1802.08435 (block sparsity)...
(iii) distillation of large models into smaller models.  https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1702.01802
(iv) it might not be necessary to compare, but at least mentioning approaches which predict weights from a meta network would be good.  https://arxiv.org/abs/1609.09106
As a reviewer, I am a bit annoyed that made no effort to have a decent list of related work and that they delegate that work to the reviewers to do so.
Details:
"transformation layer": this is not common terminology, please prefer linear layer or fully-connected layer.
please define all acronyms, e.g. FC.
The experimental section does not define \alpha (end of page 6).


Recent advances in Deep Learning have led to a significant performance increase on several NLP tasks, however, the models become more and more computationally demanding. Therefore, this paper tackles the domain of computationally efficient algorithms for NLP tasks. In particular, it investigates distributed representations of n-gram statistics of texts. The representations are formed using hyperdimensional computing enabled embedding. These representations then serve as features, which are used as input to standard classifiers. We investigate the applicability of the embedding on one large and three small standard datasets for classification tasks using nine classifiers. The embedding achieved on par F 1 scores while decreasing the time and memory requirements by several times compared to the conventional n-gram statistics, e.g., for one of the classifiers on a small dataset, the memory reduction was 6.18 times; while train and test speed-ups were 4.62 and 3.84 times, respectively. For many classifiers on the large dataset, the memory reduction was about 100 times and train and test speed-ups were over 100 times. More importantly, the usage of distributed representations formed via hyperdimensional computing allows dissecting the strict dependency between the dimensionality of the representation and the parameters of n-gram statistics, thus, opening a room for tradeoffs.

This paper proposes the use of hyperdimensional (HD) vectors to represent n-gram statistics. The HD vectors are first generated from the whole corpus. Then, it is aggregated or bundled to a vector for each sample as an input of a classifier training. The evaluation is conducted on four datasets: Chatbot, AskUbuntu, WebApplication and 20 News Group using a bunch of classifier including KNN, Random Forest, MLP etc.
It is interesting to see how to hash/project the high dimensional n-gram vector into a lower space for efficiency. The approach is useful in online production systems, and it is eco-friendly. However, there are a few concerns detailed as follows:
1. Can it be generalized to contemporary learned embeddings, e.g., word2vec and GloVe?
2. Lack of proper baselines for comparison. Word2vec, GloVe are trained on large corpora once and can be applied directly to other tasks, and they should be served as baselines. Furthermore, the simple bag of word/TF-IDF should be included as baselines as well.
3. Lack of analysis: it is hard to understand what kind of HD vectors are generated. Are these n-grams semantically related projected nearby in the HD space? This helps readers to understand the constructed embeddings.
4. The SentEval benchmark is popular in sentence level representation learning and it is well known. It is better to see some evaluations on it as well. http://www.lrec-conf.org/proceedings/lrec2018/pdf/757.pdf
Minor comments:
1. The Subword Semantic Hashing is originally from DSSM published in 2013.  (https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/cikm2013_DSSM_fullversion.pdf)
2. What is $v_c$ in 4.2?


We introduce Continual Learning via Neural Pruning (CLNP), a new method aimed at lifelong learning in fixed capacity models based on neuronal model sparsification. In this method, subsequent tasks are trained using the inactive neurons and filters of the sparsified network and cause zero deterioration to the performance of previous tasks. In order to deal with the possible compromise between model sparsity and performance, we formalize and incorporate the concept of graceful forgetting: the idea that it is preferable to suffer a small amount of forgetting in a controlled manner if it helps regain network capacity and prevents uncontrolled loss of performance during the training of future tasks. CLNP also provides simple continual learning diagnostic tools in terms of the number of free neurons left for the training of future tasks as well as the number of neurons that are being reused. In particular, we see in experiments that CLNP verifies and automatically takes advantage of the fact that the features of earlier layers are more transferable. We show empirically that CLNP leads to significantly improved results over current weight elasticity based methods. CLNP can also be applied in single-head architectures for completely fixed capacity continual learning.

This paper proposes a method for mitigating catastrophic forgetting in neural networks, which is an important and unsolved problem in the sequential training of multiple tasks.  The method works by (i) identifying an active subnetwork after training on one task that can perform the task to a sufficient degree of accuracy, (ii) freezing the subnetwork and pruning interfering connections to it from the rest of the network, and (iii) training the rest of the network on the next task and iterating. The contribution is a method that allows for training of sequential tasks in a fixed capacity architecture with zero catastrophic forgetting guaranteed, while still allowing for transfer between tasks via feature sharing. Additionally, it allows for controlled fine-tuning of the trade-off between accuracy and network capacity, which the authors refer to as ‘graceful forgetting’, by adjusting two hyperparameters that adjust the level of sparsity during the training of a task. The method is shown to outperform several other continual learning methods on permuted MNIST and one other method on split CIFAR-100, both standard evaluations in continual learning.
While the model demonstrates the enviable properties for a continual learning algorithm of zero forgetting and ability to transfer in a fixed network, it suffers from a significant limitation (acknowledged by the authors) that leads me to recommend it for rejection in its current incarnation:
* Due to the progressive freezing of the parameters, the network eventually reaches full capacity, making it theoretically difficult (in the multi-head setting, depending on the degree of transfer between tasks) or impossible (in the single-head setting) to learn how to perform new tasks.
* This is particularly problematic because the mechanism for 'graceful forgetting’ can only be applied on the subnetwork for a task that has just been trained on; once N tasks have been learnt, the subnetwork for task k<N cannot be altered without potentially affecting the performance on tasks k to N.
* In the paper, subnetworks are sparsified to within a fixed margin of the maximum accuracy of each task - with this procedure, you cannot predict how much capacity will be taken up by each task, and so you cannot know in advance how many tasks will fit into the network. If instead you chose to sparsify to a given subnetwork capacity per task in order to guarantee being able to fit a certain number in, then you can not control the accuracy level for each task.
* Another consequence of the method is that it is not clear how one can easily resume training on a previous task (perhaps to take advantage of transfer from subsequent tasks). You could reinstate connections from the features of subsequent tasks to the earlier task and train, but presumably these would initially interfere with the original subnetwork. Have the authors considered any ways of achieving this?
In fairness to the authors, they (a) show that the growth in utilised capacity slows as more tasks are added and (b) propose that the layers of the network could be dynamically extended as a way of overcoming a network at full capacity. For the reasons given above, however, it is not a practical fixed capacity algorithm for a lifelong learning setting, which constitutes an indefinite stream of sequential tasks, since at capacity it can suffer from sudden 'catastrophic remembering’, as opposed to the ‘graceful forgetting’ advertised in the paper. I do think that the idea has potential and that the algorithm would be *significantly* strengthened if there were a mechanism for graceful forgetting of all or selected previous tasks when the network is at capacity.
Further Comments / questions:
* The method is claimed to be the “first viable algorithm for single-head continual learning”. This statement is unspecific - what does ‘viable’ mean in this context? Online EWC [1] is a continual learning algorithm that can theoretically applied in a single-head setting - what makes it non-viable?
Minor comments not affecting review:
* Better to label the weight types in the caption of Figure 1 rather than just in the main text
* In order to see the effect of depletion more clearly, it would also be useful to see the performance of the current task in isolation rather than just the average of all previous tasks.
* Section 2, line 8: “speci[c]fic"
* The main text reference to Figure 5b compares it to the single-head MNIST usage graph - where is this?
[1] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." International Conference on Machine Learning. 2018.