Stochastic neural networks (SNNs) are currently topical, with several paradigms being actively investigated including dropout, Bayesian neural networks, variational information bottleneck (VIB) and noise regularized learning. These neural network variants impact several major considerations, including generalization, network compression, and robustness against adversarial attack and label noise. However, many existing networks are complicated and expensive to train, and/or only address one or two of these practical considerations. In this paper we propose a simple and effective stochastic neural network (SE-SNN) architecture for discriminative learning by directly modeling activation uncertainty and encouraging high activation variability. Compared to existing SNNs, our SE-SNN is simpler to implement and faster to train, and produces state of the art results on network compression by pruning, adversarial defense and learning with label noise.

Summary:
This paper presents an efficient stochastic neural network architecture by directly modeling activation uncertainty and adding a regularization term to encourage high activation variability by maximizing the entropy of stochastic neurons. Compared with other existing approaches, such as Bayesian neural networks and variational information bottleneck, the proposed architecture is simpler to implement and faster to train. The authors also achieve state of the art results in various fields, including network compression by pruning, adversarial defense and learning with label noise.
Major comments:
- Overall, I find the paper is easy to follow and the experimental evaluation shows promising results, but my major concern is about the novelty of this work, given the fact that the structure of the proposed stochastic layers is quite similar to VIBNet.
- The derivation of the max-entropy term is somewhat unclear and I think the paper needs a major revision on this part. The authors suggest using a Gaussian with a finite mean and an infinite variance as the non-informative prior for the produced Gaussian random variable z (in Eq. 1), and then minimize the KL divergence between the produced Gaussian and the infinite-variance Gaussian. However, this may be questionable from a Bayesian viewpoint, in the sense that the infinite variance leads to an improper prior as the variance increases without bound and thus may produce an improper posterior distribution. This is not discussed and needs to be clarified in Sect. 2 (Max-entropy Regularization).
- There are things unclear in the derivation (last line of Eq. 2), since log(\sigma_2) trends to infinity.
- In addition, the penalty terms for different types of tasks are directly given only based on some of the assumptions that the authors have made, there does not seem to be any theoretical justification for such choices.
Minor comments:
- Some of the notations used in this paper seem a bit confusing, which may hinder readability. For example, on page 3, in “The non-informative prior is a Gaussian with arbitrary mean (\mu_1) and infinite variance (\sigma_1)”, I guess \sigma means the standard deviation? I would like to recommend using N(\mu, \sigma^2) to denote a Gaussian distribution, where \sigma means the standard deviation and \sigma^2 the variance.
- On page 3, in “where \sigma(h|\theta) denotes the predicted standard deviation of hidden unit h given the neuron uncertainty prediction parameter \theta”, there is no discussion on the neuron uncertainty prediction parameter. Does it means the predicted standard deviation is again parameterized by \theta?


Researches shows that attempts done to apply existing dependency parser on morphological rich languages including Amharic shows a poor performance. In this study, a dependency parser for Amharic language is implemented using arc-eager transition system and LSTM network. The study introduced another way of building labeled dependency structure by using a separate network model to predict dependency relation. This helps the number of classes to decrease from 2n+2 into n, where n is the number of relationship types in the language and increases the number of examples for each class in the data set. Evaluation of the parser model results 91.54 and 81.4 unlabeled and labeled attachment score respectively. The major challenge in this study was the decrease of the accuracy of labeled attachment score. This is mainly due to the size and quality of the tree-bank available for Amharic language. Improving the tree-bank by increasing the size and by adding morphological information can make the performance of parser better.

The paper claims to build a transition-based dependency parser for Amharic. To produce a parse tree, the paper takes a two-stage approach, first producing the structure of the tree, followed by annotating each individual arcs on the tree. The paper also explores attention and finds it useful for improving the performance.
I am giving a score 1, because the paper lacks precision and the contribution is not significant.
The major problem of the paper is precision. There is a lack of formal descriptions of the parser, e.g., the representation of the input and output, what each individual block in the figures are doing, how the arc-eager transition parser works. The text description is certainly helpful, but without formal descriptions, it is impossible to reproduce the results in the paper. Besides, without setting up a precise language/notation, the paper is not easy to follow.
In terms of contribution, the paper begins with the motivation that Amharic has certain language characteristics that are absent in languages, such as English, where most research effort has been spent. However, the paper still uses a parser for English, and does not utilize the special features of Amharic. Generic architectural modification, such as adding attention, has been explored, but this has little contribution on its own and has little to do with Amharic.
In addition to the above, other minor weaknesses of the paper includes typos, poor experimental practices, missing citations and related work, and other wording issues.
In particular, choosing the split of the data set that achieves the best result, as quoted below, is concerning.
"We experimented on 60/40, 70/30 and 80/20 train-test splitting ratios ... Therefore, we select 70/30 train-test splitting ratio throughout the experiments."


Adversarial learning has shown its advances in generating natural and diverse descriptions in image captioning. However, the learned reward of existing adversarial methods is vague and ill-defined due to the reward ambiguity problem. In this paper, we propose a refined Adversarial Inverse Reinforcement Learning (rAIRL) method to handle the reward ambiguity problem by disentangling reward for each word in a sentence, as well as achieve stable adversarial training by refining the loss function to shift the stationary point towards Nash equilibrium. In addition, we introduce a conditional term in the loss function to mitigate mode collapse and to increase the diversity of the generated descriptions. Our experiments on MS COCO show that our method can learn compact reward for image captioning.

The authors propose using a recent method for adversarial inverse reinforcement learning (AIRL) for the task for generating high-quality image captions. Leveraging the GAN framework, a discriminator is trained to distinguish real captions from those produced by the generator, while the generator is optimized with policy-gradients (REINFORCE) to maximize the pseudo-reward from the discriminator. The main difference from prior work seems to be that the discriminator acts on a word-level, rather than sentence-level (as done, for instance in Dai et. al. 2017). Correspondingly, the generator policy is updated with the objective of 1-step reward maximization (more like contextual bandits), rather than with a long-term sequential decision-making objective (as done in Dai et. al. 2017). The evaluation is done using 2 data splits – standard and robust, with various metrics such as SPICE, CIDEr, BLEU, CHAIR. Diversity analysis and ablations are also performed to dissect the performance of the proposed approach.
My 2 main issues with the paper are confusing motivation (in section 1) and various imprecise parts (in section 3 and 4).
1.	The authors argue that current GAN-based captioning models provide ill-defined rewards due to the “reward ambiguity problem”. This problem is not explained or motivated well in the paper, but instead the readers are referred to the AIRL paper. “Reward ambiguity” in inverse-RL arises because there could be many reward functions that yield the same optimal policy. The AIRL algorithm recovers one of these possible reward functions, and since such a recovered reward could be shaped by the environment dynamics, AIRL attempts to disentangle the reward from the dynamics. The motivation there is to use the recovered reward on a new system with different transition dynamics. In the context of this paper though, I would like to understand the angle of reward ambiguity. The authors disentangle the sentence reward into word-wise rewards; however, I’m not sure if there’s any relation between this and the disentanglement done in AIRL for solving reward ambiguity.
2.	One of the objectives is learning compact rewards. It is claimed that addition of a constant term to the reward provided to the generator policy results in this, but what’s the intuition behind this? As for evaluation, it needs to be shown that words with similar semantics have similar discriminator score. How do we conclude this from Figure 2.? Also, please include Up-Down method results in Figure 2.
3.	Section 3 questions
     a.	How is state s_t defined? It is very hard to follow sections 3 and 4 without a clear definition and example for this.
     b.	“Finn et. al. 2016 proved that IRL is mathematically equivalent …” --- this is imprecise. Maximum-Entropy-IRL is equivalent to the GAN formulation, not general IRL.
     c.	p_theta(a,s) is referred to as “reward distribution”. I don’t think it’s a distribution.
     d.	Equation 3. AIRL defines g only as a function of state (s_t) for the disentanglement, and not like what the authors have written.
     e.	Equation 4. How is s_t sampled?
4.	Section 4 questions
     a.	4.1 says discriminator “maximizes the divergence”. This doesn’t seem correct.
     b.	f is referred to as state-value. This doesn’t seem correct.
     c.	Shouldn’t the -1 term in Equation 8 disappear under expectation?
     d.	Don’t understand how second line of Equation 11 is arrived at.
There are quite a few other sources of mathematically imprecise writing that I noticed. I would recommend the authors to be more robust in their presentation.


We propose a generic confidence-based approximation that can be plugged in and simplify the auto-regressive generation process with a proved convergence. We first assume that the priors of future samples can be generated in an independently and identically distributed (i.i.d.) manner using an efficient predictor. Given the past samples and future priors, the mother AR model can post-process the priors while the accompanied confidence predictor decides whether the current sample needs a resampling or not. Thanks to the i.i.d. assumption, the post-processing can update each sample in a parallel way, which remarkably accelerates the mother model. Our experiments on different data domains including sequences and images show that the proposed method can successfully capture the complex structures of the data and generate the meaningful future samples with lower computational cost while preserving the sequential relationship of the data.

The authors consider the problem of sampling time series.
To solve the problem they propose a method that is based on the autoregression model. The novelty here lies in the proposed sampling methods: we start with a sampling of a prior and then try to generate data according to the restored distribution. We learn two functions: signal recovery and confidence prediction.
The main hyperparameter of the algorithm $\varepsilon$ identifies how much samples we accept.
The distinctive feature of the algorithm is speed-up for the sample generation process.
Weak reject
There are a significant number of works on video generation, see e.g. [1, 2], references therein and articles that cite these two articles. The problem setting seems pretty similar. It seems like a good idea to compare to these methods (and it seems that video generation is a very resource-demanding procedure, and they don't use parallel applications similar to proposed in the paper. What is the reason?) Most of the approaches use only one frame to generate video, but it seems that LSTM in these methods will benefit from using of multiple frames as input (and will be able to transfer information in autoregression manner by transferring all they need in a hidden state).
The article, in my opinion, will benefit from comparison to these approaches or at least by using some benchmarks from these works to demonstrate feasibility of the considered approach, also it seems that these works are good for demonstration of parallelization capabilities (as in many cases the same idea applies).
Not minor comments:
1. In Figure 2 (a) it is not clear how the data and prediction were generated. According to the procedure in Figure 1 and text we use the same input for all approaches. However solid lines for different epsilons are different.
2. The effect of the dependence of recovery of quality for Figure 2 (b) is not explained and is controversial: we get the smallest error for intermediate acceptance ratio, however, there is also a decrease of error if we further increase the gauge threshold (btw the term gauge threshold is new to machine learning community, consider replacement of it)
3. More simpler examples will benefit the paper, as we'll be able to know more fundamental properties of the proposed approach.
Minor technical comments:
1. s. 3.1. predictor predicts
commas in equation (8)
2. Figure 2: no axis labels for the left plot, use for label "acceptance ratio" red color font & for label "L1 error" blue color font
3. Table 1 bracket after l_1 is missing
4. Maybe $\sigma$ is not the best designation of confidence, as it can be confused with the variance
5. Figure 1: some indexes should be not $x_{i + 2}$, but $x_{i + j}$, $x_{i + M}$. Also, some ">" before \epsilon should be "<"
6. "a auto-encoder architecture" ->
"an auto-encoder architecture"
[1] J. He et al. Probabilistic Video Generation using Holistic Attribute Control. 2018. ECCV
[2] E. Denton, R.Fergus. Stochastic Video Generation with a Learned Prior. 2018.


Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner. Recently, several frameworks have been developed which enable deep learning to be deployed in this learning scenario. A key modelling decision is to what extent the architecture should be shared across tasks. On the one hand, separately modelling each task avoids catastrophic forgetting but it does not support transfer learning and leads to large models. On the other hand, rigidly specifying a shared component and a task-specific part enables task transfer and limits the model size, but it is vulnerable to catastrophic forgetting and restricts the form of task-transfer that can occur. Ideally, the network should adaptively identify which parts of the network to share in a data driven way. Here we introduce such an approach called Continual Learning with Adaptive Weights (CLAW), which is based on probabilistic modelling and variational inference. Experiments show that CLAW achieves state-of-the-art performance on six benchmarks in terms of overall continual learning performance, as measured by classification accuracy, and in terms of addressing catastrophic forgetting.

This paper introduces CLAW, a complex but effective approach to continual learning with strong performance in the sequential task learning setting, as demonstrated on a number of standard benchmarks.
I recommend acceptance because:
- While conceptually similar to VCL, CLAW is convincingly shown to have superior performance across standard benchmarks and measures. The evaluation is thorough across the board, as far as I can tell.
- Forward transfer is shown to be substantially better compared to other methods. Experiments with long sequences of tasks (Omniglot, CIFAR-100) are particularly telling.
- Overall, the balance between not forgetting and still learning new tasks seems particularly favourable for the proposed method. This has been an elusive goal of continual learning research, hence the importance of accepting this work.
Here are some good reasons why an adversarial reviewer would reject this paper:
- The sequential task setting for continual learning has very little to add in practice, and to other branches of machine learning: it has little to say for domains where continual learning problems occur naturally, such as reinforcement learning, GAN training, multi-agent learning; all of these domains need continual learning solutions; while progress on standard benchmarks is important, we may be overfitting to these benchmarks.
- The paper is well written but the method is rather complex and presumably non-trivial to tune. This is actually characteristic of several top competing methods on these benchmarks; getting the last bit of performance seems to require this complexity, but it also makes it that much harder to generalize such methods beyond these benchmarks. For example, exploiting well partitioned datasets into different tasks and known task labels is a good starting point, but once such information is not available all bets are off. Acceptance means encouraging work on these benchmarks; is this really what we should do?
- It's perfectly tractable to store some small amount of old data for these problems in an 'episodic memory', so one could claim that an entire class of relevant baselines is missing, e.g. A-GEM, iCaRL, etc.
Luckily, I am not an adversarial reviewer, but I want to see progress across a more diverse and widely relevant set of continual learning challenges.


Graph neural networks have gained significant interest from the research community for both node classification within a graph and graph classification within a set of graphs. Attention mechanism applied on the neighborhood of a node improves the performance of graph neural networks. Typically, it helps to identify a neighbor node which plays more important role to determine the label of the node under consideration. But in real world scenarios, a particular subset of nodes together, but not the individual nodes in the subset, may be important to determine the label of a node. To address this problem, we introduce the concept of subgraph attention for graphs. To show the efficiency of this, we use subgraph attention with graph convolution for node classification. We further use subgraph attention for the entire graph classification by proposing a novel hierarchical neural graph pooling architecture. Along with attention over the subgraphs, our pooling architecture also uses attention to determine the important nodes within a level graph and attention to determine the important levels in the whole hierarchy. Competitive performance over the state-of-the-arts for both node and graph classification shows the efficiency of the algorithms proposed in this paper.

The paper proposes a novel attention approach to graph neural networks which is applicable to both of node and graph classification. The proposed method gives an attention to a subgraph instead of a node by which importance can be controlled as a set of nodes. Further, the authors also introduce two types of attention in the hierarchical structure of the network called intra- and inter- level attention.
Considering subgraph attention would be novel and a reasonable idea. The sampling-based approach is a bit naive though it would be easy to implement. Reliability of some results are not clear for me because of the small training set.
The intra- and inter- attention approach would be a reasonable, but the relation with subgraph attention is not mentioned in my understanding. These two are independent approaches? Nothing is related to each other?
The MUTAG dataset has only 188 graphs, and so, in the smallest case in Table 3, the training data only contains 188*0.15 = 28.2 graphs. For me, learning an attention neural network with less than 30 sample is difficult to evaluate. Is there any rationale that the proposed method works on such a small dataset?
In sensitivity analysis in A.1, the performance on different max subgraph size (T) is shown, and the change of the performance is moderate. One of the main claims of the paper is that considering a subgraph (not a node) increases the performance. This results does not show the increase of the performance with the increase of the subgraph size T. Showing the performance with T = 1 can be informative to verify improvement brought by the subgraph attention.
Is Figure 3 training set or test set?


Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different views of the data. This comes with several immediate problems: For example, MI is notoriously hard to estimate, and using it as an objective for representation learning may lead to highly entangled representations due to its invariance under arbitrary invertible transformations. Nevertheless, these methods have been repeatedly shown to excel in practice. In this paper we argue, and provide empirical evidence, that the success of these methods cannot be attributed to the properties of MI alone, and that they strongly depend on the inductive bias in both the choice of feature extractor architectures and the parametrization of the employed MI estimators. Finally, we establish a connection to deep metric learning and argue that this interpretation may be a plausible explanation for the success of the recently introduced methods.

The paper addresses a question on whether mutual information (MI) based models for representation learning succeed primarily thanks to the MI maximization. The motivation of the work comes from the fact that although MI is known to be problematic in treatment, it has been successfully applied in a number of recent works in computer vision and natural language processing. The paper conducts a series of experiments that constitute a convincing evidence for a weak connection between the InfoMax principle and these practical successes by showing that maximizing established lower bounds on MI are not predictive of the downstream performance and that contrary to the theory higher capacity instantiations of the critics of MI may result in worse downstream performance of learned representations. The paper concludes that there is a considerable inductive bias in the architectural choices inside MI models that are beneficial for downstream tasks and note that at least one of the lower bounds on MI can be interpreted as a triplet loss connecting it with a metric learning approach.
I consider this paper to be a considerable contribution to the understanding of what underlies the performance of unsupervised representation learning with MI maximization and provides a good discussion and analogous insights from parallel works and a number of possible directions to explore. Although I still have a couple of questions addressing which will help to understand the paper.
-  In experiment 3.1, when using RealNVP and maximizing the lower bound of MI may it be the case that the representations are learned to benefit the downstream task because of the form of the lower bounds? In other words, the lower bound is possibly not very tight and its maximization has a side effect of weights adjusting to yield simple representations useful for a linear classifier?
-  It would be interesting to see which factor contributes more to the performance: I_NCE being a triplet loss or an inductive bias in the design choice?


One of the biggest issues in deep learning theory is the generalization ability of networks with huge model size. The classical learning theory suggests that overparameterized models cause overfitting. However, practically used large deep models avoid overfitting, which is not well explained by the classical approaches. To resolve this issue, several attempts have been made. Among them, the compression based bound is one of the promising approaches. However, the compression based bound can be applied only to a compressed network, and it is not applicable to the non-compressed original network. In this paper, we give a unified framework that can convert compression based bounds to those for non-compressed original networks. The bound gives even better rate than the one for the compressed network by improving the bias term. By establishing the unified framework, we can obtain a data dependent generalization error bound which gives a tighter evaluation than the data independent ones.

The paper presents novel theoretical results on generalization bounds via compression. Similar ideas in the last few years appeared, but only bounds on a compressed network were obtained. In contrast, the current submission gives a bound on the original (uncompressed) network in terms of the complexity of the compressed network class.
Overall, the paper seems to be well-written. I appreciate that the outlines of the proofs are included in the main text, which helps the reader follow the ideas. The result is novel and quite interesting. The new bounds seem to be still quite far from giving tight generalization theory, but I believe the paper provides some nice theoretical results for other researchers to improve upon. I think the paper could be improved immensely by some empirical analysis of the rank of compressed standard vision networks and rank of activation covariance matrices.  There are also some citation issues (see detailed comments below).
Citation issues:
In the introduction, paragraph 2, the authors cite Neyshabur et al. 2019 for the observation that networks generalize well despite being overparameterized. It seems like an odd choice. Why is Barttlet’s ‘99 paper [“Size of the weights…”]  not cited? Or at least Neyshabur et al. 2015?
Then the authors mention that classical learning theory cannot explain the phenomena mentioned above, and classical theory “.. suggests ” that overparameterized models cause overfitting…”. The authors need to be more precise and add citations (I am assuming that the authors are talking about VC bounds for worst-case ERM generalization).
In the third paragraph, where the authors talk about norm-based bounds being lose, it seems that Nagarajan and Kolter 2019 should be cited (not only at the end), as well as Dzigaite and Roy 2017 (they look into the looseness of path-norm and margin-based bounds).
Could the authors comment more on how the bound in Theorem 2 is superior to VC dimension bound and whether conditions under which the bound is tight are realistic for standard compressed vision networks. Having weight matrices to be close to rank 1 seems unrealistic.I would like to see some sort of empirical evidence if the authors believe that this is the case. And for larger ranks, the bound seems to be close to VC bound.
In general, I found the notation a bit hard to follow and had to constantly be looking through the paper to find the definitions of various quantities. Having three different r’s, multiple mu’s with dots, bars, stars, etc., was definitely confusing and required extra attention to detail.
Other minor comments:
In section 2, marginal distributions over x and y are introduced. Are those used in the main text?
Is that a definition of \mu with the dot on top in assumption 5, or is this mu with the dot defined earlier? Using notation := would make it clearer whether the quantity is being defined.
In Section 3, “The main difference from the…” paragraph, there is \Psi(\dot r) used. Where is that defined?


Informed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people. We formulate this as a Bayesian Reinforcement Learning problem over latent Markov Decision Processes (MDPs). While Bayes-optimality is theoretically the gold standard, existing algorithms do not scale well to continuous state and action spaces. We propose a scalable solution that builds on the following insight: in the absence of uncertainty, each latent MDP is easier to solve. We split the challenge into two simpler components. First, we obtain an ensemble of clairvoyant experts and fuse their advice to compute a baseline policy. Second, we train a Bayesian residual policy to improve upon the ensemble's recommendation and learn to reduce uncertainty. Our algorithm, Bayesian Residual Policy Optimization (BRPO), imports the scalability of policy gradient methods as well as the initialization from prior models. BRPO significantly improves the ensemble of experts and drastically outperforms existing adaptive RL methods.

This paper considers Bayesian Reinforcement Learning problem over latent Markov Decision Processes (MDPs). The authors consider making decisions with experts, where each expert performs well under some latent MDPs. An ensemble of experts is constructed, and then a Bayesian residual policy is learned to balance exploration-exploitation tradeoff. Experiments on Maze and Door show the advantages of residual policy learning over some baselines.
1. The Bayesian Reinforcement Learning problem this work considered is important. However, using experts immediately make the problem much easier. The original Bayesian Reinforcement Learning problem is then reduced to making decision with experts. Under this setting, there are many existing work with respect to exploration-exploitation tradeoff (OFU, Thompson Sampling) with theoretical guarantees. I did not see why using this residual policy learning (although as mentioned residual/boosting is useful under other settings) is reasonable here. There is not theoretical support showing that residual learning enjoys guaranteed performance. The motivation of introducing this heuristic is not clear.
2. The comparisons with UPMLE and BPO seems not convincing. Both BPO and UPMLE do not use experts, and ensemble of experts outperforms them as shown in the experiments. And the ensemble baseline here is kind of weak (why sensing with probability 0.5 at each timestep?) Always 0.5 does not make sense (exploration should decrease as uncertainty reduced). Other exploration methods should be compared, to empirically show the advantages/necessities of residual policy learning.
Overall, I consider the proposed BRPO a simple extension of BPO, with a heuristic of learning ensemble policy to make decisions. BRPO is lack of theoretical support, and it is not clear why residual policy learning here is necessary and what exactly the advantage is over other exploration methods. Comparisons with simple baseline like exploration with constant probability is not enough to justify the proposed method.
=====Update=====
Thanks for the rebuttal. The comparison with PSRL improves the paper. However, I still think this paper needs more improvement as follows.
Theorem 1 looks hasty to me. Batch policy optimization Alg is going to solve n_{sample} MDPs, which are generated from P_0. But Eq. (6) or Theorem 1 does not contain information about P_0, implying that P_0 has no impact, which is questionable (an uniform P_0 that can generate different MDPs and a deterministic P_0 can only generate one MDP should be very different). I suggest the authors do more detailed analysis.
On the other hand, I expected whether this special "residual action" heuristic has any guarantees in RL? Can decomposing action into a_r + a_e provide us a better exploration method (than others like PSRL, OFU...)? Since this is the main idea of this paper as an extension of BPO, I think this point is important. The experiments shows that it can work in some cases, but I do not see an explanation (the "residual learning" paragraph is high level and I do not get an insight from that.).


Adaptive optimization algorithms such as RMSProp and Adam have fast convergence and smooth learning process. Despite their successes, they are proven to have non-convergence issue even in convex optimization problems as well as weak performance compared with the first order gradient methods such as stochastic gradient descent (SGD). Several other algorithms, for example AMSGrad and AdaShift, have been proposed to alleviate these issues but only minor effect has been observed. This paper further analyzes the performance of such algorithms in a non-convex setting by extending their non-convergence issue into a simple nonconvex case and show that Adam's design of update steps would possibly lead the algorithm to local minimums. To address the above problems, we propose a novel adaptive gradient descent algorithm, named AdaX, which accumulates the long-term past gradient information exponentially. We prove the convergence of AdaX in both convex and non-convex settings. Extensive experiments show that AdaX outperforms Adam in various tasks of computer vision and natural language processing and can catch up with SGD.

This paper introduces a new step-size adaptation algorithm called AdaX. AdaX
builds on the ideas of the Adam algorithm to address instability and non-convergence issues.
Convergence of AdaX is proven in both convex and non-convex settings. The paper
also provides an empirical comparison of AdaX against its predecessors
(SGD, RMSProp, Adam, AMSGrad) on a variety of tasks.
I recommend the paper be rejected. I believe the convergence results could be a significant contribution, but the
quality of the paper is hampered by its experimental design. The paper felt generally unpolished, containing
frequent grammatical errors, imprecise language, and uncited statements.
My main issue with the paper is the experimental design. I am not convinced that we can
draw valid conclusions from the experimental results for the following reasons:
  - The experiments are lacking important details. How many independent runs of the experiment
    were the experimental results averaged over? All of the experiments have random initial conditions
    (e.g. initialization of the network), and should be ran multiple times, not just once.
    There's no error bars in any of the plots, so it's unclear whether AdaX really
    does provide a statistically significant improvement over the baselines.
    Similarly, the data in all the tables is quite similar, so without indicating the
    spread of these estimates its impossible to tell whether these results are significant or not.
  - How were the hyperparameters and step-size schedules chosen? The performance of Adam, AMSGrad, and
    RMSProp are quite sensitive to their hyperparameters, and the optimal hyperparameters are problem-dependent.
    Some of the experiments just use the default hyperparameters; this is insufficient when trying to directly
    compare the performance of these methods, as their performance can vary greatly with different values of
    these parameters. I'm not convinced that we should be drawing conclusions about the relative
    performance of these algorithms from any of the experiments for this reason.
Of course, meaningful empirical results are not necessarily characterized by statistically outperforming the baselines.
Well designed experiments can highlight important ways in which the performances differ, providing the community
with a deeper understanding of the methods investigated. I would argue that the experiments in the paper do not
achieve this either; the experiments do not provide any new intuition or understanding of the methods, showing
only the relative performances in terms of learning curves on a somewhat random collection of supervised learning problems. Why were these specific problems chosen? What makes these problems ideal for showcasing the performance of AdaX? If AdaX is an improvement over Adam, why? What exactly is happening with it's effective step-sizes that leads
to the better performance? Can you show how their step-sizes differ over time?
Statements that need citation or revision:
  - "Adaptive optimization algorithms such as RMSProp and Adam... as well as weak performance
     compared to the first order gradient methods such as SGD" (Abstract). This needs a citation.
     Similarly, "AdaX outperforms various tasks of computer vision and natural language processing and can catch
     up with SGD"; as above, I'm unaware of work (other than theoretical) that shows that SGD significantly
     outperforms Adam in deep neural networks.
  -  "In the era of deep learning, SGD ... remains the most effective algorithm in training deep neural
      networks" (Introduction). What are you referring to here? Vanilla SGD? Or are you including Adam etc here?
      As above, this should have a citation. Adam's popularity is largely due to its effectiveness in training
      deep neural networks.
  - "However, Adam has worse performance (i.e. generalization ability in testing stage) compared with SGD"
     (Introduction). Citation needed.
  - In the last paragraph of the Introduction, you introduced AdaX twice: "To address the above issues, we propose a
    new adaptive optimization method, termed AdaX, which guarantees convergence...", and, "To address
    the above problems, we introduce a novel AdaX algorithm and theoreetically prove that it converges..."


State-of-the-art quantization methods can compress deep neural networks down to 4 bits without losing accuracy. However, when it comes to 2 bits, the performance drop is still noticeable. One problem in these methods is that they assign equal bit rate to quantize weights and activations in all layers, which is not reasonable in the case of high rate compression (such as 2-bit quantization), as some of layers in deep neural networks are sensitive to quantization and performing coarse quantization on these layers can hurt the accuracy. In this paper, we address an important problem of how to optimize the bit allocation of weights and activations for deep CNNs compression. We first explore the additivity of output error caused by quantization and find that additivity property holds for deep neural networks which are continuously differentiable in the layers. Based on this observation, we formulate the optimal bit allocation problem of weights and activations in a joint framework and propose a very efficient method to solve the optimization problem via Lagrangian Formulation. Our method obtains excellent results on deep neural networks. It can compress deep CNN ResNet-50 down to 2 bits with only 0.7% accuracy loss. To the best of our knowledge, this is the first paper that reports 2-bit results on deep CNNs without hurting the accuracy.
In this supplementary material, we provide the mathematical analysis for the additivity of output error, and the additional results of Pareto-optimal bit allocation. Section 1 provides the proof of Proposition 1. Section 2 provides the results of Pareto-optimal bit allocation on ResNet-50  and MobileNet-v1 Howard et al. (2017) at different bitrates.

Very good paper that studies the error rate of low-bit quantized networks and uses Pareto condition for optimization to find the best allocation of weights over all layers of a network. The theoretical claims are strongly supported by experiments, and the experimental analysis covers state-of-the-art architectures and demonstrates competitive results. The paper in addition also analyzes the inference cost of their approach (in addition to the accuracy results), and shows positive results on ResNet and MobileNet architectures.
The paper primarily shows that the mean squared error of the final output of a quantized network has the additive property of being equal to the sum of squared errors of the outputs obtained by quantizing each layer individually. Although there is no reason why this should be case, experimental results from the authors on AlexNet and VGG-16 validate this. Based on this assumption, the authors then use a Lagrangian based constrained optimization to minimize the sum of squared errors of outputs when individual weights/activations are quantized, with the constraint being the total bit budget for weights and activations. The authors show that this can be optimized under the Pareto condition easily.
The experimental section is quite detailed and covers the popular architectures instead of toy ones. The accuracy results compared to other 2-bit and 4-bit approaches are competitive. It's also nice to see analysis of inference cost where unequal bitrate allocation performs better than other methods.
The authors show that given the constrained optimization, layers that have a large number of weights receive lower bitrates and vice-versa. While it makes sense that this would contribute to stronger inference speedup compared to methods with either equal bitrate allocation across layers or those that allocate higher bitrate to layers with large number of weights, it's not entirely clear why the optimization would produce this allocation in the first place. Do the authors mean to conclude that layers with large number of weights hold a lot of redundancy and don't have a significant impact on the overall accuracy of the model? This needs to be clarified further.


Unordered, variable-sized inputs arise in many settings across multiple fields. The ability for set-and multiset-oriented neural networks to handle this type of input has been the focus of much work in recent years. We propose to represent multisets using complexweighted multiset automata and show how the multiset representations of certain existing neural architectures can be viewed as special cases of ours. Namely, (1) we provide a new theoretical and intuitive justification for the Transformer model's representation of positions using sinusoidal functions, and (2) we extend the DeepSets model to use complex numbers, enabling it to outperform the existing model on an extension of one of their tasks.

This paper proposed a complex weights based multiset automata designed to represent unordered data. The main idea of multiset automata is that the transition matrices of the automata is pairwise commutative. To achieve this property, the authors proposed to restrict the transition matrices to be diagonal and shows that the latter is a close approximation of the former. The authors proceed to give two practical applications of the multiset automata: position encoding of the transformer and deepset networks. For the former, the authors showed that the position encodings from Vaswani et al. can be written as a weighted unary automaton and therefore it is a generalization of the original position encodings. For the latter, the authors extended the classical deepset networks into its complex domain, allowing more efficient representation of the data.
I think this paper overall did a good job, and I really like the construction of the multiset automata and the theoretical guarantees the authors derived and the two applications are straight-forward to see. However I do find the motivation of this paper is a bit weak,  and I’m having a hard time finding the highlight of the paper. Therefore, I’m giving this paper a weak accept.
Here are some general comments:
How is the multiset automata learnt? For weighted automata, one classical way is to use spectral learning algorithm (see Balle et. al. 2014). In this paper, the learning aspect of the multiset automata was not mentioned. I assume that the authors use some kind of gradient descent to optimize the weights w.r.t the whole networks. However, I do think it’s important to let the readers know this key step.
For the first experiment on the position encoding. I really like the derivation here and it seems that theoretically, multiset automata should be a generalization of the position encodings. However, the experiments didn’t show much difference. Is there any potential explanation here? Moreover, what is the advantage of using multiset automata in transformers instead of the original position encoding? Is it the runtime is faster? Cause you only need to compute the diagonal parameters and thus drastically reduce the number of parameters? If so, a comparison of runtime might be useful here to further showcase the advantage of the multiset automata.
For the first application, it is great that the authors shows the connection, but it seems that it just stops at the level of showing the position encodings can be viewed as a multiset automata. For example, what happens if you don’t restrict it to be sinusoidal functions? e.g. set the transitions to be diagonal and directly optimize with gradient descent?
For the second experiment, I’m a little confused about the experiment setup and the baselines. First how do the authors incorporate multiset automata into the deepset models? In Figure 1, every neural architectures have this embedding layers with diagonal transition matrices, does this mean the multiset automata is applied to encode the input for every architecture? If so is it possible that the use of multiset automata in LSTM, GRU and deepset, is actually hurting the performance? Maybe a comparison with vanilla LSTM, GRU and deepset is also needed.
Moreover, for the second experiment, the size of LSTM, GRU, deepset seems to be smaller comparing to the complex product layer in the authors’ architecture (about half of the size). It is true that the authors mentioned that with non-unary automata, the size of the automata is significantly larger, but is this set-up fair for the baselines, e.g. if you use 150 size of LSTM, will it perform equally well to the multiset automata?
In addition, I have a bit of difficulty understanding why three embedding layers are needed to learn the complex number, is it possible to learn r, a, b jointly with one embedding?
Is there some real data experiment done for the second application (the extension to deepset), to further showcase the significance of using complex domain?
Here are some writing comments (did not affect the decision):
In page 6, the bullet point “Diagonal polar”, what does (#1 above) mean? Same goes for (#2 above) in the latter point.
In “Full matrix”, the sentence “…, and transition matrix using orthogonal initialization…” does not have a verb and is a bit confusing to read.
Page 7, second paragraph, line 4, in the brackets. There are two sentences in the brackets, and it feels a bit heavy and it actually says something important. Maybe either put it out or leave it as a footnote?
Overall, I like the concept of the multiset automata, but I feel there is a lack of highlights to further showcase this paper. I feel maybe a further investigation of either of these application could make a great paper.


Decision-time planning policies with implicit dynamics models have been shown to work in discrete action spaces with Q learning. However, decision-time planning with implicit dynamics models in continuous action space has proven to be a difficult problem. Recent work in Reinforcement Learning has allowed for implicit model based approaches to be extended to Policy Gradient methods. In this work we propose Policy Tree Network (PTN). Policy Tree Network lies at the intersection of Model-Based Reinforcement Learning and Model-Free Reinforcement Learning. Policy Tree Network is a novel approach which, for the first time, demonstrates how to leverage an implicit model to perform decision-time planning with Policy Gradient methods in continuous action spaces. This work is empirically justified on 8 standard MuJoCo environments so that it can easily be compared with similar work done in this area. Additionally, we offer a lower bound on the worst case change in the mean of the policy when tree planning is used and theoretically justify our design choices.

The paper proposes a modification to Policy Prediction Networks (PPN) in which the learned transition-, reward- and value function models are used at test-time in a planning procedure.
A second contribution is the "pi-Q-backup" which uses the geometric mean of both the policy and the value function as maximisation target of the planning step.
 Overall, I find the idea interesting and the experimental evaluations promising. However, I am voting for "weak reject" for the reasons outlined below. If some (or all) of them are address, I'd be happy to raise my score.
- I found the paper hard to understand. In particular, the algorithm PPN on which this work is build is not explained at all, requiring the reader to read the original PPN paper. Including a description of PPN, including it's main features, would greatly help the paper. Second, I am still not sure I correctly undestand when each component is used. As I currently understand it, the main usage of the model during training time is to compute the Advantages in equation (2)? Or are those computed based on rollouts? If so, where is the model actually being used during training?
- Figure 1 is taken directly from the PPN paper without any reference or citation (as far as I can tell).
- For the comparison in Figure 5, it would be great if PPN could also be tested with the newly introduced parameter \beta_i. At the moment, it is hard to tell whether the performance gains are due to \beta or due to the proposed planning scheme.
- I'm confused about Theorem 1: Wouldn't we want an upper bound on the difference of means?  Also, what does 'worst-case' mean? Is that for the 'worst' Q-function we could choose?
Edit:
Thank you for your comments and updated manuscript.
I think the writing has improved significantly, but could still be further improved and clarified. In particular, the question of how the model and other components at various points in time could be made more obvious. I found the authors' response to R3 here helpful as well.
At least for me some of the confusion arises not due to the complexity of the proposed approach, but just because combining real and 'simulated' transitions can be used and mixed in so many different ways that it's important to be clear about it. Also, at least personally, I found the explanation "Learning is done with a model-based approach that follows the behaviour policy’s rollout trajectory. However, the test policy follows a model-based approach" still not very helpful.
Overall, I think the presentation is on a good way but needs some more work.
With that being said and now having a better understanding of the algorithm I think this is very interesting work. However, I share R1's concerns about the computation of the \pi-Q backup, in particular that it seems arbitrary and doesn't handle negative values. I'm also not convinced that adding |min(Q)| is a good solutions as a) we don't always have access to that value and b) If I'm not wrong, than \pi_F is not invariant under a shift of Q.
I'm wondering why the authors decided to take the geometric mean instead of following the more typically used approach of using exp(Q/Temperature)*pi to combine Q-function and a policy distribution (see e.g. the "Control as Inference" literature, the "Maximum a Posteriori Policy Optimisation" or "Soft Actor Critic" algorithms, in particular the "Soft Value functions". I think this should at the very least be an ablation study, and could be even performing better and at the very least be robust to negative values.
Overall, I think this is very interesting work and could become a very strong paper, but I will remain to recommend a "weak reject" because I think it needs some more work to get there.


Federated learning makes a large amount of edge computing devices jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (FedAvg) runs Stochastic Gradient Descent (SGD) in parallel on a small subset of the total devices and averages the sequences only once in a while. Despite its simplicity, it lacks theoretical guarantees under realistic settings. In this paper, we analyze the convergence of FedAvg on non-iid data and establish a convergence rate of O( 1 T ) for strongly convex and smooth problems, where T is the iteration number of SGDs. Importantly, our bound demonstrates a trade-off between communication-efficiency and convergence rate. As user devices may be disconnected from the server, we relax the assumption of full device participation to partial device participation and study different averaging schemes; low device participation rate can be achieved without severely slowing down the learning. Our results indicate that heterogeneity of data slows down the convergence, which matches empirical observations. Furthermore, we provide a necessary condition for FedAvg's convergence on non-iid data: the learning rate η must decay, even if full-gradient is used; otherwise, the solution will be Ω(η) away from the optimal.

Federated learning is distinguished from the standard distributed learning in the following sense:
1) training is distributed over a huge number (say N) of devices and communication between the central server and devices are slow.
2) The central server has no control of individual devices, and there are inactive devices that does not respond to the server; full participation of all devices is unrealistic.
3) The local data distribution at each device is different from each other; i.e., the data is non-iid.
Due to property 1), communication-efficient algorithms such as Federated Averaging (FedAvg) have been proposed and studied. FedAvg runs SGD in parallel on K (≤N) local devices using their local datasets, and updates the global parameter after E local iterations by aggregating the updates from the local devices.
Properties 2) and 3) makes analysis of FedAvg difficult, and previous results have proven convergence of FedAvg assuming that the data is iid and/or all devices are active. In contrast, this paper studies FedAvg on the non-iid data and inactive devices setting and shows that, with adequately chosen aggregation schemes and decaying learning rate, FedAvg on strongly convex and smooth functions converges with a rate of O(1/T).
Overall, I enjoyed reading this paper and I would like to recommend acceptance. This is the first result showing convergence rate analysis of FedAvg under presence of properties 2) and 3), which is a nontrivial, important, and timely problem. The paper is well-written and reads smoothly, except for some minor typos. The convergence bounds provide insights of practical relevance, e.g., the optimal choice of E, the effect of K in convergence rate, etc. The authors also provide empirical results supporting their theoretical analysis.
Some questions I have in mind:
- What is "transformed Scheme II"? Is it the scaling trick described at the end of Section 3.3? The name appears in the experiment section before being defined.
- What happens if we choose \eta_t that is decaying but slower than O(1/t), say O(1/\sqrt t)? Can convergence be proved? If so, in what rate?
Minor typos:
- Footnote 3: know -> known
- Assumptions 1 & 2: f in $f(w)$ is math-bold
- Choice of sampling schemes: "If the system can choose to active..." -> activate
- mnist balanced and mnist unbalanced: the description after them suggests they should be switched
- Apdx D.1: widely -> wide, summary -> summarize


Neural network has attracted great attention for a long time and many researchers are devoted to improve the effectiveness of neural network training algorithms. Though stochastic gradient descent (SGD) and other explicit gradient-based methods are widely adopted, there are still many challenges such as gradient vanishing and small step sizes, which leads to slow convergence and instability of SGD algorithms. Motivated by error back propagation (BP) and proximal methods, we propose a semi-implicit back propagation method for neural network training. Similar to BP, the difference on the neurons are propagated in a backward fashion and the parameters are updated with proximal mapping. The implicit update for both hidden neurons and parameters allows to choose large step size in the training algorithm. Finally, we also show that any fixed point of convergent sequences produced by this algorithm is a stationary point of the objective loss function. The experiments on both MNIST and CIFAR-10 demonstrate that the proposed semi-implicit BP algorithm leads to better performance in terms of both loss decreasing and training/validation accuracy, compared to SGD and a similar algorithm ProxBP.

The work is based on the recent paper 'Proximal Backpropagation', Frerix et al., ICLR 2018, which views error back propagation as block coordinate gradient descent steps on a penalty functional comprised of activations and parameters.
Instead of taking proximal steps on the linear layers as in (Frerix et al., 2018), the authors also pull the non-linearity \sigma into the proximal steps. Another interesting deviation is the idea to consider the newly updated weights {W_i}^{k+1} when updating the activations F_i^{k+1} in the backward pass.
While potentially offering a faster convergence with respect to epochs, the nonlinear updates have two major drawbacks:
1) While there are preliminary theoretical results (fixed points of the method are critical points), it remains unclear whether the computed update is still a descent direction on the original energy.  While not crucial, such a result would be reassuring and might give further insights into the method.
2) Each update requires the solution of a nonconvex, nonlinear least squares problem which is prohibitively expensive to solve. Note that such nonlinear least squares updates are already proposed in (Carreira-Perpinan & Wang, 2014). When using ReLU activation, the non smoothness might be an issue for standard nonlinear least squares solvers such as Levenberg-Marquadt.
Furthermore, the numerical results are unfortunately a bit discouraging. The experiments evaluate toy models on toy datasets and even there only a minor improvement with respect to epochs over SGD and Prox-BP is shown. Furthermore, the plots only consider epochs and not the running time. Due to the non-linear least squares problem, I assume that each epoch for the proposed method is way more costly. Therefore I consider the experimental evaluation too preliminary. A proper evaluation would require an implementation as an optimizer in state-of-the-art deep learning frameworks and a comparison with respect to running time to standard optimizers such as SGD with momentum or Adam on the GPU.
The reported performances for MNIST are surprisingly poor. Note that vanilla SGD with momentum reaches ~98.6% test set performance on such an architecture, while the overall highest reported accuracy in this paper is 98.0%. This might be due to momentum, and it would be interesting whether the proposed method could be combined with momentum or other optimizers such as Adam as in (Frerix et al. 2018).
Overall, I don't see this a practical algorithm for training deep networks and there are few theoretical results. Therefore, I cannot recommend acceptance at this stage.
To improve the paper, I would like to see an implementation of the method on the GPU in a recent deep learning framework and an evaluation on larger models / datasets. But I am doubtful this will reach competitive performance to standard optimizers. Also, it would be interesting to see how the precision in the inner nonlinear conjugate gradient solver effects outer convergence. It might be that the subproblem does not have to be solved with very high accuracy.
Minor comments:
* Missing citation: 'Difference Target Propagation', https://arxiv.org/abs/1412.7525, studies a similar type of algorithm.


Graph Neural Networks as a combination of Graph Signal Processing and Deep Convolutional Networks shows great power in pattern recognition in nonEuclidean domains. In this paper, we propose a new method to deploy two pipelines based on the duality of a graph to improve accuracy. By exploring the primal graph and its dual graph where nodes and edges can be treated as one another, we have exploited the benefits of both vertex features and edge features. As a result, we have arrived at a framework that has great potential in both semisupervised and unsupervised learning.

This paper utilizes two pipelines based on the duality of a graph and the primal graph for semi-supervise learning. More specifically, the authors transform the primal graph into its dual form and build a two-pipelines architecture to learn on the primal graph and dual graph together. The goal of including a dual pipeline is to use the predictions on the dual node to affect the predictions of the primal nodes.
I decided to give a weak reject to this paper for the following shortcomings:
1. Novelty is not enough. Dual graphs are explored utilized to graph neural networks in some related works (such as [1]). This paper utilizes dual graphs to affect or assist the prediction on the primal nodes, which is very similar to the methods used in [1].
2. Experiments can not prove the effectiveness of this method very well. The experiments only conduct on 3 datasets and the analysis of the experimental results is not convincing and more details should be included. For example, if GCN(double-pipeline) performs better than GCN, this can somehow support the effectiveness of a double-pipeline that includes a dual graph. However, the results of TwinGCN can not support the effectiveness of regularization by KL-divergence. I think more experiments and analyses about why this method works should be explored in Section Experiments.
3. The organization should be improved. In general, when introducing a model, it is better to introduce the prediction part of the model first. For example, I can get the idea of how to train the DualGCN by reading the Section Method. However, I have no idea about how the authors make a prediction on the trained DualGCN, how to transform the prediction of the dual graph to the prediction of primal nodes. I think these details are significant for this paper. Also, there are several incomplete sentences in this paper.
Overall, this idea that utilizes the dual graph to assist the primal graph learning is good and can be explored in the future. For the current version, I give a weak reject for the above reasons.

Reference
[1] Chen et al. Supervised community detection with line graph neural networks.


Continual learning in neural networks (NN) often suffers from catastrophic forgetting. That is, when learning a sequence of tasks on an NN, the learning of a new task will cause weight changes that may destroy the learned knowledge embedded in the weights for previous tasks. Without solving this problem, it is difficult to use an NN to perform continual or lifelong learning. Although researchers have attempted to solve the problem in many ways, it remains to be challenging. In this paper, we propose a new approach, called principal components projection (PCP). The idea is that in learning a new task, if we can ensure that the gradient updates will only occur in the orthogonal directions to the input vectors of the previous tasks, then the weight updates for learning the new task will not affect the previous tasks. We propose to compute the principal components of the input vectors and use them to transform the input and to project the gradient updates for learning each new task. PCP does not need to store any sampled data from previous tasks or to generate pseudo data of previous tasks and use them to help learn a new task. Empirical evaluation shows that the proposed method PCP markedly outperforms the state-of-the-art baseline methods.

The method proposes a method for continual learning. The method is an extension of recent work, called orthogonal weights modification (OWM) [Zheng,2019]. This method aims to find gradient updates which are perpendicular to the input vectors of previous tasks (resulting in less forgetting). However, the authors argue, that the learning of new tasks is happening in the solution space of the previous tasks, which might severely limit the ability to adapt to new tasks. The authors propose a ‘principal component’-based solution to this problem. The method is considering the ‘task continual learning’ scenario (also known as task-aware) which means that the task label is given at inference time.
Conclusion:
1. The paper is not well-positioned in related works. I think the work is more related to works with ‘parameter isolation methods’ such as Piggyback, Packnet, HAT. These methods reserve part of the capacity of the network for tasks. I think the authors should relate their work with these methods, and provide an argument of the problem with these previous methods, which is addressed by their approach. I can see that rather than freezing weights (PackNet) or features (HAT) , the method freezes linear combinations of features. But it is for me not directly clear that that is desirable. In HAT the backpropagated vector is projected on the mask vector which coincides with the neurons (activations).
2. The experimental verification of the paper is too weak, and only comparison to EWC and OWM (not well known) are provided. At least a comparison with the more related works PackNet and HAT should be included. For more recent method for task-aware CL see also ‘Continual learning: A comparative study on how to defy forgetting in classification tasks’. Also results seem bad. For example on CIFAR10, 5 tasks in TCL setting is two-class problem per task; I would expect better results.
3. The authors claim that OWM is effective if tasks are similar, but not when dissimilar. And the proposed PCP solves this problem. However, all experiments are on similar tasks, and no cross domain tasks are considered, e.g. going from MNIST (task1) to EMNIST-26 (task2) etc. This would empirically support the claim. Also, the authors expect the difference between PCP and OWM to be even larger then.
4. Some more analysis of the success of PCA in representing the distribution would be appreciated, e.g. the percentage of total energy which is captured (sum of selected eigenvalues divided by sum of all eigenvalues). Such an analysis of P_l^k as a function of the tasks (and for several layers) would be interesting to see, for example for EMNIST-47(10 tasks).
5. Novelty with respect to OWM is rather small.
6. The authors should mention that the method is pretrained on ImageNet in section 4.3. Given these datasets, I think it makes more sense to train from scratch and I would like to see those results.
Minor remarks:
- I wonder if you use OWM or PCP you discard the possibility of positive backward transfer. Maybe the authors could comment on that.
- The authors write that ‘TCL setting the classification results are usually better than those of the CCL’ is that not per definition true ? Anything correctly classified under CCL is correctly classified under TCL but not the other way around.


As gradual typing becomes increasingly popular in languages like Python and Typescript, there is a growing need to infer type annotations. While type annotations help with tasks like code completion and static error catching, these annotations cannot be fully inferred by compilers and are tedious to annotate by hand. This paper proposes a probabilistic type inference scheme for Typescript based on a graph neural network. Our approach first uses lightweight source code analysis to generate a program abstraction called a type dependency graph, which links type variables with logical constraints as well as name and usage information. Given this program abstraction, we then use a graph neural network to propagate information between related type variables and eventually make type predictions. Our neural architecture can predict both standard types, like number or string, as well as user-defined types that have not been encountered during training. Our experimental results show that our approach outperforms prior work in this space by 14% (absolute) on library types, while having the ability to make type predictions that are out of scope for existing techniques.

This paper proposed to use Graph Neural Networks (GNN) to do type inference for dynamically typed languages. The key technique is to construct a type dependency graph and infer the type on top of it. The type dependency graph contains edges specifying hard constraints derived from the static analysis, as well as soft relationships specified by humans. Experiments on type predictions for TypeScript have shown better performance than the previous methods, with or without user specified types.
Overall this paper tackles a nice application of GNN, which is the type prediction problem that utilizes structural information of the code. Also the proposed type dependency graph seems interesting to me. Also the pointer mechanism used for predicting user specified types is a good strategy that advances the previous method. However, I have several concerns below:
About formulation:
1) I’m not sure if the predicted types for individual variable would be very helpful in general. Since the work only cares about individual predictions while no global consistency is enforced, it is somewhat limited. For example, in order to (partially) compile a program, does it require all the variable types to be correct in that part? If so, then the predicted types here might not be that helpful. I’m not sure about this, so any discussion would be appreciated.
About type dependency graph:
1) Comparing to previous work (e.g, Allamanis et.al, ICLR 18), it seems the construction of the task specific graph is the major contribution, where the novelty is a bit limited.
2) The construction of the dependency graph is heuristic. For example, why the three contextual constraints are good? Would there be other good ones? Also why only include such limited set of logical constraints. For example, would expression like (x + y) induce some interesting relationships? Because such hand-crafted graph is lossy (unlike raw source code), all the questions here lead to the concern of such design choices.
3) The usage of graph is somewhat straightforward to me. For example, although the hard-constraints are there, there’s no such constraints reflected in the prediction. Adding the constraints on the predictions would be more interesting.
About experiments:
1) I think one ablation study I’m most interested in is to simply run GNN on the AST (or simply use Allamanis et.al’s method). This is to verify and support the usage of proposed type dependency graph.
2) As the authors claimed in Introduction, ‘plenty of training data is available’. However in experiment only 300 projects are involved. Also it seems that these are not fully annotated, and the ‘forward type inference functionality from TypeScript’ is required to obtain labels. It would be good to explain such discrepancy.
3) Continue with 2), as the experiment results shown in Table 2, TS compiler performs poorly. So how would it be possible to train with poor annotations, while generalize much better? Some explanations would be helpful here.
4) I think only predicting non-polymorphic types is another limitation. Would it be possible to predict structured types? like nested list, or function types with arguments?


Existing methods for AI-generated artworks still struggle with generating high-quality stylized content, where high-level semantics are preserved, or separating fine-grained styles from various artists. We propose a novel Generative Adversarial Disentanglement Network which can disentangle two complementary factors of variations when only one of them is labelled in general, and fully decompose complex anime illustrations into style and content in particular. Training such model is challenging, since given a style, various content data may exist but not the other way round. Our approach is divided into two stages, one that encodes an input image into a style independent content, and one based on a dual-conditional generator. We demonstrate the ability to generate high-fidelity anime portraits with a fixed content and a large variety of styles from over a thousand artists, and vice versa, using a single end-to-end network and with applications in style transfer. We show this unique capability as well as superior output to the current state-of-the-art.

The paper proposes an image generation method with the focus on generating anime faces from various artists. The proposed method which is a combination of conditional GANs and conditional VAEs manages to generate high fidelity anime images with various styles.
The paper is well-written and easy to follow and understand. The goals are clearly stated and the background (which is more of history), as well as related work, is comprehensive. The decision behind every design decision has been mentioned in detail which makes the paper stronger. The main writing flaw of the paper is in the figures where more annotation and caption is required to make them easy to understand. For example:
- The significance of colors in Figure 1 (architecture of the model) is not annotated at all and the letters are not clear either (although they are described in the text itself a figure should be comprehensive by itself).
- Figure 2 and Figure 4 are really hard to understand with very limited annotation and caption. I had to read the text multiple time to Figure out what is what in these figures which is not a good sign for clarity.
In terms of experiments, I think where the paper suffers the most is in comparison with other conditional methods. In Section 5 it has been clearly mentioned that "this result can be expected from a class-conditional GAN and the focus in on Disentanglement" however very little evidence has been provided for superior disentanglement. More experiments are required to demonstrate the capabilities of the model compared to other conditional methods (which is currently only limited to StarGAN) as well as its capability of disentanglement. I agree with the authors that quantitative evaluation of generated anime faces is not easy (although it is possible with a carefully designed human study), however, the disentanglement (which is the focus of the paper) is easy to evaluate quantitatively. This demands for more experiments on disentanglement datasets with known generative factors. Although the current ablation study in the Appendix provides more details for architectural decisions, a more qualitative and quantitative comprehensive ablation study (by actually ablating the final model) can help to demonstrate these decisions.
In conclusion, the paper has great results. We all know a big part of writing this kind of paper is to make the model "work" and authors truly demonstrate that they worked hard. However, the impact of the paper (in the current form) is not clear. With the focus on disentanglement, little evidence has been provided to justify the capability of the proposed method. I believe by addressing my comments on the experiments the paper can be easily pushed above the acceptance bar. Also releasing the code dataset should increase the impact of the paper.


Data augmentation(DA) is a useful technique to enlarge the size of the training set and prevent overfitting for different machine learning tasks when training data is scarce. However, current data augmentation techniques rely heavily on human design and domain knowledge, and existing automated approaches are yet to fully exploit the latent features in the training dataset. In this paper we propose an adaptive DA strategy based on generative models, where the training set adaptively enriches itself with sample images automatically constructed from deep generative models trained in parallel. We demonstrate by experiments that our data augmentation strategy, with little model-specific considerations, can be easily adapted to cross-domain deep learning/machine learning tasks such as image classification and image inpainting, while significantly improving model performance in both tasks.

This paper proposes an method based on multiple GANs trained on different splits of the training data in order to generate additional samples to include to the training data. The number of added samples from each GAN depends on the quality of the generation based on their inception scores. This approach helps to improve the performance of a convolutional neural network on different tasks such as image classification, image generation and image inpainting.
I rated this paper as weak reject because this paper is weak on several aspects
- In related work many similar approaches are missing (see below).
- In the methodology the actual formulation for the GAN is not presented.
- The main novelty of the paper seems to be the use of multiple GAN on different splits of the data, which seems a bit limited.
- The experimental evaluation is limited (see below).
Related work:
Several works with very similar intentions have been overlooked:
("A Bayesian Data Augmentation Approach for Learning Deep Models, Toan Tran, Trung Pham, Gustavo Carneiro, Lyle Palmer, Ian Reid) uses GAN during training for generating samples for data augmentation.
("Dada:  Deep adversarial data augmentation for extremely low data regime classification", Xiaofeng Zhang,  Zhangyang Wang,  Dong Liu,  and Qing Ling) uses another GAN model for generating samples on low data regime for data augmentation
("Adversarial Learning of General Transformations for Data Augmentation", Saypraseuth Mounsaveng, David Vazquez, Ismail Ben Ayed, Marco Pedersoli), uses GAN for generating samples for data augmentation on reduced datasets.
(Triple Generative Adversarial Nets, Chongxuan Li, Kun Xu, Jun Zhu, Bo Zhang) uses a gan model for generating samples for semi-supervised learning with few labelled samples.
(Triangle Generative Adversarial NetworksZhe Gan∗, Liqun Chen∗, Weiyao Wang, Yunchen Pu, Yizhe Zhang,Hao Liu, Chunyuan Li, Lawrence Carin) in which again GAN is used to improve in semi-supervised settings.
Experimental evaluation:
In the experimental evaluation the importance of the use of k-fold is shown only on Fig.4. Additionally, in table 1 the method is compared with weak baselines (no DA, flip, crop, but not the combination of flip and crop which is standard on CIFAR10) and it is not compared with any other approach (and there are several as shown in related work).
For generating images, no quantitative values are provided, just some generation examples.


We introduce a sparse scattering deep convolutional neural network, which provides a simple model to analyze properties of deep representation learning for classification. Learning a single dictionary matrix with a classifier yields a higher classification accuracy than AlexNet over the ImageNet 2012 dataset. The network first applies a scattering transform that linearizes variabilities due to geometric transformations such as translations and small deformations. A sparse 1 dictionary coding reduces intra-class variability while preserving class separation through projections over unions of linear spaces. It is implemented in a deep convolutional network with a homotopy algorithm having an exponential convergence. A convergence proof is given in a general framework that includes ALISTA. Classification results are analyzed on ImageNet.

The work considers a new architecture for artificial neural networks to be used in image classification tasks. The network combines several promising previous research directions into an interesting approach (to the best of my knowledge, the architecture is novel).
In the first step, the input representation (i.e., image) is processed using deep scattering spectrum. This is an operator proposed by Mallat (2012) and it is known for extracting robust features with properties such as translation invariance and Lipschitz continuity of the operator mapping. As a result, such a representation is likely to be more stable to perturbations and noise in the inputs.
The scattering operator of the second and higher orders tends to produce a large number of coefficients. Such a representation is, in general, sparse with many of the higher order coefficients equal to zero. To lower the dimension of the scattering representation, the architecture employs a linear projection operator. For example, one can take principal component analysis and project to a sub-space retaining most of the variation present in the data.
The first two blocks (scattering and linear projection) are unsupervised and, thus, kept fixed during learning.
The third block in the architecture aims at finding a sparse coding dictionary that will take into account instances and labels. It builds a dictionary using a convolutional neural network and finds sparse coding using previous work on dictionary learning (e.g., Donoho & Elad, 2006: Marial et al., 2011; Jiao et al., 2017 etc). The algorithm works by computing the sparse encoding vector (see Eq. 1) in the forward pass and updating the convolutional parameters as well as sparsity controlling hyperparameter in the backward step. To speed up the convergence, the authors rely on homotopy iterated thresholding illustrated in Figure 2.
The approach is evaluated on ImageNet and empirical results demonstrate that the removal of the sparse encoding block amounts to a significant performance degradation. The results also establish a minor improvement in the accuracy as a result of adding a linear projection matrix (i.e., principal component analysis applied to scattering coefficients). Overall, the network shows promising performance on ImageNet by doing better than AlexNet. The result is not yet 'competitive' with ResNets but it might be worth to pursue this direction of research in the future.
The work is properly structured with well organized materials from previous work. The clarity, however, could be improved in several places. In particular, the last paragraph in Section 3.1 does not explain the algorithm clearly. The confusing part is how to update the dictionary (i.e., convolutional network) parameters. One might infer from the current materials that first the problem in Eq. (1) is solved to find \alpha and that solution is fixed. Then, for that setting of the sparse encoding vector one will train the network parameters. Section 3.2 then givens an iterative procedure in Eq. (3) and Figure 2, which suggest that in a forward pass the dictionary representation is computed using some setting of parameters and \alpha is updated per Eq. (3). Following this, the gradient of the convolutional parameters is computed (\alpha_{n + 1} is differentiated, which means that the gradient depends on other \alpha iterates). I believe that this is the crux of the methodological/conceptual contribution and requires proper explanation with proper illustration of the forward-backward pass.
In Section 3.2 (homotopy iterated thresholding and ALISTA), there is a matrix W which comes ad-hoc. There should be some motivation and gentle introduction. At the moment, it is completely justified to ask why one needs this matrix and why the approach would not work without Proposition 3.1. Please add some discussion and make sure things are properly motivated and gently introduced. This will also place the theoretical contribution in the proper context and strengthen the work.
Just below Figure 2, I fail to follow how the number of layers N relates to the iterative algorithm? Does it mean that you would actually have blocks per each \alpha_n for some N indices (this again refers to previous comment on clarity)?
Can you please use \ell_1 or l_1 notation for sparse dictionary coding. The current symbol reads as 1 to the power of 1 and it is very confusing (never seen it before in the context of sparse encodings).


Mixed-precision arithmetic combining both single-and half-precision operands on the same operation has been successfully applied to train deep neural networks. Despite the advantages of mixed-precision arithmetic in terms of reducing the need for key resources like memory bandwidth or register file size, it has a limited capacity for diminishing computing costs and requires 32-bits to represent its output operands. This paper proposes two approaches to replace mixed-precision for half-precision arithmetic during a large portion of the training. The first approach achieves accuracy ratios slightly slower than the state-of-the-art by using half-precision arithmetic during more than 99% of training. The second approach reaches the same accuracy as the state-of-the-art by dynamically switching between half-and mixed-precision arithmetic during training. It uses half-precision during more than 94% of the training process. This paper is the first in demonstrating that half-precision can be used for a very large portion of DNNs training and still reach state-of-the-art accuracy.

This proposes two techniques to replace mixed-precision arithmetic with half-precision training for a large part of the training process. In the first approach, the authors simply switch all mixed-precision operations with half-precision operations, and can achieve performances slightly lower than SOTA. In the second approach, the authors propose to dynamically switch between mixed-operations and half-precision operations during training. The authors claim that this second approach can match SOTA results while using half-precision arithmetic for more than 94% of training.
Overall the paper is fairly well written, and easy to follow. The proposed techniques seem to empirically work well. However, I have a number of concerns about the paper, which explains my score. I list these concerns below.
1. The proposed approach has a number of additional hyperparameters, which makes it less likely for the algorithm to be widely used if the algorithm is very sensitive to the values of this. For very extreme values of these hyperparameters, I would expect the algorithm to start behaving quite poorly. But it would help a lot to provide some sensitivity analysis to these hyperparameters for reasonable values of these hyperparameters.
2. How much do the optimal hyperparameters (like numBatchesMP, numBatchesBF16, emaT) vary across problems?
3. How much do the above-mentioned optimal hyperparameters vary with mini batch size?
4. How are the other hyperparameters like learning rates selected? Is the learning rate tuned?
5. Are the experiments repeated multiple times?
6. It seems a bit weird to call a modification that simply uses half-precision arithmetic for most FMA operations a significant contribution of the paper, especially since it can't reach SOTA performance.
7. Algorithm 1 should be written out in a better way that shows the training loop. It is slightly confusing the way it is written up right now.
Overall I think the paper would significantly benefit from a more thorough empirical evaluation.
=============================
Edit after rebuttal:
I thank the reviewers for their response and for running the additional experiments. However, I find the updated version of the paper to be inadequate in fully answering my concerns. While the authors have included a hyperparameter sensititivity analysis, I find the experiment to be unconvincing. Only two of the three hyperparameters are swept over a very small range of values, and the results presented are only for the first 12 epochs, while the actual model is typically trained for 90 epochs. While I appreciate the added experiment and realize that 10 days is too short a time to put in a proper sensitivity analysis, based on the current draft of the paper, I cannot recommend accepting this paper. I am however raising my score to a weak reject.


Individuals with type 1 diabetes (T1D) lack the ability to produce the insulin their bodies need. As a result, they must continually make decisions about how much insulin to self-administer in order to adequately control their blood glucose levels. Longitudinal data streams captured from wearables, like continuous glucose monitors, can help these individuals manage their health, but currently the majority of the decision burden remains on the user. To relieve this burden, researchers are working on closed-loop solutions that combine a continuous glucose monitor and an insulin pump with a control algorithm in an 'artificial pancreas.' Such systems aim to estimate and deliver the appropriate amount of insulin. Here, we develop reinforcement learning (RL) techniques for automated blood glucose control. Through a series of experiments, we compare the performance of different deep RL approaches to non-RL approaches. We highlight the flexibility of RL approaches, demonstrating how they can adapt to new individuals with little additional data. On over 21k hours of simulated data across 30 patients, RL approaches outperform baseline control algorithms (increasing time spent in normal glucose range from 71% to 75%) without requiring meal announcements. Moreover, these approaches are adept at leveraging latent behavioral patterns (increasing time in range from 58% to 70%). This work demonstrates the potential of deep RL for controlling complex physiological systems with minimal expert knowledge.

The paper describes an RL based approach to administer insulin for blood glucose control among  type-1 diabetic patients.  The paper formulates this blood glucose control problem as a closed-loop reinforcement learning problem and demonstrates its effectiveness on data generated from an FDA-approved simulator of glucoregulatory system. Compared to existing approaches, the proposed method can operate without meal announcement by potentially making use of latent meal intake patterns. The authors also demonstrate how a learned policy for one particular subject can be used as initialization to train/fine-tuned the policy of another subject so as to combat the issue of high sample complexity.
Overall, the reviewer finds that the authors provide a reasonable approach to model the blood glucose management problem for type-1 diabetes. Each component of the paper is well-explained and the paper is easy to follow. The algorithmic design adopted in this paper, such as the representation of the problem, the choice of the reward function, and the choice of RL controller, are reasonably justified. While the experiments conducted in this paper are not based on real-world data, the reviewer finds the use of data from an FDA-approved simulator of glucoregulatory system sufficiently convincing for this type of problem. The limitation of the proposed method is also well discussed.
The reviewer has the following concerns:
1. The reviewer views the major contribution of this paper as formulating and solving the glucose management problem as an RL problem.  From a machine learning perspective, the reviewer finds the contribution made in this paper to advance ML methodology very limited as the components used in the algorithmic design of the proposed method are already available in the existing literature. Therefore, the reviewer finds that the paper could be of limited interest to the audience in ICLR while it might be more suitable to the audience of diabetes management.
2. Compared to competing methods reported in this paper, a major characteristic of the proposed method is that it can operate without meal announcements. Methods with meal announcements seem to operate reasonably well compared to the proposed method. Therefore, the authors can consider further justifying why meal announcement is an important bottleneck to alleviate in blood glucose management, which is currently not well explained in the paper. Related to this question, the authors may also consider justifying why making meal announcements more convenient/automated is not a good alternative to handle the blood glucose management problem. It will also be interesting to see how the proposed method will behave when augmented with meal announcements since in reality, the proposed solution might not always be reliable and intervention options like meal announcements could potentially improve the robustness of the solution.
Miscellaneous:
in Section 3.1 of the glucoregulatory system model G, the carbohydrate input $c_t$ seems to be considered as an aspect of the action. Based on the understanding of the reviewer, such an action is a proxy to the meal announcement and is not considered as an input from the user for the deployed policy. For better clarity, the authors can consider reporting the formula of the deployed policy and explain how the quantity $c_t$ is related to this policy.


Artistic style transfer is the problem of synthesizing an image with content similar to a given image and style similar to another. Although recent feed-forward neural networks can generate stylized images in real-time, these models produce a single stylization given a pair of style/content images, and the user doesn't have control over the synthesized output. Moreover, the style transfer depends on the hyper-parameters of the model with varying "optimum" for different input images. Therefore, if the stylized output is not appealing to the user, she/he has to try multiple models or retrain one with different hyper-parameters to get a favorite stylization. In this paper, we address these issues by proposing a novel method which allows adjustment of crucial hyper-parameters, after the training and in real-time, through a set of manually adjustable parameters. These parameters enable the user to modify the synthesized outputs from the same pair of style/content images, in search of a favorite stylized image. Our quantitative and qualitative experiments indicate how adjusting these parameters is comparable to retraining the model with different hyper-parameters. We also demonstrate how these parameters can be randomized to generate results which are diverse but still very similar in style and content.

The paper presents an approach for style transfer with controlable parameters. The controllable parameters correspond to the weights associated to "style losses" or ordinary style transfer models (distance between gram matrices of generated vs style image at specific layers of a network). The authors propose to learn a single architecture that takes these parameters as input to generate an image that resembles what would be generated by optimizing directly on these parameters. Examples of transfer and of the effect of these parameters are given. A quantitative evaluation shows that the effect of changing the parameters of the new network has the effect of reducing the loss at the desired layers.
Overall, I found the idea of learning controllable parameters interesting. The technical contribution is to show that learning the weights of instance normalization as a function of the hyperparameters actually is satisfactory. To be honest, I am not sure to understand why playing on instance normalization weights turns to be effective for learning an equivalent of a reduction of per-layer loss. I would have liked more motivation and intuition behind it.
The paper seems overall a bit incremental with respect to prior work on style transfer. While adjustable parameters could have application in more general tasks of image generation (for instance, in the line of work on disentangled representations), it is unclear how the specific approach of learning instance normalization weights extends beyond style transfer. As such, my feeling is that the paper is borderline.
other comments:
- I found Figure 7 important because it guarantees learning has the desired effect. However, there is a bit a lack of baseline/topline: how do the true losses decrease when their weights increase?