Several stochastic optimization algorithms are currently available. In most cases, selecting the best optimizer for a given problem is not an easy task. Therefore, instead of looking for yet another absolute best optimizer, accelerating existing ones according to the context might prove more effective. This paper presents a simple and intuitive technique to accelerate first-order optimization algorithms. When applied to first-order optimization algorithms, it converges much more quickly and achieves lower function/loss values when compared to traditional algorithms. The proposed solution modifies the update rule, based on the variation of the direction of the gradient during training. Several tests were conducted with SGD, AdaGrad, Adam and AMSGrad on three public datasets. Results clearly show that the proposed technique, has the potential to improve the performance of existing optimization algorithms.

Summary:
This paper presents an adaptive method which can be used alongside existing accelerated gradient methods. The paper is difficult to read due to mistakes and poorly defined mathematical notation. I believe that the paper is missing reference to related methods. The theoretical analysis in the paper is difficult to follow and provides little insight into the benefits of the proposed approach.
Overview:
There are many mistakes throughout the paper which have made it difficult to read.
Overall, I felt that this paper was missing a discussion of the effect of stochasticity on the proposed method. The issue with measuring the variation in the gradient direction is that in regimes where the gradient noise is dominating the signal the gradient direction at each time step is poorly correlated with overall optimization progress --- thus it seems intuitively ineffective to rely on the gradient direction to adjust the algorithm.
1)  At the bottom of page 2, the authors write "knowing that we do not have any knowledge of what this function looks like". While minor, I would point out that we are able to compute local statistics of the function and so certainly we have _some_ knowledge.
2) The authors claim that no techniques exist which use the variation of the direction of the gradient. One such example is in [1] which uses (in one case) the variation of the gradient direction to determine and appropriate time to restart the momentum computation.
3) In section 3.2, the Adam moment computation is missing a "diag". Assuming that AMRSGrad is AMSGrad (mistyped), then this term is incorrect and matches Adam.
4) There are many mistakes in the Algorithm 1 box.
- The wrong $F$ is used in the input (should be $\mathcal{F}$).
- The algorithm takes as input a sequence of functions ($\phi, \psi$) which are not used.
- Within the if statement, $gm_t = g_t + m_t$. I believe this should be an $m_{t-1}$. It is not clear what the vector $gm$ is exactly, and then $\dot{g}$ is used afterwards which is also not defined.
- The algorithm checks for $|m_{t-1} - g_t| > S$ while the text uses $|g_{t-1} - g_t| > S$.
5) The first line of section 3.3 is quite worrying: "We assume that if we are able to prove that modifying one optimizer with the proposed method does not alter its convergence, then the same applies for the other optimizers". This seems like a dangerous assumption to make and should at the very least be carefully verified empirically. Following this, I am not sure what the authors mean by "deterministic" and "non-deterministic" methods.
6) I do not understand the claim above Theorem 2 that $\nabla f(x_{T-1}) = k \nabla f(x_T)$. Under what conditions does this hold and how is $k$ computed? If I understand correctly, the bound provided in Theorem 2 is worse than that given for gradient descent. Moreover, the bound does not depend on the hyperparameter $S$ introduced in Algorithm 1 and provides limited insights into the method. I could not find a proof of Theorem 2 in the paper or appendix.
7) There are serious flaws with the experimental evaluation in this paper.
a) There is no tuning over hyperparameter settings for any of the optimizers.
b) The basic problems are very limited, even for toy problems. The 1D deterministic quadratic tells us very little about the performance of the optimizer. And the 1D cubic problem is particularly confusing. Unless I am mistaken, the gradient will always have the same sign (3x^2) and thus the acceleration condition will never be triggered.
c) I believe that Figure 2 explores stochastic optimization problems which as discussed at top is a crucial evaluation. Unfortunately, due to lack of parameter tuning it is difficult to infer much about the comparison between the methods.
d) Figure 4 compares performance variation over changing the threshold. The y-axis scale across each plot changes making the comparison unnecessarily difficult --- the scale should be the same.
Minor:
- TYPO Line 2, "minimize ---,"
- End of intro, MNIST and CIFAR not cited while IMDB is. Citation uses citet not citep.
- Bottom of page 2, ""
- Top of section 3.2, "The pseudo code of our the method"
References:
[1] Adaptive Restart for Accelerated Gradient Schemes, Brendan O'Donoghue and Emmanuel Candes


A central challenge in multi-agent reinforcement learning is the induction of coordination between agents of a team. In this work, we investigate how to promote inter-agent coordination using policy regularization and discuss two possible avenues respectively based on inter-agent modelling and synchronized sub-policy selection. We test each approach in four challenging continuous control tasks with sparse rewards and compare them against three baselines including MADDPG, a state-of-the-art multi-agent reinforcement learning algorithm. To ensure a fair comparison, we rely on a thorough hyper-parameter selection and training methodology that allows a fixed hyper-parameter search budget for each algorithm and environment. We consequently assess both the hyperparameter sensitivity, sample-efficiency and asymptotic performance of each learning method. Our experiments show that the proposed methods lead to significant improvements on cooperative problems. We further analyse the effects of the proposed regularizations on the behaviors learned by the agents.

This paper proposed two approaches to encourage cooperation among multi-agents under the *centralized training decentralized execution* framework. The main contribution of this paper is that they propose to allow agents to predict the behavior of others and introduce this prediction loss into the RL learning objective. One approach is teammate-regularization where each agent predicts the behavior of all others (and therefore makes the total model complexity increasing quadratically with the number of agents), while the other is a centralized coach-regularization. The performance of these two approaches are compared with MADDPG and its two variants on 4 games. Experiments show that their approaches surpass a few baselines in certain game settings. However, the novelty of this algorithm is not strong, given that the similar idea of 'predicting the behavior of other agents' can be found in related work with discrete action spaces. Experimental results also show unstable advantages of their approaches.
Methodology:
Team-reg: One main difference between this approach and Jaques work: * Intrinsic social motivation via causal influence in multi-agent*  (or other MARL algorithms for discrete action spaces) is that in this work, each agent predict the action of other peers instead of the policy, since this algorithm is only based on DDPG with continuous action space. However, It seems that this idea is transferrable to discrete action spaces as well, so long as we change the MSE loss between actions to KL loss between policy over states. The novelty of this approach is not strong. Also, one hypothesis on which their work is based is that promoting agents’ predictability could foster such team structure and lead to more coordinated behaviors. Does predictability really improve cooperation? Could it be the other way around? Experiment results in the right-below penal (CHASE Game ) and left-above penal (SPREAD game) of Figure 5 actually strength my concern.
Coach-reg: Descriptions of the coach regularization approach is quite vague. I doubt the motivation and effectiveness of this approach. More questions are as follows:
* What is the role of the policy mask in deriving a better action? Is it just a dropout before the activation layer with a fixed $p$ proportion (which is related to the choice of $K$)? If so, this is like you first assume the policy network is somewhat overfitting, then alleviate this issue by letting the coach adjust which part to keep and which part to drop.  How would different $K$ values affect the performance?
* Does this framework really improve cooperation? Intuitively, all agents will finally reach a point where they agree on the same policy mask distribution (which is the one generated by the coach). How does the same policy mask lead to an improvement in cooperation? Empirical or theoretical explanations are strongly needed in the main results, not in the appendix section.
Experiments:
- All variants of MADDPG in the experiments are weak baselines, assuming that $MADDPG + sharing$ means all agents share the same policy and Q-function. However, since this algorithm only works for continuous action space, available relate work to compare with is quite limited. All environments only include two agents. Experiments with more than two agents should be implemented.
- Even for only 3 games (excluding the adversarial game setting), both branches failed to beat the *sharing* baseline for the CHASE game. For the ablation study, *agent-moduling* even works better than *TeamReg* for the CHASE game, so as the case when *policy mask* beats *CoachReg* in the SPREAD game (Figure 5). They seem to show that under these two proposed frameworks, the predictability of agents does not always encourage cooperation, at least for 2 out of 4 game settings mentioned in this section.
- This work makes a lot of effort on the hyper-parameter tuning part, which however does not provide a systematic solution to the tradeoff between improving predictability and maximizing reward.


Multiagent reinforcement learning (MARL) attempts to optimize policies of intelligent agents interacting in the same environment. However, it may fail to converge to a Nash equilibrium in some games. We study independent MARL under the more demanding solution concept of iterated elimination of strictly dominated strategies. In dominance solvable games, if players iteratively eliminate strictly dominated strategies until no further strategies can be eliminated, we obtain a single strategy profile. We show that convergence to the iterated dominance solution is guaranteed for several reinforcement learning algorithms (for multiple independent learners). We illustrate an application of our results by studying mechanism design for principal-agent problems, where a principal wishes to incentivize agents to exert costly effort in a joint project when it can only observe whether the project succeeded, but not whether agents actually exerted effort. We show that MARL converges to the desired outcome if the rewards are designed so that exerting effort is the iterated dominance solution, but fails if it is merely a Nash equilibrium.

This paper studies reinforcement learning algorithms in a specific subset of multi-agent environments that are 'dominance solvable'. This means that, given an initial set of strategies in the game, if we iteratively remove 'dominated strategies' (those whose utility is strictly less than another strategy independent of the strategies used by other agents), then only one strategy remains for each player. The remaining strategy is called the iterated dominance solution. The paper proves the convergence of certain RL algorithms (REINFORCE in the 2-action case, and importance weighted monte-carlo policy iteration in the multi-action case) for normal-form games. The paper demonstrates the utility of this via mechanism design: in a principal-agent problem where one can design the rewarding scheme given by a 'principal agent' to various (RL) sub-agents, rewarding schemes motivated by iterated dominance guarantees the best solution for the principal agent, whereas schemes motivated by Nash equilibria do not.
The paper is quite well-written and understandable. To my knowledge, the idea is novel and has not yet been explored in the RL literature (UPDATE: based on Reviewer #1's review, this may not be the case. I'll wait to hear the author response to this). I did not check the proofs thoroughly. However, the experiments in the principal-agent problem make sense, and it's interesting to see that iterated dominance reward schemes results in good performance for the principal agent. I appreciate that, while the main results in the paper are limited to normal-form games (which are quite restricted), there are empirical results in the appendix showing the extension to Markov games with multiple timesteps, suggesting that the applicability of iterated dominance reward schemes extend beyond the simple two-action case, where no temporally extended decisions need to be made. Even so, the Markov game considered is fairly simplistic.
My personal curiosity about this paper revolves around scaling to real-world applications. This is not really discussed in the paper; the conclusion talks about directions for future work, for example expanding the number of RL algorithms where convergence can be proven, or producing complexity bounds for convergence. What I want to know is: what sorts of games can we compute the iterated dominance reward schemes for? How can this be applied when the space of policies becomes too large to be enumerated (and thus determining whether a policy is strictly dominated becomes impossible)? I don't expect this paper to solve these issues, but it would be nice to have a discussion of them.
Overall, I'd say this paper is interesting to the multi-agent RL community and I could imagine others building off of this work, so I err on the side of acceptance.
Small fixes:
- Our proof of Theorem 3.1 -> Theorem 3.2
- I'd recommend extending the captions of figures 6-8 and 9-11 in the Appendix.
- Close bracket in Section 6.3 title


In many applications labeled data is not readily available, and needs to be collected via pain-staking human supervision. We propose a rule-exemplar model for collecting human supervision to combine the scalability of rules with the quality of instance labels. The supervision is coupled such that it is both natural for humans and synergistic for learning. We propose a training algorithm that jointly denoises rules via latent coverage variables, and trains the model through a soft implication loss over the coverage and label variables. Empirical evaluation on five different tasks shows that (1) our algorithm is more accurate than several existing methods of learning from a mix of clean and noisy supervision, and (2) the coupled rule-exemplar supervision is effective in denoising rules.

The paper addresses the problem that labelled data is often unavailable in the quantities required to train effective models. It deals with classification problems, and proposes a method to obtain more (but weaker) labels data with minimal involvement from human labellers, by asking them to generalize their labelling decisions into rules and then learning restrictions on those rules to avoid learning incorrectly generalized labels. The motivating observation is that human labellers are often able to make such generalizations in much less time than it would take them to apply that rule to a large dataset themselves. This is an interesting idea, especially for cases where labelling capacity is limited. The point being made about the labelling noise not being random in this situation is an interesting one - it might be worth exploring this notion further on its own also in contexts where the source of the noise is unknown.
The presentation of the implementation the authors choose for their proposed approach is clear, and the implementation is sensible. The experimental section includes comparisons to a number of alternative methods, and the authors find that their method outperforms all others, including recent methods for combining (noisy) rule-based labels and (clean) human-sourced labels.
I would argue for accepting this paper. It studies an interesting question, which if answered has the potential to make access to machine learning solutions to certain types of problem significantly cheaper and therefore more widespread. The experiments are well-chosen and show that, depending on the data available and the task, significant gains can be made using the proposed method.
Some remarks on how the paper could become stronger: The type of problem that can be assessed with the proposed method seems to be fairly specific: most tasks studied are classification of natural language utterances. That is a natural class of tasks, since it is easy to imagine how labellers can formulate rules. However, it would have been very interesting if the authors had found ways to allow for more diversity here.  In general, I have the impression that there are more interesting ideas and results to be found in the direction explored by this paper - what about, e.g., allowing the classifier to add rules of its own?
The paper would benefit from some general editing with regards to appearance; for example, the supplementary material sections continue the regular section numbering, instead of having their own; the images are missing captions, and sometimes have somewhat unorthodox axis tick labelling.


Quantum computing is a powerful computational paradigm with applications in several fields, including machine learning. In the last decade, deep learning, and in particular Convolutional Neural Networks (CNN), have become essential for applications in signal processing and image recognition. Quantum deep learning, however, remains a challenging problem, as it is difficult to implement non linearities with quantum unitaries Schuld et al. (2014). In this paper we propose a quantum algorithm for evaluating and training deep convolutional neural networks with potential speedups over classical CNNs for both the forward and backward passes. The quantum CNN (QCNN) reproduces completely the outputs of the classical CNN and allows for non linearities and pooling operations. The QCNN is in particular interesting for deep networks and could allow new frontiers in the image recognition domain, by allowing for many more convolution kernels, larger kernels, high dimensional inputs and high depth input channels. We also present numerical simulations for the classification of the MNIST dataset to provide practical evidence for the efficiency of the QCNN.

The authors present a quantum algorithm for approximating the forward pass and gradient computation of a classical convolutional neural network layer with pooling and a bounded rectifier activation. This algorithm has complexity bounds that would open up (for instance) the possibility of exponentially large filter banks, and the authors show through a simple, classical simulation approach that the resulting network is also likely to be trainable.
Feedback:
A few typos/formatting issues:
- The title accidentally includes "Conference Submissions"
- The in-text citation format frequently has the parentheses in the wrong place; this is surprisingly distracting!
Preliminaries:
- Maybe explain what the ith vector in the standard basis is in terms of |0> and |1>? I assume the answer is along the lines of |000>, |001>, |010>, etc.?
Main results:
- The sentence "a speedup compared to the classical CNN for both the forward pass and for training using backpropagation in certain cases" is ambiguous; does "in certain cases" qualify only training speed or also forward pass speed?
- There's a clear separation of background (which is concise and well explained) and contributions, but maybe it would be worth connecting the introduced algorithm more closely to existing work in non-convolutional quantum neural networks?
- Can you briefly justify (or cite) the claim that "most of the non linear functions in the machine learning literature can be implemented using small sized boolean circuits"?
- I'm a little confused about the discussion of quantum importance sampling on page 4. Could you give some intuition for the relationship between eta and the fraction of output values that are on average flushed to zero (is this 1 minus sigma?), and perhaps connect this to the literature about activation pruning and sparse NNs?
- Maybe define what you mean by "tomography" for ML folks without the quantum background?
- I'm convinced by the simulations, even though I shouldn't really be convinced by anything on MNIST... It just seems like the perturbations you're applying are all things that modern neural networks take in stride.
- The discussion of using a sigma-based classical sampling rather than the eta-based quantum importance sampling mentions a "Section C.1.15" which does not exist (I think you mean the end of Section C.1.5).
- Re: "We will use this analogy in the numerical simulations (Section 6) to estimate, for a particular QCNN architecture and a particular dataset of images, which values of σ are enough to allow the neural network to learn." My understanding is that you're getting empirical estimates of which values of sigma are enough; it would be valuable to convert those to estimates of which values of eta would be enough (given quantum networks of the size used in the classical simulation experiment, or given larger networks).
- The sampling procedure based on sigma might be inefficient in your PyTorch implementation, but it's certainly something that GPUs are fairly well suited to computing. There might be other PyTorch operators that would help here (perhaps Bernoulli sampling?) or if nothing else you could write a small custom CUDA kernel.


Bayesian inference promises to ground and improve the performance of deep neural networks. It promises to be robust to overfitting, to simplify the training procedure and the space of hyperparameters, and to provide a calibrated measure of uncertainty that can enhance decision making, agent exploration and prediction fairness. Markov Chain Monte Carlo (MCMC) methods enable Bayesian inference by generating samples from the posterior distribution over model parameters. Despite the theoretical advantages of Bayesian inference and the similarity between MCMC and optimization methods, the performance of sampling methods has so far lagged behind optimization methods for large scale deep learning tasks. We aim to fill this gap and introduce ATMC, an adaptive noise MCMC algorithm that estimates and is able to sample from the posterior of a neural network. ATMC dynamically adjusts the amount of momentum and noise applied to each parameter update in order to compensate for the use of stochastic gradients. We use a ResNet architecture without batch normalization to test ATMC on the Cifar10 benchmark and the large scale ImageNet benchmark and show that, despite the absence of batch normalization, ATMC outperforms a strong optimization baseline in terms of both classification accuracy and test log-likelihood. We show that ATMC is intrinsically robust to overfitting on the training data and that ATMC provides a better calibrated measure of uncertainty compared to the optimization baseline.

Summary:
The main idea of the paper is the introduction of ATMC, an adaptive noise MCMC algorithm that dynamically adjusts the momentum and noise applied to each parameter update while sampling from the posterior of a neural network. A modified version of ResNet architecture called ResNet++ has been introduced in the paper and later used to train it on ImageNet using MCMC which is great work considering the complexities associated. Furthermore, the authors claim ATMC to be robust to overfitting and it provides a measure of uncertainty with predictions. The paper is well written with clear flow and good mathematical explanation.
Questions from authors:
1. The approach of Stochastic Gradient MCMC (SG-MCMC) has been there for quite some time and in this paper, there is no clear explanation of the advantages of ATMC over SG-MCMC methods. The authors can argue that the introduction of dynamically adjustable momentum and noise can make the paper novel and unique. However, in Stochastic gradient Markov chain Monte Carlo (SG-MCMC) [1], the authors used a meta-learning algorithm to learn Hamiltonian dynamics with state-dependent drift and diffusion which makes the system scalable to large datasets, very similar to this work. Also, changing the momentum variable in Hamiltonian dynamics with thermostat variable in stochastic gradient thermostats, the meta learner algorithm of [1] can be used to make the stochastic gradient thermostats approach adaptive. What are the advantages of the approach proposed in this paper over [1]?
2.  The paper follows the approach of stochastic gradient thermostats as introduced in the paper  [2] and builds upon it to make it adaptive to improve stability and for better convergence.  The advantages are clearly mentioned in the paper except for the case when 0 < ξ < D_c the total amount of noise added to the momentum is D_c and the friction coefficient β(ξ) = D_c. At this stage, the authors claim that 'the stochastic gradient noise is compensated for by adding less noise to the momentum update.' How is this done? Can authors please explain it a bit more.
3.  For the experiments, the authors introduced ResNet++ architecture which is based on ResNet architecture but is novel in its design with the use of SELU and removal of BatchNorm and with different initialization schemes. The design idea is well explained by the authors. The experiments were performed on CIFAR10 and ImageNet dataset to show the large scale scaling of the approach. A comparison with SGNHT is provided in the paper but a fair comparison with other approaches like SGHMC [3], PSGLD [4] and MCMC [1] is missing which might have proved the effectiveness of the approach when compared to other methods. Any comments on why the experiments were restricted to the ones mentioned in the paper?
4. Lastly, the ResNet++ network is trained on ImageNet and CIFAR10 datasets. There is no clear mention of the time duration it took for training and evaluating the network. The authors mentioned ‘BatchNorm did result in an overhead of roughly 20% compared to the ResNet++ model’, which is not clear. I would urge the authors to explain the above line and provide an estimate of the training time and testing time.
A closing remark: The mention of the acknowledgement section in a double-blind review is not advisable and in future please refrain from doing so.
References:
[1] https://openreview.net/pdf?id=HkeoOo09YX#page=11&zoom=100,0,754
[2] http://people.ee.duke.edu/~lcarin/sgnht-4.pdf
[3] https://arxiv.org/pdf/1402.4102.pdf
[4] https://arxiv.org/abs/1512.07666


Counterfactual regret minimization (CFR) methods are effective for solving twoplayer zero-sum extensive games with imperfect information. However, the vanilla CFR has to traverse the whole game tree in each round, which is time-consuming in large-scale games. In this paper, we present Lazy-CFR, a CFR algorithm that adopts a lazy update strategy to avoid traversing the whole game tree in each round. We prove that the regret of Lazy-CFR is almost the same as the regret of the vanilla CFR and only needs to visit a small portion of the game tree. Thus, Lazy-CFR is provably faster than CFR. Empirical results consistently show that Lazy-CFR is fast in practice.

The paper proposes an improvement to Counterfactual Regret Minimization, avoiding traversing the whole tree on each iteration. The idea is not to change the strategy in those infosets, where the reach probability of opponents is low. The strategy in such infosets is only updated once in several iterations, when the sum of reach probabilities over these iterations if higher than the threshold. The straightforward implementation of the idea still has the same running time as CFR. Therefore, the paper presents an efficient implementation, exploiting the structure of the game tree. However, this implementation comes at the cost of additional memory requirements. Overall, the paper proves the theoretical result of about O(sqrt(|I|)/D) times faster than CFR to achieve the same approximation error, while the memory requirements increase by a factor of O(|H|/|I|). Here |I| is the number of infosets, D is the depth of the game tree and |H| is the number of histories.
The idea of eliminating unnecessary computations for infosets with low probability is a valuable contribution. The presented theoretical analysis takes an important place in the series of works refining the regret upper bound of CFR and its variants. The experiment confirms performance of the idea.
That being said, I follow up with some questions/criticism.
1.	Implementation in Section 3.2.1 and Appendix E is rather hard to follow. Is there any intuition on how the segment [\tau_t(h), t] is divided, i.e. what does t_1, t_2 and \tau’(h) mean? Also, clarity could be increased if these variables would be defined before they are used.
2.	How is a segmentation rule for Lazy-RM in OLO designed in such a way, that equation \sum_{i=1}^n \max_a c’_i(a)^2 \approx \sum_{j=1}^T \max_a c_j(a)^2 holds?
3.	Section 3.2: “following step (1)”. (1) is an equation for RM in OLO, probably some other reference was meant.
4.	Recently, Linear Cfr was introduced, which outperforms Cfr+. Thus, citation is needed Brown, Noam and Sandholm, Tuomas “Solving Imperfect-Information Games via Discounted Regret Minimization”. Worth to mention, LazyCfr is straightforwardly compatible with Linear Cfr.
5.	The specified space requirements significantly limit the applicability of the presented Lazy-RM implementation. For example, in state-of-art approaches to solve/resolve No-Limit Holdem (Libratus, DeepStack, Pluribus), either the game tree is too large, making the space requirements unrealistic, or the game tree is small enough for getting a good equilibrium approximation fast even with CFR+.
UPD: score updated


Meta-learning methods learn the meta-knowledge among various training tasks 1 and aim to promote the learning of new tasks under the task similarity assumption.
2 However, such meta-knowledge is often represented as a fixed distribution, which 3 is too restrictive to capture various specific task information. In this work, we 4 present a localized meta-learning framework based on the PAC-Bayes theory. In 5 particular, we propose an LCC-based prior predictor that allows the meta learner to 6 adaptively generate local meta-knowledge for specific tasks. We further develop

In this work, the authors introduce PAC-Bayesian generalization bounds for Meta-Learning. In their framework, they have a hyper-prior distribution, a class of hyper-posteriors and an algorithm A that takes a sample set Si from task Di and a prior P and returns a posterior Q.
Pros:
-- Overall, the main text of the paper is finely written.
-- The motivation is well articulated.
-- The relationship with local coordinate coding seems like an interesting direction.
-- The experiments seem sensible.
The novelty of the bound in Thm. 1:
-- It seems that several aspects of the high-level derivation methodology of the bound are not new. Instead of applying McAllester’s bound twice (as done by Galanti et al. 2016, Amit and Meir 2018), the authors employ Catoni’s bound (a different variant of PAC-Bayes) twice. In addition, the authors apply the standard Gaussian randomization which leads to L2 regularization terms as the capacity terms -- also well known in the literature (see for example Hazan et al. 2013, etc’).
-- I would be happy if the authors point out which parts of their derivations are novel, for instance, the application of local coordinate coding, etc'.
I think the authors' claim that their bound (Thm. 1) is tighter than previously existing bounds is a bit questionable:
-- There are already PAC-Bayesian bounds with the proposed orders of magnitudes with the exact same coefficients by Catoni 2007 and Pascal Germain et al. 2009. In fact, the authors employ Catoni’s bound within the appendix. In my opinion, it should be addressed in the main text.
-- In addition, their bound has two coefficients that are being ignored in their analysis of its magnitude: c/(1-exp(-c)) (here, c stands for c1 or c2) for the error term and 1/(1-exp(-c)) for the capacity term and an additional constant that depends on n,mi and \delta. In the previous bounds, the coefficients are 1 for the error term and 1 for the capacity terms. I think the paper would greatly benefit from a direct comparison between the two.
For instance, as a direct comparison between the two bounds, I would select c, such that, 1/(1-exp(-c)) is close to 1. However, in this case, the coefficient 1/(1-exp(-c)) of the capacity term is huge and makes the overall capacity term very large, which is not favorable. Therefore, it seems to me that the proposed bound is favorable only when the training error is very small (realizable case) and c can be selected to be large. However, when the training error is assumed to be small, it is well known that the gap between the generalization risk and the empirical risk is of order O(capacity/m) (see Thm. 6.8 (bullet 3) in S. Shalev-Schwarz and S. Ben-David 2014). Again, this is also a property of the original bound by Catoni.
Finally, the authors mention that the parameters c1 and c2 control the tradeoff between the empirical error and the capacity terms. I think this is a bit inaccurate, since, c1 and c2 are selected a-priori to the estimation of the error term. Therefore, should be independent of the error term.
-- The presented bound has an additional constant “E_i[const(n,mi,delta)]”.
It is unclear to me what is the magnitude of this quantity. I think it might improve the paper if the authors explicitly analyze this term.
From the proof of Thm. 2 (Eq. 36) it seems to be of order >= ( O_{\alpha,\beta}(\gamma,C) + (1/m_{ik})^{1/2} )^2.
What is the definition of m_{ik} (couldn’t find it, maybe it was defined somewhere -- I think it should have been recalled)? I’m assuming it is mi or something similar.
Second, I’m not sure how to estimate the size of O_{\alpha,\beta}(\gamma,C). From Lem. 1 it depends on some arbitrarily selected quantity \epsilon > 0 and |C| is a function of \epsilon, so I’m not sure how to measure it.
------------------------------------
Soundness:
There are a few things that I'd be happy if clarified regarding Thms. 1 and  2.
-- I’m not sure what is w^Q_i (couldn’t find its definition anywhere). I guess w^Q_i is the center of Q_i = A(Si,P), where P ~ \mathcal{Q}. How can w^Q_i be free within the inequality without being bound to an expectation over P? Especially since you take an expectation over P within the training error. I guess the correct formulation of the bound is one that has  E_{P ~ \mathcal{Q}}[||w^Q_i - w^{\mathcal{Q}}||^2] instead of ||w^Q_i - w^{\mathcal{Q}}||^2.
Maybe the issue is somewhere in between Eq 39 and 40, where the authors take an expectation over Pi (which is not clearly defined as well), but not over Qi?
-- In Eq. 27 last equality: we have a term of the form: E_v ||w^Q - w^P||^2. Where is v in the distance squared?
-- I’m not sure what is the motivation of applying Eq. 28, the bound should be tighter with the LHS term instead of the RHS.
-- In the top half of page 15, P stands for a prior independent of \mathcal{Q}, while in the main text, P~\mathcal{Q}. Therefore, the relationships between different arguments are unclear to me.
-- Finally, I think the paper could be improved if the authors would organize the appendix.
Experiments:
In your experiments you compare the derived algorithm to other PAC-Bayesian baselines. Can you show that your algorithm outperforms other existing baselines in the literature of Meta-Learning (in terms of generalization, etc')?
Typos:
-- “Catino” ---> “Catoni”.
Finally, I think the authors should address the public comment by Tomer Galanti. It seems that Thm. 9 in Galanti et al. (2016) introduces a PAC-Bayesian theorem for Meta-Learning (they call it transfer learning), similar in its nature to Thm. 1 in the current paper. In Thm. 9 they have a hyperprior, denoted by P, learn a hyper posterior Q, for selecting posteriors qi for many different tasks di. In their framework, for each task di, their method returns a posterior qi that minimizes the bound for a specific task. This distribution, qi, is a function of a prior B selected by Q (P in your notation) and the i’th task’s dataset si (Si in your notation). Therefore, instead of denoting it by A(Si,P) as done by the authors, they simply call it a minimizer (but it is a function of the same argument that the authors address). Overall, it seems to me that Galanti et al. had a very similar setting, with different notations and focus on a specific algorithm A.
I think that overall, the paper has interesting insights and the relationship with local coordinate coding seems like an interesting direction. However, I think the paper should not be published in its current form.


Whatever information a deep neural network has gleaned from past data is encoded in its weights. How this information affects the response of the network to future data is largely an open question. In fact, even how to define and measure information in a network entails some subtleties. We measure information in the weights of a deep neural network as the optimal trade-off between accuracy of the network and complexity of the weights relative to a prior. Depending on the prior, the definition reduces to known information measures such as Shannon Mutual Information and Fisher Information, but in general it affords added flexibility that enables us to relate it to generalization, via the PAC-Bayes bound, and to invariance. For the latter, we introduce a notion of effective information in the activations, which are deterministic functions of future inputs. We relate this to the Information in the Weights, and use this result to show that models of low (information) complexity not only generalize better, but are bound to learn invariant representations of future inputs. These relations hinge not only on the architecture of the model, but also on how it is trained.

Summary of the paper:
This is a theoretical paper that builds on top of Achille and Soatto (2018), Achille et al. (2019), McAllester (2013), and Berglund (2011),. The paper attempts to answer the relationship between the inductive bias of SGD, generalization of DNNs, and the invariance of learned representation from an information theoretical point of view.
The paper mentioned many interesting links. In my opinion, the contributions are the following:
1. Invoking the theoretical result of Berglund (2011) to justify why Fisher information is relevant -- SGD tends to avoid local minima with high Fisher information.
2. Relating the Fisher information and the stability of SGD to I(w; D).
3. Introducing the definition of effective information in the activation, and show that which is closely related to the Fisher information.
About the rating:
This is basically a good paper but I have a few concerns:
1. A large fraction of this paper are taken from Achille and Soatto (2018), Achille et al. (2019).
2. In terms of impact, the paper is somehow incomplete -- it only demonstrates that the Fisher information is important, but the insights didn't lead to any substantial improvement over the current deep learning framework.
Detailed comments:
1. In my opinion, defining "information in the weights for the task D" by KL(Q||P) is inaccurate.
The weights themselves are information, which form a representation or a lossy compression of the data (which is also an information).
According to the rate-distortion theory, what we care about is the amount of information the representation attains rather than "where" the information are. Therefore, we should talk about "rate" or "amount of information" or "mutual information" rather than "information" itself.
A missing reference regarding this point:
Hu et al. "β-BNN: A Rate-Distortion Perspective on Bayesian Neural Networks." 2018,
which derives the information Lagrangian directly from rate-distortion theory.
2. More discussions on Xu and Raginsky (2017) is expected, since it proposes to use I(w; D) as a generalization bound. It seems, in terms of generalization, minimizing I(w; D) is a sufficient condition while minimizing Fisher is a necessary condition.
3. There are in fact 4 key aspects: sufficiency, minimality, invariance and generalization. It would be great to have a theorem to summarize the relationships between them.
4. Could you elaborate on the footnote 3?


Differentiable neural architecture search has been a popular methodology of exploring architectures for deep learning. Despite the great advantage of search efficiency, it often suffers weak stability, which hinders it from being applied to a large search space or being flexibly adjusted to different scenarios. This paper investigates DARTS, the currently most popular differentiable search algorithm, and points out an important factor of instability, which lies in its approximation on the gradients of architectural parameters. In the current status, the optimization algorithm can converge to another point which results in dramatic inaccuracy in the re-training process. Based on this analysis, we propose an amending term for computing architectural gradients by making use of a direct property of the optimality of network parameter optimization. Our approach mathematically guarantees that gradient estimation follows a roughly correct direction, which leads the search stage to converge on reasonable architectures. In practice, our algorithm is easily implemented and added to DARTS-based approaches efficiently. Experiments on CIFAR and ImageNet demonstrate that our approach enjoys accuracy gain and, more importantly, enables DARTS-based approaches to explore much larger search spaces that have not been studied before.

--- response after the author's rebuttal ---
Thank the authors to provide their response and let me clearly understand their contribution.
However, after considering those, I will not change my rating. The paper identifies the inaccurate gradient computation in the original DARTS and propose a new estimation and achieves constant improvement. While interesting, the experiments show this approach is just as good as some simple human tricks like early stopping. In their response, they also didn't intend to generalize their approach to other baselines, and it will significantly limit the impact of this work.
Authors insist that involving human prior is not the primary goal of AutoML. This is a subjective opinion, and my subjective opinion is it is wrong. If you take a closer look at all search spaces currently used in the NAS field, human priors are why NAS works up to now. For example, to use a convolutional operation, people use Conv-Bn-ReLU instead let NAS also search the activation functions. To search a cell rather than entire network is also designed by a human. If applied correctly, it is not wrong to include human priors.
As stated in guideline, we should apply a higher standard to a 10-page paper, and unfortunately, in my opinion, this paper does not meet such standard with the current version, and heavily revision will be necessary. I encourage the authors to shorten their paper by putting some of the derivations to the appendix and to better show their contribution. If not planning to generalize their method, show more ablation and toy example of gradient trap and how the approach solves that.
--- original review---
Summary
The paper presents a novel way to refactor second-order gradient term in Differentiable Architecture Search (DARTS) into the inverse of Hessian matrix (H^{-1}) corresponding to the optimal weight w^*, by leveraging a mathematical based on a property that gradient of loss w.r.t. w^* is 0. It provides an estimate relying on H instead of H^{-1} and showing mathematically that the proposed estimate has an angle (w.r.t ground-truth gradient vector) < 90 degrees, while the original estimate in DARTS is not bounded. The paper claims phenomenon (namely, gradient trap in DARTS) is the reason why DARTS and its successors constantly converge into a poor architecture while the proposed amendment will not.
It shows the experiments in both original DARTS search space with changes in the search phase, as well as a larger space with fewer operations.
The observation that DARTS converging to a fixed point is interesting and the motivation to find the reason and solution is well justified. The technical novelty to compute the actual and estimate term for second-order DARTS is okay, and the identified "gradient trap" seems reasonable. However, I have some doubts regarding both theoretical and empirical aspects of this paper. The evidences in the current version is not sufficient to support all the claims. Especially considering the 10-page length, I hope the author could clarify these concerns during the rebuttal period.
Major concerns
About the derivation
- Based on the derivation in Section 3.3, the proposed gradient amendment relies on Hessian Matrix regarding the optimum w^*(\alpha), while in reality, it is rarely satisfied. Will this brings another gradient trap? Could you comment on the theoretical bound between the proposed amendment and the original DARTS estimation? In addition, as stated in DARTS paper (Liu et al.) section 2.3, if w is already a local optimum, the \nabla L_train (w, \alpha) = 0, where this will also make g2 or g2' to 0.  However, to obtain a good estimate of the Hessian matrix, one will need to have a good estimate of w^*. Does this contradict the proposed approach?
- Computation of the Hessian matrix is missing, and the motivation to use g2'.
The proposed estimate depends on the Hessian matrix, however, it is not mentioned in the paper how to compute such matrix, especially "... is computationally intractable due to the large dimensionality of H, ... usually exceeds one million ..." as in Section 3.3. Also, authors proposed to use g2' on top of the Hessian matrix instead its inverse, could author add experiments to show the difference between g2 and g2'?
I came across to a concurrent paper submission [1], the derivation of second-order gradient estimate is the same as yours, i.e., one could use negative Hessian to improve the DARTS (see Appendix A.2 in [1]). However, the numerical computation of this Hessian is hard is also mentioned in [1]. Could the author also comment on that?
- Why the paper introduces \nabla_{\alpha} w^*(\alpha) with regarding L_train while the equation 2,3 is about validation loss? In DARTS, both training and validation dataset is used in training for better generalization.
About experiments
- Could the author provide an additional experiment, to show in reality, even for a toy example, this gradient trap make DARTS converge to an architecture full of skip-connection, i.e., showing the gradient estimate (i.e., g1+g2) w.r.t. \alpha, obtained by original DARTS, proposed approach, if possible, ground-truth estimate, after a certain epoch number (e.g. 50 or any number that DARTS start turning to skip-connection)? I think this will empirically reveals if the gradient trap is causing the DARTS problem and better demonstrate the paper's method effectiveness.
- Additional experiments for a fair comparison with baseline DARTS.
Experiment settings for both S1 and S2 seem to make the comparison to DARTS unfair. As shown in some previous work [2], different search space has different characteristic and is usually non-trivial in NAS domain. In addition, [3,4] shows that, weight sharing NAS is usually sensitive to random initialization, and results across runs can be quite different. To have a fair comparison, the paper should include additional experiments, running original DARTS and proposed one on the search space, S1 and S2, **with the same hyper-parameter setting**, for 3 runs with different random initialization. If the new results are still statistically significant, it will be strong evidence that the proposed algorithm indeed improves original DARTS.
Minor comments.
- Would the author kindly clarify the following points I found confusing during the reading?
1. Introduction third last paragraph: "Our final solution involves using the amended second term of [eq] brings \alpha meanwhile keeping the first term unchanged". What's does "brings \alpha?" means?
2. Section 3.3 line 7, is "fundamentals in mathematics" referring math foundations?
3. Equation 2, 3 are repetitive for no good reason?
4. In table 1,2 Random search results for S2 are missing. Could the author also listed for better comparison?
5. The second line after equation (3), "estimating \nabla ... ^2" why is this a square while in equation (3) it's not?
- generalization comparison with DARTS+, XNAS, etc.
Since the proposed gradient amendment is a general approach, adding this on top of other approaches seems natural. Could the author extend their method to other modified DARTS algorithms. It should obtain even better results with extra human-expertise after fixing the gradient trap. Adding this will further strengthen the paper.
- The writing is somehow informal and could be polished, e.g. "This is to say" in paragraph 1 of page 2, misuse of bold style, and some typos:
1. Introduction page 2, "In all experiments", 'I' should not be in bold.
2. line after equation 5, "throughout the remainder of this paper" remainder -> remaining.
--- Reference ---
[1] Anonymous, UNDERSTANDING AND ROBUSTIFYING DIFFERENTIABLE ARCHITECTURE SEARCH, link: https://openreview.net/forum?id=H1gDNyrKDS.
[2] Radosavovic et al., On Network Design Spaces for Visual Recognition, ICCV'19.
[3] Li and Talwalker, Random Search and Reproducibility for Neural Architecture Search, UAI'19.
[4] Scuito et al., Evaluating the search phase of neural architecture search, arxiv'19.


Neural architecture search (NAS) methods have shown promising results discovering models that are both accurate and fast. For NAS, training a one-shot model has became a popular strategy to approximate the quality of multiple architectures (child models) using a single set of shared weights. To avoid performance degradation due to parameter sharing, most existing methods have a two-stage workflow where the best child model induced from the one-shot model has to be retrained or finetuned. In this work, we propose BigNAS, an approach that simplifies this workflow and scales up neural architecture search to target a wide range of model sizes simultaneously. We propose several techniques to bridge the gap between the distinct initialization and learning dynamics across small and big models with shared parameters, which enable us to train a single-stage model: a single model from which we can directly slice high-quality child models without retraining or finetuning. With BigNAS we are able to train a single set of shared weights on ImageNet and use these weights to obtain child models whose sizes range from 200 to 1000 MFLOPs. Our discovered model family, BigNASModels, achieve top-1 accuracies ranging from 76.5% to 80.9%, surpassing all state-of-the-art models in this range including EfficientNets.

Parameter sharing (PS) is an important approach to speed up neural network search (NAS), which further allows the development of differential architecture search (e.g., DARTS) methods. However, PS also deteriorates the performance of learning models on the validation/testing set.
This paper first changes the search space from a DAG (micro+marco) in e.g., DARTS to a stacked one based on MBConv; and then, propose to use several tricks to train the super-net well. Finally, a search method is constructed for the supernet to find the desired architectures.
Overall, the paper is too experimental. The method is an ensemble of existing approaches, i.e., every single component in the paper has been visited in the literature. Expect for experimental results, I do not see many general lessons we can learn from the paper. Finally, why the proposed method can be better than others is not well-explained and clarified.
Please see the questions below:
Q1. Is NAS a method only for ImageNet? Can the method generalize to more applications/datasets?
- While ImageNet is a good dataset for CV experiments, I think NAS should be a method for deriving architectures with certain requirements.
- So, with so many tricks proposed in the paper, I wish authors can carry on experiments on other data sets as well, e.g., CIFAR and MNIST, which can still be preferred.
Q2. On motivation, could authors explain more about the difficulties of combining all these techniques?
- Each method is brought from some other paper, what motivate authors to combine them together? What makes them believe this is possible?
Q3. On presentation, could authors draw a figure of the search space in the main text and give an overall algorithm for Section "3.2 COARSE-TO-FINE ARCHITECTURE SELECTION". It is hard for a reader to see novelties there.
Q4. "We also use the swish activation (Ramachandran et al., 2017) and fixed AutoAugment V0 policy" - are all other compared methods using swish activation and AutoAugment V0 policy?
Q5. How about the search efficiency of the proposed method? Only the accuracy is reported in the paper.
Q6. Could authors give STD (i.e., gray area to represent STD) in Figure 4, 5 and 7? Some curves are too close, I am not sure they are statistically different.
Q7. How is the performance of the super-net?
Q8. Could the authors add an ablation study on this point? -  "The motivation is to improve all child models in our search space simultaneously, by pushing up both the performance lower bound (the smallest child model) and the performance upper bound (the biggest child model) across all child models."
- It is important to avoid fine-tune
- From the paper, I am not sure whether the problem is solved by changing the space or the proposed training method (See Q1).


Simulation of molecular and crystalline systems enables insight into interesting chemical properties that benefit processes ranging from drug discovery to material synthesis. However, these simulations can be computationally expensive and time consuming despite the approximations through Density Functional Theory (DFT). We propose the Valence Interaction Message Passing Neural Network (VIMPNN) to approximate DFT's potential energy calculations. VIMPNN integrates prior physics knowledge, namely the information exchange between atoms being driven and modulated by different interatomic bonds, and the relevance of specific physical properties to the problem of estimating more accurate energies. Furthermore, while many previous machine learning methods consider only stable systems, our proposed method is demonstrated on unstable systems at different interatomic distances. VIMPNN predictions can be used to determine the stable locations for atoms, a necessary step for the future simulation of crystal growth. Our method is extensively evaluated on an augmented version of the QM9 dataset that includes unstable molecules, as well as 2 new datasets of periodic and growing crystals. VIMPNN has comparable accuracy with DFT, while allowing for 5 orders of magnitude in computational speed up. It also produces more accurate and informative potential energy curves than Message Passing Neural Network.

The paper tackles the problem of estimating the electronic structure's ground state energy of a given atomic system by means of supervised machine learning, as a fast alternative to conventional explicit methods (DFT).
This is done by improving on a previous method, MPNN, Message Passing Neural Networks, and in particular by including information on bond type as input, so that the NN can learn the appropriate weight for messages going through bonds of that type.  In addition, training on several target labels (multi-regression) is attempted, with the idea that more physical outputs may help building better hidden representations (on this, there are mixed results).  Separately, training sets are enriched with non-equilibrium structures, so as to confront the NN with more diverse data.
The method is tested on 3 kind of training sets. The first is a simple but time-consuming augmentation of QM9, with inter-atomic distances varied, so as to increase the training set's size (and in particular, including non equilibrium configurations). The second consists in a periodic and thus infinite simple crystal structure (with, again, variations in inter-atomic distances, enriching the dataset). The third is a pseudo-cristalline structure with atoms randomly placed on a regular grid, forming a somewhat random structure, also named crystal (this is not a very good name).
The paper is overall rather well written, sometimes being a bit cumbersome (long sentences), but mostly it is stating clearly what is done or discovered.  The work is situated within the existing literature (that I am not familiar with at all).  The idea of using physics to guide architecture choices is gaining a lot of attention recently and seems to be well-suited to this particular problem, and well applied.  Several ways to use the bond type information have been attempted in this work, and several of them are reported and compared (a couple of them are discarded).  The results convincingly show that using bond type information indeed increases performance, both for small systems and for regular crystals.  The impact of performing multi-regression is less important, but still positive.  For large ''random crystals'', the method does not perform very well, and this represents a challenge for future work. Such a confession on the method's limitations is welcomed.
Given the idea (using physical information as bond type) is clearly and honestly presented, produces significant improvement compared with previous works, and has perspective for multiple future developments, I recommend acceptation of this paper.
There are however a number of points that could be improved.
1. There is a physical mistake that is not crucial but should be corrected, when talking about the ground state, and in particular before this sentence: ''accurate ground-state energy estimation of out-of-equilibrium molecule''. Ground state means minimal, T=0K energy level, so by definition it is at equilibrium. Thus, the sentence seems quite contradictory to a physicist.
What DFT and VIMPNN actually compute is the electronic structure's ground state's energy (at fixed positions of the atom kernels). I think this distinction should be mentioned just once, and then you could proceed with saying ground state energy.
Because of this, I would recommend to edit the title so as to suppress ''out of equilibrium'' from it.  Otherwise readers may think the method deals with non-equilibrium electronic structures (non ground states), which clearly it does not at all, or they may think that it is especially good at estimating energies for out of equilibrium systems, which is not its primary goal.
2. I do not understand very well the training procedure.  Also there are some tests that seem to be interesting and that are not performed (as far as I understood).
Does each training set contains the 90%-150% data augmenations, for each non-augmented training configuration ?
Why is training performed separately for each kind of data set ? Wouldn't the ultimate goal be to transfer learning from a type to another, e.g. from QM9-style to crystal style, etc ?  (as far as I understood, this was not done)
Isn't it interesting to see how much training on an augmented data set (let's say QM9) improves performance on the non-augmented data (the ''true data'' in a sense) ? Although the augmented data is obtained by DFT, and comparing models trained on different data sets is unfair, I think it may be interesting to see if VIMPNN benefits more than MPNN from this strategy (so compare the performance gains of both algorithms obtained by augmenting a data set). This kind of comparison may also be done for the ''augmentation'' of a training set by the concatenation of it with another one (although in that case it may be detrimental to the test accuracy?).
If you actually did some of this, then I misunderstood and I am sorry, but then this also means you should clarify.
3. Section 3.3:
I would not call this a crystal, but more something like ''random finite structure''. This should be done everywhere in the paper.
4. Section 4.1 is a bit too short for the inexperienced reader. I suggest to be a bit more explicit on what is learned
5. Section 4.2: It is nice to say you tried other ways, keep that.
However, try to be more explicit on what is shared and what isn't, in the architecture you finally pick. Is lambda(v,w) a common value for all bonds of the same type, like C-C ? Maybe you could provide an example or some more detailed notation to make your choice fully explicit (after all this is the core of the paper).
6. Section 4.3: could you quickly comment on why you don't use more of the 13 physical observables available in QM9 ?
7. Table 1: do the three last lines correspond to ''no BT information + a single auxiliary estimate'' ?  It seems to be the case, but then you say you will continue with the auxiliaries, in addition to the BT information.  Why don't you display the result of using BT information AND the 3 auxiliary estimates ?  If you did, then I misunderstood, but it would also mean you did not explain well enough a counter-intuitive result (which would be that adding auxiliary information actually hurts the performance of the VIMPNN).
8. Please include a couple of explicit numbers of your training/test/validation sets sizes.
9. section 5.2 could be made more concise. In particular, there is no need in repeating what can be seen directly in the figures (stating numbers). It is useful to comment on the meaning of the results however (as you currently do).
10. section 5.4  is a good idea, promising, however it does not really conclude into a very strong statement, and takes essentially 1 full page, which could be used to better clarify the architecture and/or the training procedure (or to reduce towards the ideal page length).


Deep convolutional neural networks trained for image object categorization have shown remarkable similarities with representations found across the primate ventral visual stream. Yet, artificial and biological networks still exhibit important differences. Here we investigate one such property: increasing invariance to identity-preserving image transformations found along the ventral stream. Despite theoretical evidence that invariance should emerge naturally from the optimization process, we present empirical evidence that the activations of convolutional neural networks trained for object categorization are not robust to identity-preserving image transformations commonly used in data augmentation. As a solution, we propose data augmentation invariance, an unsupervised learning objective which improves the robustness of the learned representations by promoting the similarity between the activations of augmented image samples. Our results show that this approach is a simple, yet effective and efficient (10 % increase in training time) way of increasing the invariance of the models while obtaining similar categorization performance.

The paper proposes to explicitly improve the robustness of image-classification models to invariant transformations, via a secondary multi-task objective. The idea is that the secondary objective makes intermediate-layer representations invariant to transformations of the image that should lead to the same classification. The paper also establishes that the typical models do not actually learn such representations by themselves.
This is an interesting idea (I am not specialist enough on image classification to know whether this idea has been tried before.) and highly relevant to this conference. The paper is g. well-written, easy to read, and correct.
I do, however, rate it as Weak Accept only for one reason: I would expect that making the model more robust should improve classification accuracy. But according to the paper, accuracy does not improve (and even degrades slightly). The paper does not experimentally demonstrate that the proposed methods objectively improves the model.
In a sense, it is to be expected, since the model is no longer optimized 100% for the classification task.
I can think of three changes to the paper that would flip my review to a Strong Accept:
* Try using the multi-task objective as a pre-training, followed by fine-tuning without multi-task objective. This should foster robust internal representations while allowing to fully optimize for the classification task. Alternatively, you could anneal alpha to 0. Try whether this alleviates the losses on the tests that got worse, and leads to higher gains on the others.
* Maybe the robust models, while being worse on benchmarks, are better on real-life data, e.g. where training and test mismatches are higher. Can you find a test set that demonstrates a larger, reliable accuracy improvement from robustness?
* "However, note that the hyperparameters used in all cases were optimized to maximize performance in the original models, trained without data augmentation invariance. Therefore, it is reasonable to expect an improvement in the classification performance if e.g. the batch size or the learning rate schedule are better tuned for this new learning objective." -- Then that should be done.
Besides this, I have a few detailed feedback points:
Eq. (2): Using sigma for "invariance", which is the opposite of the usual meaning of sigma... I wish you had used a different symbol. Not a dealbreaker, but if you have a chance, it would be great to change to a different one.
"we normalize it by the average similarity with respect to the *other* images in the (test) set" -- If you use only *other* images, I think it is theoretically possible that sigma becomes negative. I think you should include the numerator image in the denominator as well. I understand that in practical terms, this is never going to be a problem, so it should be OK, no need to rerun anything.
"we first propose to perform in-batch data augmentation" -- This increases correlation of samples within the batch, and may therefore affect convergence. To be sure that this is not the cause of the degradation, it would be good to rerun the baseline with the same in-batch augmentation (but without the additional loss). Was that already done?
Figure 5: I was a little confused at first because I read this as the invariance score (like Figures 3 and 4), not the invariance loss. They seem to be opposite of each other. So I wondered why the right panel would show poorer invariance score as training progresses. Do you actually need the concept of "invariance score" at all? Or can you redefine the invariance score as a non-invariance score (1- of it), so that its polarity is the same as the invariance loss? If not, an easy fix would be to add "(lower=better)" to the caption of Fig. 5, and likewise ("higher=better") to Fig 3 and 4.
"DATA AUGMENTATION INVARIANCE" -- You really want more. You want to be robust to all valid transformations of the object in the images. Obviously it is not possible to augment data for that, but it would be good to state this somewhere as the idealized goal of this work, which you approximate by data augmentation.


Value estimation is a critical component of the reinforcement learning (RL) paradigm. The question of how to effectively learn predictors for value from data is one of the major problems studied by the RL community, and different approaches exploit structure in the problem domain in different ways. Model learning can make use of the rich transition structure present in sequences of observations, but this approach is usually not sensitive to the reward function. In contrast, model-free methods directly leverage the quantity of interest from the future but have to compose with a potentially weak scalar signal (an estimate of the return). In this paper we develop an approach for representation learning in RL that sits in between these two extremes: we propose to learn what to model in a way that can directly help value prediction. To this end we determine which features of the future trajectory provide useful information to predict the associated return. This provides us with tractable prediction targets that are directly relevant for a task, and can thus accelerate learning of the value function. The idea can be understood as reasoning, in hindsight, about which aspects of the future observations could help past value prediction. We show how this can help dramatically even in simple policy evaluation settings. We then test our approach at scale in challenging domains, including on 57 Atari 2600 games.

This paper presents a new model-based reinforcement learning method, termed hindsight modelling. The method works by training a value function which, in addition to depending on information available at the present time is conditioned on some learned embedding of a partial future trajectory. A model is then trained to predict the learned embedding based on information available at the current time-step. This predicted value is fed in place of the actual embedding to the same value model, to generate a value prediction for the current time-step. So instead of just learning a value function based on future returns, the method uses a two-step process of learning an embedding of value relevant information from the future and then learns to predict that embedding.
The paper gives some motivating examples of why and when such an approach could yield an advantage over standard value learning methods like Monte-Carlo or temporal difference learning. The basic idea is that when the returns obey some causal structure like X->Y->Z it may be easier to learn P(Y|X) and P(Z|Y) than to learn P(Z|X) directly. In particular, the authors point out that in the discrete case when Y takes relatively few values the size of the respective probability tables can be smaller in the former case than the latter. This motivates the approach of discovering a set of future variables to predict which are themselves predictive of return, rather than predicting the expected future return directly.
On a high level, I like the idea of specifically learning to model relevant aspects of the environment. However, I lean toward rejecting this work in its current form because I feel the motivation for when this particular method would be useful is unclear.
In particular, I don't really understand how this method could, in general, be expected to improve on regular bootstrapping. Why learn a prediction of the return at time t based on future information when we could just use the value function at a later time to improve the prediction at time t? It seems to me that the future value function itself concisely summarizes the information in the future state that is relevant for predicting the past return, while better exploiting the structure of the problem. Of course in cases like partial observability, it could be that the future value function lacks information from the past that is important for accurately predicting the return (for example in the portal example of this paper). However, if partial observability is really the case of interest, the method presented in this paper seems like a rather roundabout solution method. For example, instead of conditioning v+ on a future hidden state h_{t+k} (as the authors do in the experiments) perhaps one could simply condition the value function on a past hidden state h_{t-k} and obtain similar benefit from bootstrapping?
Aside from partial observability, for which I feel there are better approaches, the only situation I can understand the method having an advantage is when later states contain information which helps to predict earlier rewards. This is essentially the situation presented in the illustrative example. However, currently I feel such situations are rather contrived and unintuitive so I would need more supporting evidence to accept these situations as a good motivation.
On a deeper level, I don't see how the probability table motivation given in the introduction applies when what is being learned is an expectation (i.e. a value function) and not a distribution.
The approach also suffers from well-known issues with using the output of an expectation model of a variable as the input to a nonlinear function approximator in place of the variable itself. Namely, there is no guarantee that the expectation value of a variable is a possible value for the variable so giving it as input to a predictor trained on the variable itself could easily yield nonsense output in the stochastic case. As far as I can tell the method does nothing to mitigate this (please correct me if I'm wrong), so there is no reason to assume the method is generally applicable in settings with nontrivial stochasticity.
Despite these concerns, I feel the experiments are for the most part quite well thought out and executed. The paper is also quite well written, motivation issues aside, so I would not be upset if it was accepted with the hope that it leads to future work addressing the above-mentioned concerns.
If possible I think this paper would benefit significantly from a detailed explanation of how and when the proposed approach should be expected to improve on bootstrapping, including bootstrapping off a value function which uses an analogous architecture to v+.
Questions for authors:
Given the hyper-parameters of R2D2 deviate somewhat from those used in the original paper, and nothing is said about how they were chosen, how confident can we be that the observed advantage of hindsight modelling is not simply due to hyper-parameters being selected which are more favourable for the proposed method?
Given that you are not learning distributions but expectations in the form of value functions, how pertinent is the motivation of learning P(Y|X) and P(Z|Y) instead of P(Z|X) directly described in the introduction?
How much of the benefit observed in the portal example an ATARI could also be gained from simply providing the value function approximation with h_{t-k} as input to help span larger time-gaps?
Update:
While I still feel the exposition could be improved to make the underlying idea clearer, I feel the authors did a good job of addressing my major concerns in their reply, hence I have raised my score to a weak accept.
I have to admit I missed the point that v^+ and v^m were using entirely different parameter sets. In light of this, I agree that the expectation model issue I mentioned is not a major concern.
I also appreciate the clarification of the hyperparameters, if they were really tuned to improve the baseline then this detail should be added to the paper and would negate my concern there.
Finally, I thank the authors for providing the value-function oriented example. I found this example to be more illustrative than the one in the introduction of the paper, and I now feel that I have a better grasp of the motivation. I still have doubts about the general benefit of the approach over bootstrapping but since it is not entirely clear to me one way or the other I feel the idea at least warrants further exploration, and it would be reasonable to accept the paper to make the community aware of it.


Recent efforts in Dialogue State Tracking (DST) for task-oriented dialogues have progressed toward open-vocabulary or generation-based approaches where the models can generate slot value candidates from the dialogue history itself. These approaches have shown good performance gain, especially in complicated dialogue domains with dynamic slot values. However, they fall short in two aspects: (1) they do not allow models to explicitly learn signals across domains and slots to detect potential dependencies among (domain, slot) pairs; and (2) existing models follow auto-regressive approaches which incur high time cost when the dialogue evolves over multiple domains and multiple turns. In this paper, we propose a novel framework of Non-Autoregressive Dialog State Tracking (NADST) which can factor in potential dependencies among domains and slots to optimize the models towards better prediction of dialogue states as a complete set rather than separate slots. In particular, the non-autoregressive nature of our method not only enables decoding in parallel to significantly reduce the latency of DST for realtime dialogue response generation, but also detect dependencies among slots at token level in addition to slot and domain level. Our empirical results show that our model achieves the state-of-the-art joint accuracy across all domains on the MultiWOZ 2.1 corpus, and the latency of our model is an order of magnitude lower than the previous state of the art as the dialogue history extends over time.

The authors build on recent work for non-autoregressive encoder-decoder models in the context of machine translation (most significantly [Gu, et al., ICLR18]) and adapt this to dialogue state tracking. Specifically, as in [Gu, et al, ICLR18], they use a fertility decoder modified for DST to be on a per-slot basis which is input to a second decoder to generate the (open-vocabulary) tokens representing dialogue state. An interesting aspect of the resulting model as formulated is that the latent space also takes into account interdependencies between generated slot values, which leads to a direct structured prediction-like loss of joint accuracy. Additionally, as slot values have a smaller (and likely peakier) combinatorial space, NAT models actually are more applicable to DST than MT. The resulting model achieves state-of-the-art empirical results on the MultiWOZ dataset while incurring decoding times that are an order of magnitude faster.
Once one understands this conceptual modification of modifying the NAT string encoder-decoder to a more structured NAT encoder-decoder (which in DST is more of a normalized string), they apply all of the state-of-the-art techniques to build a DST system: gating of all potential (domain, slot) pairs as an auxiliary prediction (e.g., [Wu et al., ACL19]), de-lexicalizing defined value types [Lei, et al., ACL18], content encoder, and domain-slot encoder (with pretty standard hyper-parameters, etc.). The significant addition is the fertility decoder and the associated NAT state decoder. Thus, from a conceptual level, this isn’t a huge leap and something many researchers *could* have done (i.e., I think many people, including myself, to have expected this paper to come out) — thus, it is more of a ‘done first’ paper than a ‘done unexpectedly’ paper. However, it is done well and the results are convincing and interesting. Given the impressive performance, I expect others to continue building on this work and potentially even influencing people to combine encoder-decoder models with a more structured prediction approach to DST.  Thus, I would prefer to see it accepted if possible.
That being said, I do have a few questions regarding this work — but these are more questions that might be able to be addressed than actual criticisms per se. First, in Table 5, why without the delexicalized dialogue history does the performance drop from 49.04% to 39.45%? This does not make sense to me as the model is much more complex than TRADE; however, TRADE does not do delexicalization yet achieves 45.6% joint accuracy. Meanwhile, with such complex model, I would expect the model can learn from raw data without delexicalization. Moreover, the proposed method use both previous predictions and *previous system actions* to do delexicalization. Also, the NATMT models don’t do delexicalization (although they have significantly more data). I think the authors should do an ablation study that do not use previous system actions, because this is extra information compared with TRADE — even if delexicalizing. Secondly, another worthy baseline would be an autoregressive decoder (with other blocks of the model remain the same). I’d assume that the performance is slight higher. It is interesting to see the gap, because this gap is the cost to speed up decoder using fertility — even if it is a bit counter-intuitive. If there is no improvement in this setting, then structured prediction in general may make more sense. Honestly, I think more would be interested in the second point than the first.
In any case, nice paper — well-written, well-motivated, interesting empirical results. The only reason I am recommending ‘weak accept’ is that the novelty is fairly straightforward and the strength is in the execution.


Learning generative probabilistic models that can estimate the continuous density given a set of samples, and that can sample from that density, is one of the fundamental challenges in unsupervised machine learning. In this paper we introduce a new approach to obtain such models based on what we call denoising density estimators (DDEs). A DDE is a scalar function, parameterized by a neural network, that is efficiently trained to represent a kernel density estimator of the data. In addition, we show how to leverage DDEs to develop a novel approach to obtain generative models that sample from given densities. We prove that our algorithms to obtain both DDEs and generative models are guaranteed to converge to the correct solutions. Advantages of our approach include that we do not require specific network architectures like in normalizing flows, ODE solvers as in continuous normalizing flows, nor do we require adversarial training as in generative adversarial networks (GANs). Finally, we provide experimental results that demonstrate practical applications of our technique.

This paper presents an approach for learning density estimates of a data distribution convolved with noise through “denoising density estimators” and shows how to leverage these density estimates to train generative samplers. The methods are evaluated on toy low-d datasets, MNIST, and fashion MNIST where they show reasonable density estimates and OK sample quality.
My major concern with this paper is that the “denoising density estimator” proposed here is identical to denoising score matching (which is not cited or discussed). The second contribution of learning a sampler given a density estimate is interesting but likely suffers from all the instabilities of GAN training, and does not compare to related work on distilling energy-based models. Unless both these concerns are addressed, I cannot recommend this paper for acceptance.
Major comments:
* The idea of learning a generative model whose energy gradient (score) matches the gradient of the data distribution has been explored extensively under the name “score matching”. The proposed approach of “Deep Denoising Density Estimation” is *identical* to denoising score matching (Vincent et al., 2011, http://www.iro.umontreal.ca/~vincentp/Publications/smdae_techreport.pdf). There’s no discussion of any prior work on score matching in this paper, or comparison with recent approaches in this space (e.g. sliced score matching https://arxiv.org/abs/1905.07088, and https://arxiv.org/abs/1907.05600 that uses denoising score matching).
* The algorithm presented for learning a generator given an energy function is not compared to other implicit sampling approaches like MCMC. Additionally, the algorithm requires alternating density estimation with updating the generative model, which is quite similar to GAN-training alternating density ratio estimation with updating the generative model. Thus the proposed algorithm likely experiences similar instabilities and challenges in how to partially solve the density estimation step. There are no comparisons to any GAN-based approaches anywhere in the paper. That said, mixing DSMs and explicit generators is a neat area of research.
* The experiments are too limited, and do not compute any quantitative metrics on the image datasets.
Minor comments:
* “Defining property of PGMs is that they provide functionality to sample” ->  what about density estimates? Likelihood ratios? etc.
* Boltzmann machines don’t allow for efficient inference
* Intro could should spend more time discussing relation to energy-based models, score matching, noise-contrastive estimation
* Eqn 2 only holds for sigma^2 -> 0, please add and discuss this limitation
* Divergenc -> divergence, above eqn 12
* I found the math in section 4 very confusing. What’s the <> notation in this context? Do you need to introduce \Delta? Alternating density (ratio) estimation and generative model updates is super common in GAN literature and is not discussed here.
* Table 1: are the estimates of log-likelihood upper or lower bounds? Unlike other approaches, you don’t have exact likelihoods so I’m not sure your #s are comparable.
* 2048 samples per iteration -> batch size?
* “For faster convergence, we take 10 DDE gradient steps…”  -> please add an experiment showing how results are impacted by # of gradient descent steps. Is larger # steps always better?
* MNIST/Fashion MNIST samples aren’t that good to my eye and there are no quantitative metrics (e.g. KID, FID, IS)


Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks, which aim to learn novel concepts from very few examples. The hallucination process, however, is still far from generating effective samples for learning. In this work, we investigate two important requirements for the hallucinator -(i) precision: the generated examples should lead to good classifier performance, and (ii) collaboration: both the hallucinator and the classification component need to be trained jointly. By integrating these requirements as novel loss functions into a general meta-learning with hallucination framework, our model-agnostic PrecisE Collaborative hAlluciNator (PECAN) facilitates data hallucination to improve the performance of new classification tasks. Extensive experiments demonstrate state-of-the-art performance on competitive miniImageNet and ImageNet based few-shot benchmarks in various scenarios.

This paper describes a method that builds upon the work of Wang et al. It meta-learns to hallucinate additional samples for few-shot learning for classification tasks. Their two main insights of this paper are to propose a soft-precision term which compares the classifiers' predictions for all classes other than the ground truth class for both a few-shot training set and the hallucinated set and b) to introduce the idea of applying direct early supervision in the feature space in which the hallucination is conducted in addition to in the classifier embedding space. This allows for stronger supervision and prevents the hallucinated samples from not being representative of the classes. The authors show small, but consistent improvement in performance on two benchmarks: ImageNet and miniImageNet with two different network architectures versus various state-of-the-art meta-learning algorithms with and without hallucination. The authors have adequately cited and reviewed the existing literature. They have also conducted many experiments (both in the main paper and in the supplementary material) to show the superior performance of their approach versus the existing ones. Furthermore their ablation studies both for the type of soft precision loss and for their various individual losses are quite nice and thorough.
Overall the contribution of this paper is incremental over Wang et al and is mainly in the introduction of their new loss terms to regularize the hallucination process. This is clearly evident from Table 2 (comparing rows 1 and 3), where much of the performance gain is attained by including the l_learner^cls term versus the collaborative loss term (comparing row 1 and row 4).
Furthermore, I would like the author to answer the following two questions:
1. The authors claim that their method is general applicable to all meta-learning methods and can be combined with them. Yet, the meta learning methods that they apply it to: prototypical networks, prototypical matching networks and cosine classifiers are all metric-learning-based meta-learning techniques. I would like the authors to outline the procedure (and preferably also show experiments) for applying their proposed technique to meta-learning based techniques that do not involve learning a metric-embedding space and instead learn the learning procedure via nested optimization, e.g. MAML and its variants.
2. In Table 4, I would like to see the results of PNM w/G or in other words the results of (Wang et al, 2018)'s method in comparison to the authors' proposed method.
3. The authors make no attempt to solve the problem of hallucinating examples for regression tasks. This is fine as it is perhaps outside he scope of their current work. However, I would like the authors to fully qualify their claims everywhere in the paper and restrict the contribution of their work to classification tasks only.


Many approaches to hierarchical reinforcement learning aim to identify sub-goal structure in tasks. We consider an alternative perspective based on identifying behavioral 'motifs'-repeated action sequences that can be compressed to yield a compact code of action trajectories. We present a method for iteratively compressing action trajectories to learn nested behavioral hierarchies of arbitrary depth, with actions of arbitrary length. The learned temporally extended actions provide new action primitives that can participate in deeper hierarchies as the agent learns. We demonstrate the relevance of this approach for tasks with non-trivial hierarchical structure and show that the approach can be used to accelerate learning in recursively more complex tasks through transfer.

The paper proposes a method that aims at encoding trajectories (described as a sequence of actions) into a set of discrete codes with a hierarchical structure. The principle of the algorithm (as far as the article allows me to understand) is to apply multiple iterations of classical sparse coding over the trajectories. The experimental section on simple (deterministic) tasks shows that the SSC method is able to extract interesting options, which can then be used to learn faster on some close domains.
In terms of positioning, I find the idea of the paper interesting (i.e encoding trajectories through discrete symbols) since it uses sparse coding approaches which, as far as I know, are not classical in the RL domain. This type of approach can give us both a meaningful insight about the "nature" of the learned policy (as it is the case in the paper that compresses expert trajectories), and can also become a manner to constraint an RL algorithm to force it to exhibit behaviors that could seem more natural to humans.
But the way it is done in this article is disappointing. First of all, the article is badly written, and I am still not sure to fully understand how the algorithm exactly works. Indeed, many notations are not well defined (see at the end of the review), and it makes the algorithm 1 difficult to catch. Then, the authors consider that trajectories are represented as sequences of actions (using one-hot encoding) and do not discuss this hard choice: representing trajectories as a sequence of actions usually rely on the assumption that both the environment is deterministic, and the initial state is always the same. Is it the case in this paper? If it is, it clearly restricts the applicability of the technique. If it is not, then I don't see how it could work well... As far as I understand, all experimental environments are deterministic. So the algorithm description would clearly need to be rewritten, and the authors have to discuss the assumptions they are doing mainly: deterministic environments and also the fact that the "options" can only be extracted once a first policy has be learned (or by using expert traces) which limits its applicability.
In terms of experiments, the assumption made is that we have access to a set of 'good' trajectories (which is easy in the proposed environments, but may be difficult in the real-life). It is compared to the option-critic architecture which simultaneously learns the options and the policy and I think that the comparison is somehow unfair. Since SSC is more a "sequence compression" algorithm, I would prefer to compare with existing sequence compression algorithms like hierarchical recurrent neural networks for instance.  The results are illustrated in very simple environments and the article would gain by using more complex ones (for instance the Atari grand challenge dataset could be used for such a study). So it is difficult to understand if the approach as it is is really interesting and efficient for general RL purposes.
Summary: A good idea, but not well described, with strong assumptions not discussed, and with low-quality experimental results.
Some other minor remarks:
The introduction is a little bit messy and does not well allow one to understand the focus on the paper, mixing some notions of neuroscience with classical reinforcement learning aspects, the connection between the two domains being not trivial.
Equation 2 versus Equation 3: What is the difference?
s notation appears in 2.1 and 2.2 while it corresponds to different things. The variables are not defined and we don't know in which domain they rely on.
Articulation between sparse coding and MDL not clear (since sparse coding is directly a way to minimize the MDL). MDL never used after that.
section 3, paragraph 3: I do not understand what is described here. The description has to be rewritten to allow the readers to understand the algorithm e.g "the size of the dictionary elements is set to 2-timesteps. " ?  "The dictionary element a which has the highest explained variance is then selected and assigned an integer code n + 1 " Variance on what ?  what is T_i ?
[cite] appears in the introduction


Time series with missing data constitute an important setting for machine learning. The most successful prior approaches for modeling such time series are based on recurrent neural networks that learn to impute unobserved values and then treat the imputed values as observed. We start by introducing Recursive Input and State Estimation (RISE), a general framework that encompasses such prior approaches as specific instances. Since RISE instances tend to suffer from poor long-term performance as errors are amplified in feedback loops, we propose Direct Input and State Estimation (DISE), a novel framework in which input and state representations are learned from observed data only. The key to DISE is to include time information in representation learning, which enables the direct modeling of arbitrary future time steps by effectively skipping over missing values, rather than imputing them, thus overcoming the error amplification encountered by RISE methods. We benchmark instances of both frameworks on two forecasting tasks, observing that DISE achieves state-of-the-art performance on both.

The authors propose a new univariate time series analysis framework called RISE, which unifies the existing work on adapting RNNs to irregular time series. Building on top of the RISE, they propose a modification called DISE in which the algorithm skips the intervals without any observations. In that sense, DISE can be considered a marked point process analysis algorithm. They quantify the performance of the RISE and DISE on two datasets.
Table 1 is a valuable summary of the existing efforts on adapting RNNs to irregular time series. However, the paper overstates its scope. This work only studies RNNs. There are many alternatives for analysis of irregular time series, including Gaussian processes [1], ordinary differential equations [2], convolutional neural networks [3], neural point processes and spiking neural networks [4]. These references are only notable examples from each category and there are many more.
A major limitation of this paper is that it only applies to the univariate time series. Usually in domains such as healthcare, almost always different variables have different missing rates. Multiple works address the multivariate case, see [5] for example.
The main dataset used for evaluation is the Glucose dataset. However, this dataset is a peculiar and very specific dataset because its goal is to predict glucose level for type-1 diabetics only base on the past glucose measurements. While this task is meaningful, human biology states that forecasting glucose levels without knowledge of insulin injection or carbohydrate consumption is an extremely difficult task. In this setting, the most useful data is the latest data point. This dataset is an extreme forecasting task in absence of major predictors and I do not think it should be the primary dataset for evaluation of a new algorithm.
Finally, the main idea of skipping the intervals without measurements is not very novel given the existing literature on neural point processes. Also, it is not enough contribution for a full conference paper.
[1] Shukla, Marlin (2019) Interpolation-Prediction Networks for Irregularly Sampled Time Series. In ICLR.
[2] Chen, T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. K. (2018). Neural ordinary differential equations. In NeurIPS.
[3] Nguyen, P., Tran, T., Wickramasinghe, N., & Venkatesh, S. (2016). Deepr: a convolutional net for medical records. IEEE BHI.
[4] Islam, K. T., Shelton, C. R., Casse, J. I., & Wetzel, R. (2017). Marked point process for severity of illness assessment. In Machine Learning for Healthcare Conference.
[5] Che, Z., Purushotham, S., Li, G., Jiang, B., & Liu, Y. (2018). Hierarchical deep generative models for multi-rate multivariate time series. In ICML.