Online Structure Learning for Feed-Forward and Recurrent Sum-Product Networks
Sum-product networks have recently emerged as an attractive representation due to their dual view as a special type of deep neural network with clear semantics and a special type of probabilistic graphical model for which marginal inference is always tractable. These properties follow from the conditions of completeness and decomposability, which must be respected by the structure of the network. As a result, it is not easy to specify a valid sum-product network by hand and therefore structure learning techniques are typically used in practice. This paper describes a new online structure learning technique for feed-forward and recurrent SPNs. The algorithm is demonstrated on real-world datasets with continuous features and sequence datasets of varying length for which the best network architecture is not obvious.

This paper proposed an online learning algorithm for static and dynamic sum-product networks (SPNs), a type of probabilistic model with tractable inference. The authors essentially combine local structure search in SPNs with a hard variant of expectation-maximization [1]. The algorithm maintains empirical covariance estimates of product nodes and leverages statistical dependence tests to decide when to replace a product (factorized distribution) with either a new leaf or a mixture (sum node). The algorithm further includes a pruning mechanism in order to trim over-grown structures. The proposed method is called online Structure Learning with Running Average Update (oSLRAU).  Strengths: + the hard EM scheme, already used by Poon and Domingos [1], is an effective and efficient method for parameter learning in SPNs. While Poon and Domingos applied it to select a sub-structure in a pre-determined layout, its use for flexible structure learning is good idea.  + the structure updates by monitoring empirical means and covariances at product nodes is also a natural and elegant choice.  + together with network pruning, the proposed structure learning scheme is natural and has the potential to become a state-of-the-art method.  Weaknesses: - while the approach has certain potential strengths, see above, the paper does not elaborate on them in a satisfying way. This approach could break new ground in learning SPNs, but the paper is presented in a "yet-another-SPN-learning-algorithm" fashion. - the paper is overloaded: online learning of both parameters and structure in (static) SPNs is already an important topic far from being closed. The paper, however, additionally discusses dynamic SPNs in a somewhat superficial way. In that way, the paper is not focused on a particular problem. - the experimental evaluation is also rather unfocused. It is not clear to me why the particular tasks and comparison methods where chosen.  Quality: The approach in this paper is reasonably justified. In particular, it leverages the hard EM variant due to Poon & Domingos [1], which was successful in selecting a sub-structure in a large potential structure layout tailored to images. In this paper, this method is used in a completely dynamic way, which I find promising. The structural updates (replacing product nodes with multi-variate Gaussians or new sum nodes; pruning) are also natural and remind at structural search in SPNs [2,3]. However, these advantages are not elaborated in a satisfying manner. I would really like to see a serious attempt to establish a new state-of-the-art method for learning SPNs; no theoretical justification/motivation for oSLRAU is given (besides a Theorem in the supplementary, guaranteeing that the likelihood of the last single sample is guaranteed to be increased -- does this really matter?), nor is it empirically demonstrated that it improves state-of-the-art. In particular, an eager comparison with LearnSPN is left out. Additionally I don't see the benefit to also discuss oSLRAU for dynamic SPNs, in particular, as these are discussed in a quite superficial and incomprehensible way.   Clarity: The paper is reasonably clear, but could be improved at some parts. In particular, the paper does not give a focused impression (what is the goal of the paper), many choices are not motivated (e.g. why compare e.g. with NVP but not with LearnSPN), many relevant parts are deferred to the supplementary.  Originality: The paper is quite original, but it does not discuss prior art in a fully adequate and careful way.  Two examples: 1) "SPNs were first introduced in [Poon and Domgos, 2011] as new type of deep representation." While it is certainly true that Poon and Domingos introduced new insights and ideas to SPNs, it is crucial to acknowledge the work of Darwiche, e.g. [4]. 2) "SPNs are equivalent to Bayesian networks and Markov networks [Zhao et al., 2015], ..." First it is not at all explained in what sense SPNs are supposed to be equivalent to Bayesian networks and Markov networks, and second, this is not what Zhao et al. have discussed. SPNs are certainly not equivalent to Markov network (in whatever sense), and also not to Bayesian networks. Zhao et al. have shown the SPNs can be cast into special Bayesian networks incorporating a layer of latent variables. Furthermore, it is clear that Bayesian networks over discrete variables using CPTs can also be easily mapped to SPNs, but, clearly, somewhat more exotic Bayesian networks are not easily mapped to SPNs.   Significance: The ideas in this paper have the potential to establish new state-of-the-art and are therefore of potential significance. However, as mentioned above, due to the rather unfocused presentation, motivation, and empirical evaluation, the paper does not unfold this potential.  [1] Poon, H. and Domingos, P., "Sum-Product Networks: A new deep architecture", UAI, 2011. [2] Dennis, A. and Ventura, D., "Greedy Structure Search for Sum-Product Networks", IJCAI, 2015. [3] Peharz, R. and Gens, R. and Domingos, P. "Learning selective sum-product networks", LTPM, 2014. [4] Darwiche, A., "A differential approach to inference in Bayesian networks", ACM, 2003.    ***EDIT***  The authors (and reviewer #1) have discussed NP-hardness of hard EM.   Please note that hard EM is *not* NP-hard in SPNs.   In fact, it is well tractable. To see this, assume that X are the observed variables and Z are the latent variables associated with the sum nodes. Hard-EM is like soft-EM, but where the expectation over the complete log-likelihood w.r.t. the posterior P(Z | X) is replaced with the MAP solution of P(Z | X), i.e.  argmax_theta E_{P(Z | X ; theta_old)} [log P(X,Z)]       (soft EM) becomes argmax_theta log P( X, argmax_Z(P(X, Z ; theta_old)) )   (hard EM)  where theta are all parameters (sum-weights, params of input distributions). The critical bit is computing  Z* = argmax_Z(P(X, Z ; theta_old)), i.e. the most likely joint state of the latent variables (which determines an induced tree, see Zhao et al. [1]) given a sample for X. This inference query is well tractable. In fact, already Poon and Domingos [2] gave the correct algorithm for this (without proof): simply replace sum nodes with max nodes in the upwards pass and perform max-backtracking. That this algorithm is indeed correct was proved by Peharz et al. [3], Theorem 2: While reviewer #1 is right that MAP inference is in general NP-hard, inferring Z* is a notable exception, since the augmented SPN (the SPN which makes the latent variables explicit [3]) is *selective*. Note that the augmented SPN does not need to be constructed explicitly. It is clear that given Z* for each sample, updating the parameters is easy, in fact closed form.  Also note that the same (exact) inference of Z* as required for hard EM was used by Vergari et al. [4], with the goal to infer a data representation.  The authors also discussed a variant of hard EM used by Poon and Domingos [2], where sums are used in the upwards pass. It is rather clear that this is a kind of nested hard EM: perform MAP inference of the top sum layer, with the latent variables below marginalized, fix (condition on) this assignment for the top layer, and continue with MAP inference in the next layer, and so forth.  It would be desirable that the authors incorporate this well-established theory in their paper, since hard EM is the central working horse of their method.   A further point of importance is the connection between Bayesian networks, Markov networks and SPNs.   Reviewer #1 stated: """ "SPNs are equivalent to Bayesian and Markov networks" - this might be misleading, since to represent the same domain, one may need an exponentially large SPN in order to represent the same as a BN or MRF. So we should be careful with the "equivalence" usage. """  I stated: """ 2) "SPNs are equivalent to Bayesian networks and Markov networks [Zhao et al., 2015], ..." First it is not at all explained in what sense SPNs are supposed to be equivalent to Bayesian networks and Markov networks, and second, this is not what Zhao et al. have discussed. SPNs are certainly not equivalent to Markov network (in whatever sense), and also not to Bayesian networks. Zhao et al. have shown the SPNs can be cast into special Bayesian networks incorporating a layer of latent variables. Furthermore, it is clear that Bayesian networks over discrete variables using CPTs can also be easily mapped to SPNs, but, clearly, somewhat more exotic Bayesian networks are not easily mapped to SPNs.  """  The authors were evidently not alerted by the fact that two reviewers were independently puzzled by their claim. According to common understanding, Markov networks and Bayesian networks are *not* equivalent, see any common text book on graphical models [5]. So they cannot both be equivalent to SPNs.  The notion of equivalence used by the authors, by which two models are equivalent whenever they can be convert into each other -- possibly under an exponential blowup -- is not meaningful. Sure, a fully connected BN can represent any distribution, so can a fully connected Markov network. This does *not* make these models equivalent.  The authors stated that they are not aware of more exotic variants of BNs which cannot obviously be converted into an SPN. Take for example a BN over continuous variables, whose conditional distributions are Gamma distributions, parametrized via neural networks taking the parents' values as inputs (note that BNs are not restricted to discrete variables, so this perfectly defines a BN). It is not clear to me how to convert this BN into an SPN. Moreover, how would the authors convert a restricted Boltzmann machine (which is a MN) into an SPN? This will be hard, given the intractability of the RBM's partition function.  Claiming equivalence between SPNs, BNs and MNs in the very introduction of the paper is a deepity, at best.    The authors kindly invited me to re-read their paper, and re-iterated their contributions. However, the "what" was not the problem for me in the first place, but rather the "why". As said in my review, the use of hard EM for online structure learning is very promising and already challenging on its own. As illustrated above, there is also a lot of theory on this topic which should be addressed and build on. To additionally apply the framework to dynamic SPNs makes the paper rather convoluted and unfocused. Perhaps this is just personal opinion, but the authors did not explain their motivation in a satisfying manner.   Due to the the shortcomings discussed above, I stick with my initial score.     [1] Zhao et al., On the relationship between SPNs and BNs, 2015. [2] Poon and DOmingos, SPNs: a new deep architeccture, 2011. [3] Peharz et al., On the latent variable interpretation in SPNs, 2017. [4] Vergari et al., Sum-Product autoencoding, 2018. [5] Koller and Friedman, Probabilistic Graphical Models, 2009.


Relational recurrent neural networks
Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods. It is unclear, however, whether they also have an ability to perform complex relational reasoning with the information they remember. Here, we first confirm our intuitions that standard memory architectures may struggle at tasks that heavily involve an understanding of the ways in which entities are connected -i.e., tasks involving relational reasoning. We then improve upon these deficits by using a new memory module -a Relational Memory Core (RMC) -which employs multi-head dot product attention to allow memories to interact. Finally, we test the RMC on a suite of tasks that may profit from more capable relational reasoning across sequential information, and show large gains in RL domains (e.g. Mini PacMan), program evaluation, and language modeling, achieving state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord datasets.

This paper  proposes relational recurrent neural networks to incorporate relational reasoning into memory-based neural networks. More specifically, the authors design a new memory module, called relational memory core (RMC), to make memories to interact. Multi-head dot product attention is used for this purpose.  Experiments are constructed on a few supervised learning and reinforcement learning tasks, which may profit from more capable relational reasoning across sequential information, to show the effectiveness of the proposed method.  The main strengths of this paper are as follows: (1) It is a novel and well motivated idea to design a new deep learning architecture which can perform relational reasoning. (2) The proposed network structure, RMC, is a reasonable memory module to allow the memories to interact.   (3) The experiments are extensive. The results show explicit performance improvements.  The main weaknesses of this paper are as follows: (1) Some claimed points are not very understandable to me. First, it is unclear how the intuition (that memory architectures involving relational reasoning benefit some learning tasks) can be confirmed. Second, how can you claim that the proposed method has the capacity of relation reasoning from the evidence of "improved function". These points are the footstones of this paper, which should be clearly analyzed, instead of simple explanation.  (2) The analysis of (hyper)parameters should be analyzed in the experiments. It will be nice to show the influence of of the attention model with different settings. (3) The presentation can be improved. For example, Figure 3 are not clear enough for understanding. (4) Another paper titled "Relation Deep Reinforcement Learning" is closely related to this one, thus should be cited and discussed.


Thwarting Adversarial Examples: An L 0 -Robust Sparse Fourier Transform
We give a new algorithm for approximating the Discrete Fourier transform of an approximately sparse signal that has been corrupted by worst-case L 0 noise, namely a bounded number of coordinates of the signal have been corrupted arbitrarily. Our techniques generalize to a wide range of linear transformations that are used in data analysis such as the Discrete Cosine and Sine transforms, the Hadamard transform, and their high-dimensional analogs. We use our algorithm to successfully defend against well known L 0 adversaries in the setting of image classification. We give experimental results on the Jacobian-based Saliency Map Attack (JSMA) and the Carlini Wagner (CW) L 0 attack on the MNIST and Fashion-MNIST datasets as well as the Adversarial Patch on the ImageNet dataset.

The authors use tools from compressive sensing to defend classifiers against adversarial corruption of images with L0 constrained attacks. Specifically, assuming the input image is approximately sparse in the Fourier domain, they can approximately recover the adversarial perturbations and the large Fourier coefficients of the input image. They test their defense method on known L0 attacks and they are able to defend against these attacks.  I found their defense algorithm to be a clever use of compressive sensing. They show that the observed signal y is a matrix transform of the input and errors. They then show that this particular matrix satisfies the RIP condition for the input model of interest. Typically we show the RIP condition of random low-rank matrices. It was interesting to exploit the RIP condition of this particular deterministic and relatively high-rank matrix.  My issue with this work is that the theoretical results do not precisely imply defense against L0 attacks. It only shows that we can approximately recover the large Fourier coefficients of the image. While the proposed algorithm does defend against known attacks, perhaps with knowledge of the defense method it may be possible to still attack the classifier. For example in Figure 4 there are some large artifacts after the recovery. An attack may be able to exploit the artifacts left over after running the recovery algorithm.  Overall, I believe that this is a good contribution to the area with some nice ideas. However there is still work to be done and I would not be surprised if an attack is developed that can bypass this defense.  *** Post Rebuttal ***  Thank you for your response. Since this work only allows approximate recovery in L_inf, and we know that there exists adversarial examples in L_inf, I still think that an attacker with knowledge of the defense can still cause the network to output the wrong result. I still think that this work makes a nice contribution to the space.


Joint Sub-bands Learning with Clique Structures for Wavelet Domain Super-Resolution
Convolutional neural networks (CNNs) have recently achieved great success in single-image super-resolution (SISR). However, these methods tend to produce over-smoothed outputs and miss some textural details. To solve these problems, we propose the Super-Resolution CliqueNet (SRCliqueNet) to reconstruct the high resolution (HR) image with better textural details in the wavelet domain. The proposed SRCliqueNet firstly extracts a set of feature maps from the low resolution (LR) image by the clique blocks group. Then we send the set of feature maps to the clique up-sampling module to reconstruct the HR image. The clique upsampling module consists of four sub-nets which predict the high resolution wavelet coefficients of four sub-bands. Since we consider the edge feature properties of four sub-bands, the four sub-nets are connected to the others so that they can learn the coefficients of four sub-bands jointly. Finally we apply inverse discrete wavelet transform (IDWT) to the output of four sub-nets at the end of the clique up-sampling module to increase the resolution and reconstruct the HR image. Extensive quantitative and qualitative experiments on benchmark datasets show that our method achieves superior performance over the state-of-the-art methods.

Summary:  This paper proposes a CNN architecture called SRCliqueNet for single-image super-resolution (SISR), and it consists of two key parts, feature embedding net (FEN) and the image reconstruction net (IRN).   FEN consists of two convolutional layers and a clique block group. The first convolutional layer tries to increase the number of channels of input, which can be added with the output of the clique block group via the skip connection. The second convlutional layer tries to change the number of channels so that they can fit the input of clique block group. The clique block group concatenates features from a sequence of clique blocks, each of which has two stages: 1) the first stage does the same things as dense block, while the second stage distills the feature further. Following the idea of resnet, a clique block contains more skip connections compared with a dense block, so the information among layers can be more easily propagated.  IRN consists of a clique up-sampling module and a convolutional layer used to reduce the number of feature maps to reconstruct the HR image with RGB channels. The clique up-sampling module contains four sub networks, representing four sub-bands, LL, LH, HL and HH, in the wavelet domain, respectively. Each sub network has 1) subband extraction, 2) residual block, 3) a refinement stage that uses the high frequency blocks to refine/learn the low frequency blocks, 4) IDWT, and 5) a convolutional layer, which is used to reduce the number of channels and get the HR image.  Experiment evaluations on benchmark datasets show that that SRCliqueNet performs better than the state-of-the-art SR algorithms in terms of quantitative metrics.  Strengths:  - A novel idea of modeling the relationship among the four sub-bands in the clique up-sampling module in IRN. - This paper is technically sound with extensive quantitative evaluation of state-of-the-art SR algorithms on benchmark dataset, and experiment and visualization to demonstrate the usefulness of FEN and IRN. - This paper is well write with clear description of how SRCliqueNet works.  Weaknesses:  Overall it is a great paper. It will be good if training and testing time is also reported. This is helpful to know if it can be used in real-time SR.  Rebuttal Response:  After reading the review, I am more clear on the testing time of this approach. For now, it appears that it may not be fast enough to process real time video frame (1 second / image now based on rebuttal), but it does not decrease the value of this work to use DL on wavelet domain. I will remain my score to be 8.


FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network
This paper develops the FastRNN and FastGRNN algorithms to address the twin RNN limitations of inaccurate training and inefficient prediction. Previous approaches have improved accuracy at the expense of prediction costs making them infeasible for resource-constrained and real-time applications. Unitary RNNs have increased accuracy somewhat by restricting the range of the state transition matrix's singular values but have also increased the model size as they require a larger number of hidden units to make up for the loss in expressive power. Gated RNNs have obtained state-of-the-art accuracies by adding extra parameters thereby resulting in even larger models. FastRNN addresses these limitations by adding a residual connection that does not constrain the range of the singular values explicitly and has only two extra scalar parameters. FastGRNN then extends the residual connection to a gate by reusing the RNN matrices to match state-of-the-art gated RNN accuracies but with a 2-4x smaller model. Enforcing FastGRNN's matrices to be low-rank, sparse and quantized resulted in accurate models that could be up to 35x smaller than leading gated and unitary RNNs. This allowed FastGRNN to accurately recognize the "Hey Cortana" wakeword with a 1 KB model and to be deployed on severely resource-constrained IoT microcontrollers too tiny to store other RNN models. FastGRNN's code is available at [30].

This paper presents an adaptation to the peep-hole connection from [22] to create fast, accurate, and, small RNNs. There are two variants introduced:   (1) FastRNN which adds two extra learnable parameters to the vanilla RNN to regulate the computation flow between the hidden state and the nonlinear mapping of inputs and states  (2) FastGRNN which replaces the two parameters with gating functions which share input/state matrices with the nonlinear projection but have separate biases.  Furthermore, the FastGRNN utilises low-rank sparse representation for matrices with constraints that helps with compressing the size of the model.  Theoretical analysis studies the convergence and stability of these models vs normal RNNs while the extensive experimental setup shows that these models are capable of achieving comparable results to state-of-the-art or comparable models (e.g. LSTMs) with much smaller network sizes.  I really like the paper and the results reported and I'm sure this is going to have a great impact in the field. I can suggest a few improvements that could make the paper even better:  - While the performance of these models have been studied in comparison to competing RNN formulations, the effects of regularisation have not been analysed. For example, the results reported on PTB are far worse than where the SOTA is and recent results on those tasks show that heavy regularisation of LSTMs for example can result in massive improvements in performance ("Regularizing and Optimizing LSTM Language Models", Merity et al 2017). So how would these models behave under heavy regularisation if one wants to attain high performances? - Also there are other work in this area such as "Quasi-Recurrent Neural Networks", Bradbury et al 2017 which also try to provide faster alternatives to LSTMs. How does FastGRNN perform against these methods? - While the balance between \alpha and \beta has been discussed briefly in the paper, it would have been better if we had further experimental results on them. Fig 4 in appendix is a good start, but what about \beta, how does that behave? Do we see often that \beta = 1 - \alpha as speculated in the paper? How does the ratio \alpha / \beta change with sequence length or different datasets? There are many interesting results here that would be good to have a report on - The paper has considerable redundancy in the introduction Sec 1 and related work Sec 2 which can be condensed. This will provide space to bring forward some of the interesting findings and results from appendix such as Fig 4 which.


On Learning Markov Chains
The problem of estimating an unknown discrete distribution from its samples is a fundamental tenet of statistical learning. Over the past decade, it attracted significant research effort and has been solved for a variety of divergence measures. Surprisingly, an equally important problem, estimating an unknown Markov chain from its samples, is still far from understood. We consider two problems related to the min-max risk (expected loss) of estimating an unknown k-state Markov chain from its n sequential samples: predicting the conditional distribution of the next sample with respect to the KL-divergence, and estimating the transition matrix with respect to a natural loss induced by KL or a more general f -divergence measure. For the first measure, we determine the min-max prediction risk to within a linear factor in the alphabet size, showing it is Ω(k log log n/n) and O(k 2 log log n/n). For the second, if the transition probabilities can be arbitrarily small, then only trivial uniform risk upper bounds can be derived. We therefore consider transition probabilities that are bounded away from zero, and resolve the problem for essentially all sufficiently smooth f -divergences, including KL-, L 2 -, Chi-squared, Hellinger, and Alpha-divergences.

Summary: The paper’s goal is to study the minimax rates for learning problems on Markovian data. The author(s) consider an interesting setting where the sequence of data observed follow a Markovian dependency pattern. They consider discrete state Markov chains with a state space [k] and study the minimax error rates for the following two tasks: Prediction: Given a trajectory X_1 -> X_2  … -> X_n from an unknown chain M, predict the probability distribution of the next state X_n+1, i.e., P(. | X_n). Measured by  Prediction risk  = E[ Loss(P(. | X_n) , P_hat(. | X_1…n)) ] where the expectation is over the trajectory X_1…X_n. The loss function the paper focuses on is KL-divergence and presents a conjecture for how the L_1 loss should scale with respect to k and n. Estimation:  Given a trajectory X_1 -> X_2 … -> X_n from the unknown chain M, estimate the transition matrix M (call it M_hat). Goal is do well under the following loss: Estimation risk = max_{i \in [k]} E[ Loss( M( i , . ) , M_hat( i , . )) ]         If the Markov chain is such that some states are observed infrequently, this affects the estimation problem but not so much the prediction problem as a trajectory ending in such a state will be rarely observed and we only want to do well on the average trajectory hence we can afford to do badly on such ‘rare’ states. Summary of results: The paper has upper and lower bounds for the minimax rates for the prediction task which are off by a factor of k. The authors conjecture that the right dependence is probably that of the upper bound. The paper gives tight results for the estimation problem for loss functions which are ordinary f-divergences and also has additional results for learning under l_2 loss.  Overall the paper is well-written and presents the results well. It uses some interesting tools giving bounds on surprise probabilities and concentration inequalities for number of visits to a state in a Markov chain. Adding some intuition as to why the stated minimax rates are achieved would be nice.  Comments: It would be nice to have a more comprehensive related work section (at least in the supplementary material). The asymptotic analysis of the minimax rates for the prediction problem seem to be ignoring an important term which decays as O_k(1/n) in the interest of the marginally dominating term O_k(loglog n/ n). Ignoring the 1/n decay term and quantifying the exact dependence on k with respect to the loglogn/n seems to not really be capturing the whole picture with respect to the dependence on k. For instance if the 1/n decaying term has a k^3 dependence for instance, then this term would dominate the sample complexity until n > 2^2^k which can be huge. Since the paper’s main contribution for this problem appears to be identifying the range of dependencies on k, it feels like identifying the dependence on k for the O_k(1/n) term is also important. The estimation problem as formulated by the authors seems to be somewhat harsh in the notion of the risk they use. A more natural risk function to study would seem to be  \sum_{i \in [k]} \pi_i * E[ Loss( M( i , . ) , M_hat( i , . )) ] as it gives less importance to states we rarely visit and hence are hard to estimate accurate transitions from anyway. Any justification for why the chosen risk function is the natural one? Moreover, the harsh risk function used intuitively makes the lower bound somewhat straightforward. The lower bound construction essentially hides differences in one node of the chain and ensures that it is the least frequently visited node then the total sample complexity . Also the upper bound doesn’t seem very surprising given that once we wait a mixing time period (the paper state ignore the first sqrt(n) samples but treats n as increasing with the mixing time and stationary probabilities remain fixed, hence this can be interpreted as waiting for a mixing time steps?) concentration results on the number of visits to a state kick in and the problem seems to reduce to solving k IID distribution estimation problems?  Experiments: For the prediction task, it feels the right experiment to perform would have been one which scales up k the number of states in the chain as the main contribution of the paper compared to previous work is identifying the dependence on k. Also for the chosen small values of n (between 250 and 1000) it is not clear if the theoretical results for the prediction task are really being verified since  a) the theory is developed in an asymptotic manner and 1000 is too small a sample size to see a difference of loglog n factors,  and  b) since the tailing term is O_k(1/n) which yields a similar scaling down behavior to that of O_k(loglog n/n) for the chosen k value and range of n.  A discussion of the proof intuitions would be nice to have in the main body. It was hard to get intuition from reading the proofs in the supplementary material.  Also some intuition on what might be the rates for ordinary f-divergences beyond KL for the prediction tasks would be nice to have. If the problem becomes much harder with other f-divergences, some intuition as to why that is the case might be helpful.  Overall, I feel that this paper explores an interesting direction but the results seem a bit incremental/incomplete (for the prediction task) and somewhat straightforward for the estimation task given the choice of the risk function (bypasses having to deal with the Markovian nature of the data)


Adaptive Methods for Nonconvex Optimization
Adaptive gradient methods that rely on scaling gradients down by the square root of exponential moving averages of past squared gradients, such RMSPROP, ADAM, ADADELTA have found wide application in optimizing the nonconvex problems that arise in deep learning. However, it has been recently demonstrated that such methods can fail to converge even in simple convex optimization settings. In this work, we provide a new analysis of such methods applied to nonconvex stochastic optimization problems, characterizing the effect of increasing minibatch size. Our analysis shows that under this scenario such methods do converge to stationarity up to the statistical limit of variance in the stochastic gradients (scaled by a constant factor). In particular, our result implies that increasing minibatch sizes enables convergence, thus providing a way to circumvent the nonconvergence issues. Furthermore, we provide a new adaptive optimization algorithm, YOGI, which controls the increase in effective learning rate, leading to even better performance with similar theoretical guarantees on convergence. Extensive experiments show that YOGI with very little hyperparameter tuning outperforms methods such as ADAM in several challenging machine learning tasks.

Summary: This paper provides new theoretical insights for non-convex problems for the optimizer Adam and Yogi (the proposed optimizer).  In particular, the non-convergence of Adam is tackled with increasing batch size. Yogi provides more control than Adam in tuning the effective learning rate. The authors also present experiments comparing Yogi with Adam.   The paper is very clear in terms of writing.  Yogi idea is original but not sufficiently motivated. The work presented is not very significant due to certain weaknesses (see below).  Strengths: 1) Theoretical results for non-convex problems for Adam (actually RMSProp) and Yogi are interesting. 2) Many experiments were presented though not sufficient (can be biased due to hyperparameter selection).  Weaknesses: 1) Considering g_t  only rather than m_t in update step (in Appendix) makes the update step more similar to RMSProp, actually Adam will be exactly the same as RMSProp when beta_1 = 0 (if one excludes debiasing step). So there is no sync between theory and experiments. I would prefer the algorithm being called RMSProp rather than Adam in theoretical results.  In discussion, the authors mention that they set beta_1 = 0.9 inspired from theoretical analysis, but in theory they set beta_1=0. This needs to be clarified and should be consistent. 2) No proper motivation for the update step, the explanation for the update step is very minor.  I think it would be better if the authors dwell a bit more into the details what makes the algorithm special for instance explain different cases especially for different cases of v_{t-1} and g_{t}^2 where Adam fails and Yogi does it better. Since, there is only a minor modification in the update step, as compared to RMSprop, this is a critical issue. The formula itself doesn't look like a natural choice, thus it needs to be justified. 3) In theoretical results (both in Adam and Yogi), there is O(G^2/\epsilon^2) term as coefficient for variance, which can be very huge if epsilon is small. And usually epsilon is very small. It would be nice to see if one can decrease the dependence on epsilon. Regarding this aspect there is no improvement compared to the literature. 4) Weak experiments section, there is no proper rule in the choice of learning rate and \epsilon. I think one must choose the  tuned hyperparameters from a set (need not be very big). One cannot be sure if the performance is due to hyperparameter selection or actually due to performance gains.    Improvements: 1) What about debiasing results? At least brief comments on the changes in the theory. 2) Some theoretical insights when beta_1 \neq 0 for Adam and Yogi. 3) Relation to Adagrad, if v_{t-1}
Reviewer 3
This paper proposed convergence analysis for ADAM under certain parameter settings and proved the convergence to a stationary point for nonconvex problems. Moreover, the authors proposed an adaptive SGD (i.e., YOGI) and evaluated it on the different learning models.  Generally, the studied problem is interesting and the proposed algorithm demonstrated good performance on its testing problems. However, some important issues are not clearly stated in the current manuscript: 1. This paper has stated that YOGI controls the increase in effective learning rate, but it is not clear how it impacts the learning rate? 2. The proof of the convergence in ADAM only considered the case that $\beta_1=0$. Thus it is also not very clear how about the convergence proof with the condition $\beta_1\in(0,1)$? 3. There are some types and improper presentations in Appendix. For example, in the first line of page 12, the update rule $v_t,i$ of YOGI does not match with the update rule in Algorithm 2.
Reviewer 4
This paper studies the adaptive gradient methods (e.g, ADAM) for nonconvex stochastic problems. Specifically, the authors prove the convergence to the stationary point using the increasing batch size. They also propose a new algorithm of YOGI and achieve the same convergence result.  I have some concers on the "increaing minibatch size".   1. The authors claim that increasing batch size leads to convergence. It may be ambiguous. From Corollaries 2,3,4,5, we can see that the authors use a large and fixed batch size in the order of O(T), where T is the number of iterations. In my opinions, increasing batch size means that the batch size is increaing during the iterations and it may be more practical than the fixed and large batch size. Of course, it may be more chanllenging to prove the convergence with the real increaing batch size.  Comment after author rebuttal: the authors said that " the same theoretical guarantee can be obtained for increasing minibatch b_t = Θ(t)". I am not convinced of it. Hope the authors to include a detailed proof for the case of  b_t = Θ(t) in the final version. If the analysis only applies to the case of b_t = Θ(T), please point it clearly and do not mislead the readers.  2. More seriously, the theories in this paper require the batch size to be $b=O(T)$. In practice, we may run the algorithm for several epoches, e.g., 100 epoches. Let N be the sample size, then T=O(100N). However, the batchsize we ofteh use is much smaller than N, e.g., b=100, in this case, b=O(T) is an unpractical requirement. I think this issue is not due to the weakness of ADAM, but the weakness of the proof in this paper.  Comment after author rebuttal: I have made a mistake in the previous comment, that is, T should be O(100N/b) and b=O(T) leads to b=O(10\sqrt{N}). This setting is acceptable in practice.   From Table 1 in "Neon2: Finding Local Minima via First-Order Oracles", we know SGD needs O(1/\delta^4) iterations to find x such that $ ||\nabla f(x)|| \leq detla$. The complexity proved in Corollary 3 is O(1/\delta^2) such that $ ||\nabla f(x)|| \leq detla$. I do not think ADAM has a faster theoretical convergence rate than SGD. Thus it may verify that this paper has made some unreasonable assumptions.  Comment after author rebuttal: The authors persuaded me. Since the minibatch is O(T) and SFO complexity measures the total number of stochastic oracle calls. So the final compleixty of ADAM is O(T^2) = O(1/\delta^4). Hope the authors to make it clear in the final version and do not mislead the readers that ADAM is theoretically faster than SGD (they have the same theoretical complexity in the convex case).  Corollary 1 in this paper is also ambiguous. I have read the introduction of [8]. [8] proved that their method needs O(1/epsilon^2) iterations to find x such that $ ||\nabla f(x)||^2 \leq epsilon$. However, from Corollary 2, we know ADAM needs O(1/epsilon) iterations to find x such that $ ||\nabla f(x)||^2 \leq epsilon$. Thus, the comparison in Section: Discussion about Theoretical Results is not correct.  Comment after author rebuttal: The authors persuaded me. Please see the above comment.  In conclusion, the study in this paper may be significative in practice. But the theoretical analysis is not satisfied.  Final comment after author rebuttal: I am pleased with the theoretical analysis now. I do not evalute the experiments due to lack of experience.


Entropy Rate Estimation for Markov Chains with Large State Space
Entropy estimation is one of the prototypical problems in distribution property testing. To consistently estimate the Shannon entropy of a distribution on S elements with independent samples, the optimal sample complexity scales sublinearly with S as Θ( S log S ) as shown by Valiant and Valiant [4]. Extending the theory and algorithms for entropy estimation to dependent data, this paper considers the problem of estimating the entropy rate of a stationary reversible Markov chain with S states from a sample path of n observations. We show that
• Provided the Markov chain mixes not too slowly, i.e., the relaxation time is at most O( S ln 3 S
• Provided the Markov chain mixes not too slowly, i.e., the relaxation time is at most O( S ln 3 S ), consistent estimation is achievable when n S 2 log S .
• Provided the Markov chain has some slight dependency, i.e., the relaxation time is at least 1 + Ω(
), consistent estimation is impossible when n S 2 log S .
Under both assumptions, the optimal estimation accuracy is shown to be Θ(
n log S ). In comparison, the empirical entropy rate requires at least Ω(S 2 ) samples to be consistent, even when the Markov chain is memoryless. In addition to synthetic experiments, we also apply the estimators that achieve the optimal sample complex-ity to estimate the entropy rate of the English language in the Penn Treebank and the Google One Billion Words corpora, which provides a natural benchmark for language modeling and relates it directly to the widely used perplexity measure.

The paper proposes an entropy estimate for Markov chains by reduction to optimal entropy estimation for i.i.d samples. Sample complexity analysis is provided for different mixing scenarios with a minimax rate established for a particular rate. The estimator is used to assess the capacity of language models.    This is a very clear and well-written paper. I appreciate the efforts done by the authors to summarize the results.  My main question, however, is regarding the experiments. Can the authors comment on the confidence of the entropy estimate given the large size of the alphabet? This is crucial to justify its use for the suggested application. Also, could the authors include the empirical entropy estimates?  ============================================================= UPDATE: I thank the authors for the their response. I believe the empirical entropy results setrengthen the paper and I recommend including them in the final version.


Sparse DNNs with Improved Adversarial Robustness
Deep neural networks (DNNs) are computationally/memory-intensive and vulnerable to adversarial attacks, making them prohibitive in some real-world applications. By converting dense models into sparse ones, pruning appears to be a promising solution to reducing the computation/memory cost. This paper studies classification models, especially DNN-based ones, to demonstrate that there exists intrinsic relationships between their sparsity and adversarial robustness. Our analyses reveal, both theoretically and empirically, that nonlinear DNN-based classifiers behave differently under l 2 attacks from some linear ones. We further demonstrate that an appropriately higher model sparsity implies better robustness of nonlinear DNNs, whereas over-sparsified models can be more difficult to resist adversarial examples.

This paper assumes an intrinsic relationship between the sparsity and the robustness of DNN models. The authors prove that this assumption holds for linear models under the $l_infinity$ attacker model, and it also holds for nonlinear models under both $l_2$ and $l_infinity$ attacker models. Then, the authors demonstrate some experimental results which are consistent with the theory.  In general, this paper is well-written, and it is a plausible investigation on the issues of DNN robustness. The problem studied in this paper is very important and also challenging.  The methods of the paper are inspiring. However, there are also critical issues in this paper, which jeopardize the importance of the achieved results. So I tend to give a weak reject to this paper. More details are shown below.  Critical Issues: 1) The authors adopt two metrics to reflect the robustness of DNNs. However, they have not shown how general are such measurements. 2) When proving Theorem 3.1 and Theorem 3.2, the authors assume that the labels of the dataset follow uniform distributions. But they have not discussed whether the theorems hold for the scenarios of unbalanced datasets. In practice, it is very often that some labels occur more frequently than others. If the theorems cannot hold in such cases, the importance of the results would be affected. 3) The proof of nonlinear DNNs does not consider pooling functions, which are popular in present neural networks. Besides, the authors obtain two bounds in the proof, but these bounds are not reflected in the experimental results. Moreover, the robustness of Figure 2 drops dramatically after a threshold, which confuses the reviewer about the correctness of the proof.  More Issues: 1) Page 2 line 51: the related papers published in the ICLR workshop this year draw conclusions opposite to this paper. However, the authors did not address the difference between the related papers and this paper. Since these results appear conflicting to each other, this reviewer is confused about the correctness of this paper if there is no appropriate explanation.  2) Page 3 line 93: the authors claim that the $r_2$ metric makes more sense if the classifiers are less accurate. However, the question is whether it is worth improving the robustness if the classifier is inaccurate itself? 3) Page 3 line 114: the authors conjecture that the robustness of linear models is unrelated to the sparsity with respect to $r_2$. This conclusion further confuses this reviewer about the generality of the two metrics.  Minor Issues: Page 2 line 52: “shows” => “show”  Page 6 line 245: The bar above 0.3 seems a typo. Page 7 line 263: The description of the DNNs is not clear. Page 7 line 264: “2 convolutional layer” => “2 convolutional layers”


Algorithmic Linearly Constrained Gaussian Processes
We algorithmically construct multi-output Gaussian process priors which satisfy linear differential equations. Our approach attempts to parametrize all solutions of the equations using Gröbner bases. If successful, a push forward Gaussian process along the paramerization is the desired prior. We consider several examples from physics, geomathematics and control, among them the full inhomogeneous system of Maxwell's equations. By bringing together stochastic learning and computer algebra in a novel way, we combine noisy observations with precise algebraic computations.

The paper addresses the problem of constructing GP priors satisfying constraints that can be expressed as algebraic problems. They demonstrate how this is facilitated by the application of Groebner bases. They discuss some of the mathematical prorperties and show example applications.  I will first address the biggest weakness of the paper: its clarity. From the presentation, it seems evident that the author is an expert in the field of computer algebra/algebraic geometry. It is my assumption that most members of the NIPS community will not have a strong background on this subject, me including. As a consequence, I found it very hard to follow Sect. 3. My impression was that the closer the manuscript comes to the core of algebraic geometry results, the less background was provided. In particular, I would have loved to see at least a proof idea or some more details/background on Thm. 3.1 and Cor. 3.2. Or maybe, the author could include one less example in the main text but show the entire derivation how to get from one concrete instance of A to right kernel B by manual computation? Also, for me the description in Sect. 2.4 was insufficient. As a constructive instruction, maybe drop one of the examples (R(del_t) / R[sigma_x]), but give some more background on the other?  This problem of insufficient clarity cannot be explained by different backgrounds alone. In Sect. 3.2, the sentence "They are implemented in various computer algebra systems, 174 e.g., Singular [8] and Macaulay2 [16] are two well-known open source systems." appears twice (and also needs grammar checking). If the author could find a minimal non-trivial example (to me, this would be an example not including the previously considered linear differential operator examples) for which the author can show the entire computation in Sect. 3.2 or maybe show pseudo-code for some algorithms involving the Groebner basis, this would probably go a long way in the community.  That being said, the paper's strengths are (to the best of this reviewer's knowledge) its originality and potential significance. The insight that Groebner bases can be used as a rich language to encode algebraic constraints and highlighting the connection to this vast background theory opens an entirely new approach in modelling capacities for Gaussian processes. I can easily imagine this work being the foundation for many physical/empirical-hybrid models in many engineering applications. I fully agree and applaud the rationale in lines 43-54!  Crucially, the significance of this work will depend on whether this view will be adopted fast enough by the rest of the community which in turn depends on the clarity of the presentation.  In conclusion: if I understood the paper correctly, I think the theory presented therein is highly original and significant, but in my opinion, the clarity should be improved significantly before acceptance, if this work should reach its full potential. However, if other reviewers have a different opinion on the level of necessary background material, I would even consider this work for oral presentation.  Minor suggestions for improvements: - In line 75, the author writes that the "mean function is used as regression model" and this is how the author uses GPs throughout. However, in practice the (posterior) covariance is also considered as "measure of uncertainty". It would be insightful, if the author could find a way to visualize this for one or two of the examples the author considers, e.g., by drawing from the posterior process. - I am not familiar with the literature: all the considerations in this paper should also be applicable to kernel (ridge) regression, no? Maybe this could also be presented in the 'language of kernel interpolation/smoothing' as well? - I am uncertain about the author's reasoning on line 103. Does the author want to express that the mean is a sample from the GP? But the mean is not a sample from the GP with probability 1. Generally, there seems to be some inconsistency with the (algebraic) GP object and samples from said object. - The comment on line 158 "This did not lead to practical problems, yet." is very ominous. Would we even expect any problem? If not, I would argue you can drop it entirely. - I am not sure whether I understood Fig. 2 correctly. Am I correct that u(t) is either given by data or as one draw from the GP and then, x(t) is the corresponding resulting state function for this specified u? I'm assuming that Fig. 3 is done the other way around, right? --- Post-rebuttal update: Thank you for your rebuttal. I think that adding computer-algebra code sounds like a good idea.  Maybe presenting the work more in the context of kernel ridge regression would eliminate the discussion about interpreting the uncertainty. Alternatively, if the author opts to present it as GP, maybe a video could be used to represent the uncertainty by sampling a random walk through the distribution.  Finally, it might help to not use differential equations as expository material. I assume the author's rationale for using this was that reader might already a bit familiar with it and thus help its understanding. I agree, but for me it made it harder to understand the generality with respect to Groebner bases. My first intuition was that "this has been done". Maybe make they Weyl algebra and Figure 4 the basic piece? But I expect this suggestion to have high variance.


A Statistical Recurrent Model on the Manifold of Symmetric Positive Definite Matrices *
In a number of disciplines, the data (e.g., graphs, manifolds) to be analyzed are non-Euclidean in nature. Geometric deep learning corresponds to techniques that generalize deep neural network models to such non-Euclidean spaces. Several recent papers have shown how convolutional neural networks (CNNs) can be extended to learn with graph-based data. In this work, we study the setting where the data (or measurements) are ordered, longitudinal or temporal in nature and live on a Riemannian manifold -this setting is common in a variety of problems in statistical machine learning, vision and medical imaging. We show how recurrent statistical network models can be defined in such spaces. Then, we present an efficient algorithm and conduct a rigorous analysis of its statistical properties. We perform numerical experiments demonstrating competitive performance with state of the art methods but with significantly fewer parameters. We also show applications to a statistical analysis task in brain imaging, a regime where deep neural network models have only been utilized in limited ways.

OVERVIEW: At a high level, the paper extends a RNN, the Statistical Recurrent Unit (SRU) to the manifold of Symmetric Positive Definite (SPD) matrices to propose a new RNN called SPD-SRU. The idea is motivated by current trends of Geometric Deep Learning for graphs and manifolds and is of interest to the community. The main contributions of the authors lie in the technical implementation of this idea. They extend the update equations for the SRU (Eqns. 2-5), defined for a Euclidean metric space, to the SPD manifold (Eqns. 6-9). This involves (i) posing the linear or fully connected layer WX, as multiple weighted Frechet Mean (wFM) problems (ii) the bias term of the linear layer is captured a group translation operation and (iii) a ReLU non-linear activation is used in the parameter space. The trick that makes this work efficiently is a new proposed algorithm to compute the wFM efficiently using recursive closed form updates. The novelty of the paper is in the derivation of this computation with proofs of consistency and convergence. They evaluate their proposed SRU-RNN on three experiments: (1) Moving MNIST, (2) UCF11 and (3) permutation testing to detect group differences between patients with and without Parkinson's disease.   STRENGTHS: 1. The idea is well motivated and the background material, both Riemannian geometry on the SPD manifold and the SRU is covered nicely.  2. The authors have 65 citations in their submission which is indicative of the depth in which they review the literature.  3. The math is clear at a high level in motivating what they want to do, the technical challenges it involves and their solutions to these challenges.  4. They provide proofs for different mathematical properties of their proposed solution to the wFM problem for SPD matrices. 5. Their experimental evaluation shows that they achieve comparable (or better in some cases) performance for a fraction of the number of parameters to other RNN units like LSTM, SRU, TT-GRU, TT-LSTM.  WEAKNESSES: 1. The single biggest critique I have with the submission is with regards to how it is written. In two locations: (1) the proofs provided in the supplementary material and (2) the experimental evaluation, the authors can do a better job at making the paper more accessible. They skip steps or leave parts to the reader which lead to incomplete or confusing proofs. In the experimental evaluation, they provide some details but these details are probably sufficient for the authors to redo their experiments but not for a new researcher trying out their models. If the aim is to make the SPD-SRU as common and popular as the RNN/LSTM, the authors need to make these details more accessible. I agree that page restrictions play a role in how the paper is presented but all these details can and should be provided as part of the supplementary material. I specify the exact problems I refer to above: (i) In line 93, "One can easily see that, with the Stein metric, G is the set of nxn orthogonal matrices, denoted by O(n)". Should it be "a choice of G is the set"? "G is the set" seems to indicate an if and only if statement. The if direction is clear but the only if is not. (ii) All the discussions of network architectures in Sections 5.1 and 5.2 especially with regard to the number of parameters. A table outlining the final chosen network architectures (in the supplementary if needed) with specific choices of layer dimensions is required. This is important to firstly understand how you get the #params column in your tables and secondly to see if the comparison is fair. You want the reader to trust you but you also want him/her to be able to verify these statements made as facts. (iii) Proposition 1: For the \beta function, you define co-domain as S^\infty and range as \mathcal{H} = \beta (\mathcal{N}). The way I see it you have shown that for any \tilde{f} in \mathcal{H}, there exists an f and for any f there exists a \tilde{f}. What I'm missing is the connection between co-domain S^\infty and range \mathcal{H}. I'm also missing a one-to-one mapping from \mathcal{N} to \mathcal{H} to show that many f do not map to a single \tilde{f}. I think a lot of this goes away if you explicitly define what \beta is ? or what the function norm \| f \| is ? (iv) Proposition 2 is correct but Line 11, "we have used the fact that  = ..." is left as homework to the reader. It checks out with Linear algebra properties (A^-1 + B^-1)^-1 = A (A+B)^-1 B and det(AB) = det(A) det(B) used in the middle but adding this proof is a couple of extra lines of work that make the proposition complete. A secondary question is why is this proposition needed ? I do not see it explicitly being called in any future proof.  (v) Proposition 3: Line 15, I-yy^t for functional y \in \mathcal{H} is not clear. Also, how do you use it in the proof ? How do you get   \leq  ? (vi) Proposition 7: This is the most important proof of the paper and it's also the most confusing. You start by defining \alpha and \theta. Then you proceed to define g(\alpha) which I assume has some connection to Eqn.11 (not mentioned explicitly and not clear, especially the \log(\cos^2(\theta-\alpha)) term). Then you basically solve for \alpha as a function of \theta and finally write Line 31. The equation for the closed form update in Line 31 checks out but again, it's not clear how you get there. (vii) Proposition 5: Please add the approximations in the Taylor expansion \cos \theta_k = 1 - \frac{\theta_k^2}{2} + \frac{\theta_k^4}{4} + ... and \log(1-x) = -x -\frac{x^2}{2} + ... and the couple of extra steps between equality 2 and 3. (viii) Proposition 6: Not clear what inequality is used in \sum_{k=n+1}^m \sqrt(-2\log \cos \alpha_k) \leq (m-n-1) \sqrt{-\frac{2}{m-n-1} \sum... }. Not clear what happens to the 1/2c term in the approximation above line 42.  Most of the math presented here checks out and I trust the authors but complete proofs are important to verify them and for future readers. 2. A key detail that is only briefly mentioned is the implementation of the requirement that the weights are convex \geq 0 and \sum = 1. To me the biggest jump from Eqn.2 to Eqn.6 is this requirement and there is no discussion about changing the Linear layer in this manner, why this is not a sub-optimal step and why this still makes sense. 3. Experimental evaluation: The authors report time taken per epoch during training in Tables 1 and 2 for their experiments on Moving MNIST and UCF11, which is unusual. Standard practice is to report time taken at inference. It is good to be able to train a model in reasonable amount of time but for the time scales they report (a few seconds to maybe 100s per epoch) it does not seem to be a bottleneck. I would be more interested in inference times across models. Also, the permutation testing experiment in Sec.5.2 makes sense but I would propose a modification. As a machine learning practitioner, I would suggest a leave-one-out experiment where you remove one subject from the data, train your with-disease and without-disease models and assign the subject to whichever model it is closest to. This might also be a future diagnostic tool based on your work ?  MINOR COMMENTS: 1. Data is singular uncountable ? Lines 1 and 21. 2. Move references to per property in Lines 30-31. 3. Before Sec.2 motivate why SPD is important. I think current literature review is good but it is missing a survey of SPD based models especially deep learning ones. 4. Repetition in Lines 291-295 and Lines 300-305 5. Citation needed for PSRN in Line 354 6. Line 29 of Supplementary, x = \tan(\alpha)  DECISION: I have marked the overall rating as 8.  Quality: 8 Clarity: 7 Originality: 7 Significance: 8  Update post rebuttal: I have read the author response and other reviews. Based on the response and discussions with other reviewers, I have downgraded by review from 8 to 7. This is due to two reasons: (1) The authors have oversold their claim of generalizing over multiple manifolds. This should either be backed up with technical details or the text modified to reflect the current SPD + SRU setting, which might still be sufficient if convincing arguments are made for it. (2) Scalability is not clear. I am not in complete agreement with R1 that the algorithm is restricted to toy examples but I agree with his/her opinion that this has not been dealt with sufficiently by the authors.


Sequence-to-Segments Networks for Segment Detection
Detecting segments of interest from an input sequence is a challenging problem which often requires not only good knowledge of individual target segments, but also contextual understanding of the entire input sequence and the relationships between the target segments. To address this problem, we propose the Sequence-toSegments Network (S 2 N), a novel end-to-end sequential encoder-decoder architecture. S 2 N first encodes the input into a sequence of hidden states that progressively capture both local and holistic information. It then employs a novel decoding architecture, called Segment Detection Unit (SDU), that integrates the decoder state and encoder hidden states to detect segments sequentially. During training, we formulate the assignment of predicted segments to ground truth as the bipartite matching problem and use the Earth Mover's Distance to calculate the localization errors. Experiments on temporal action proposal and video summarization show that S 2 N achieves state-of-the-art performance on both tasks.

The authors describe an architecture to output a series of temporal regions (segments) of interest in a video. There is a simple RNN encoder and an interesting decoder called a "Segment Detection Unit" which points to multiple segment boundaries.  SDU is applied recurrently to the final state of the encoder, each iteration outputs a segment and a confidence score. (see End-to-End people detection, which is referenced [36]) SDU is similar to Pointer Networks in that it points back to an element of the input sequence as its output rather than outputing a regression value or an index to a dictionary. Also they 'improve' on pointer networks by using the encoded state of each input timestep (along with the current network state) (instead of the input at each timestep ) as input to the function that determines the pointer (function g). The encoder is thus able to learn to encode states with features  to help learn g.  So the model encodes the whole sequence, and the final state is input to a recurrent SDU unit, which has outputs for beginning and end. The SDU unit outputs for each output timestep are calculated for each encoded state of the input sequence, via a function g that uses the encoded states of the input sequence and the current decoder state as input) in this way the authors directly score each of the input frames as beginning and end segment positions and take the max of those scores to output a segment. They also learn to output a confidence score for each segment.  They define a loss function that considers localization loss and confidence of predictions. For localization loss, they use the EarthMover distance. Their loss function depends on an assignment of all output segments to all ground truth segments and use the "Hungarian algorithm" to do just that.  They perform reasonable experiments, compare with other methods, and show good performance.  Concern: There is no terminal output state from the deocder RNN.  In the Implementation section (201) it says: "We set the number of proposals generated from each chunk to be 15 which is the largest possible number of ground truth proposals contained in a chunk during training". Does this mean that 15 outputs are always made, but that the confidence score is used as a cutoff for accepting segments? It is not clear to me that this is explained. I am not saying this is bad, it's just not clear.. if my interpretation is true then what is the threshold used? How is it determined?  This is a clear-headed work, combining ideas from recent advances. The choices made appear creative and solid, and the results are good.  Minor points: This is not true (line 75): "However, the current RNNs are still restricted to output each time a “token” of the input sequence", but the following sentence is fine. Grammar 83: 'a 1D "object" or "objects"


Context-Aware Synthesis and Placement of Object Instances
Learning to insert an object instance into an image in a semantically coherent manner is a challenging and interesting problem. Solving it requires (a) determining a location to place an object in the scene and (b) determining its appearance at the location. Such an object insertion model can potentially facilitate numerous image editing and scene parsing applications. In this paper, we propose an end-to-end trainable neural network for the task of inserting an object instance mask of a specified class into the semantic label map of an image. Our network consists of two generative modules where one determines where the inserted object mask should be (i.e., location and scale) and the other determines what the object mask shape (and pose) should look like. The two modules are connected together via a spatial transformation network and jointly trained. We devise a learning procedure that leverage both supervised and unsupervised data and show our model can insert an object at diverse locations with various appearances. We conduct extensive experimental validations with comparisons to strong baselines to verify the effectiveness of the proposed network. Code is available at https: //github.com/NVlabs/Instance_Insertion.

In overall, I think this paper proposes a well-designed pipeline for semantic map synthesis which is helpful to image generation tasks. The rebuttal addresses my questions about the ablation study. I strongly suggest the authors to add the ablation study in the final version if this paper is accepted. My final rating is accept. ========================================= This paper proposes a where and what pipeline to synthesize and place object instances into a semantic map in an end-to-end way. This method can help to generate diverse natural semantic map which contains important structure information. The generated semantic maps can benefit the image synthesis task.  Pros: * The proposed idea is interesting that using the context information to predict the position and shape of object instances in order to generate novel semantic maps. * The writing is easy to follow, and empirical experiments show that it is able to synthesize natural semantic maps. * The decomposed where and what modules make scenes and can provide some control during the synthesis. The well-designed jointly training strategy is suitable for the semantic map synthesis task. The proposed framework can also be useful for other structure generation tasks.  Cons: * In total, there are four discriminators. I wonder how do they affect the training and results. Is there any ablation study for this?  Given that this paper is about the context-aware location and shape prediction, it is suggested that some recent work on context based structure prediction for image inpainting and generation can be included in the related work, e.g.,  [a] Q. Sun, L. Ma, S.J. Oh, L. Van Gool, B. Schiele, and M. Fritz. “Natural and Effective Obfuscation by Head Inpainting.” In CVPR, 2018. [b] K. Ehsani, R. Mottaghi, and A. Farhadi. “Segan: Segmenting and generating the invisible.” In CVPR 2018.


How to tell when a clustering is (approximately) correct using convex relaxations
We introduce the Sublevel Set (SS) method, a generic method to obtain sufficient guarantees of near-optimality and uniqueness (up to small perturbations) for a clustering. This method can be instantiated for a variety of clustering loss functions for which convex relaxations exist. Obtaining the guarantees in practice amounts to solving a convex optimization. We demonstrate the applicability of this method by obtaining distribution free guarantees for K-means clustering on realistic data sets.

This paper presents a general method to derive bounds on how close a given clustering is to an optimal clustering where optimal is defined according to certain loss functions. The method can be used for any clustering loss function for which convex relaxation exists (e.g., K-means, spectral clustering). They show in experiments they obtain much better bounds than the only related work (as far as I know) on K-means [Mei06].  The paper is well-written and easy to follow and addresses an important problem of evaluating the quality of clusterings. The main contribution of the paper is to make use of tighter convex relaxations to derive bounds on the distance between optimal clustering and a given clustering. It is obvious that tighter the convex relaxation the better the bounds are and hence they use SDP relaxation for the K-means problem (which can be computationally prohibitive for large datasets). Usually the convex relaxation already gives a bound on how far the optimal loss value is from the loss of a given clustering. Is this already indicative of how far the optimal solution is from the given clustering?  My main concern is how good the bounds are in more realistic settings. Moreover, the method can fail to get a valid bound if some certain technical condition fails (which is easy to check after solving the convex relaxation). As they show in experiments the method fails to obtain valid bounds for difficult settings (high noise, unequal sizes of clusters).  On the positive side, + the method provides much better bounds than the spectral bounds derived in [Mei06] + several synthetic experiments are conducted under various settings to illustrate the bounds obtained by the method.  Comments: - From Theorem 3 it appears that the method works for spectral clustering as well; experiments on spectral clustering would be interesting, e.g., to check the quality of the solutions obtained by recent methods like [HeiSet11, RMH14]. - It was not clear the need for removing outliers in the synthetic experiments. - The method is computationally expensive; they hint at reducing the complexity by removing data points that do not affect the clustering but this needs to be further formalized. - More experiments on real world data would enhance the paper (currently only one real world data is used). - Missing references: line 248, line 253  [HeiSet11] Beyond spectral clustering: Tight continuous relaxations of balanced graph cuts. NIPS 2011.


Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning from piece-wise stationary visual data: Variational Autoencoder with Shared Embeddings (VASE). Based on the Minimum Description Length principle, VASE automatically detects shifts in the data distribution and allocates spare representational capacity to new knowledge, while simultaneously protecting previously learnt representations from catastrophic forgetting. Our approach encourages the learnt representations to be disentangled, which imparts a number of desirable properties: VASE can deal sensibly with ambiguous inputs, it can enhance its own representations through imagination-based exploration, and most importantly, it exhibits semantically meaningful sharing of latents between different datasets. Compared to baselines with entangled representations, our approach is able to reason beyond surface-level statistics and perform semantically meaningful cross-domain inference.

By defining a generative process of images based on latent environments and shared latent generative factors which will be chosen depending on the environment, the authors propose an algorithm for life-long unsupervised representation learning. Reasonable inference procedures for the essential elements (the generative factors, the latent mask, and the environment) are proposed, and the training is done with a loss inspired in the Minimum Description Length principle and a "dreaming" feedback loop similar to the one in Shin et al. (2017).  The algorithm is a mix of several components for which the authors well state the motivations for them to be there. The work is mostly one of modeling, and no theoretical evidence supports the claims. All the evidence is experimental, which is very dense as a consequence of the several components and claims. Even though this work is of high quality, the quality comes from the engineering part. The authors proposed to tackle several scientific questions at once by mixing several components. In the end, the algorithm works and improves the state of the art, but there is not a clear principle that has been studied. However, this algorithm might be a good starting point to focus on one question such as to which extent the algorithm exhibits "semantically" meaningful sharing of latents between different datasets? Of course, one part of the experimental section is devoted to showing evidence for this question, but since there are several other claims, the evidence is superficial, there is no an attempt to falsify the claims to see if they are robust or at least to show the weaknesses of the algorithm. Doing this for each one of the claims would probably result in a paper with many more pages; this is why it would be desirable to focus on one specific question and answer it thoroughly. Studying better one aspect would increase the scientific quality of the paper, and improve the significance of the article.  The paper is written clearly, except for a few phrases which are not well explained I believe. For instance, in line 169, what motivates the use of the Wasserstein distance for the encoder and the KL divergence for the decoder? And the other examples are mostly about interpretation. To show clearly the evidence and the claims supported it might be a better idea to use words whose meaning in the given context is not up to ambiguity. For example, in line 268, "VASE can imagine the existence of moving MNIST before actually experiencing it" this phrase corresponds to an interpretation of the fact that when training a classifier for moving MNIST during the static MNIST training stage, the classifier can achieve an accuracy that is not random. The reasons that explain this fact might not have anything to do with "imagination" in the sense of artificially generated; maybe they could come from a translation invariant property of the classifier. The translation invariant explanation might not be correct, it is just an example to make a point of avoiding interpretations that implicitly make claims not supported by evidence, or at least for which the evidence has not been clearly explained.  To my knowledge, the idea of considering latent variables that might be or might not be shared across different environments and the idea of considering latent environments are new and original. I believe there might be exciting potential in exploring the latent environments idea profoundly. In my view, it would be more significant to focus on this part of the paper, giving clear empirical evidence for the proposed claims without filling the experimental part densely trying to do everything at once.  Update: I still think that it is better to tackle one specific question at a time; however, I find the number of experiments and the different ideas valuable, even if it is not an in-depth study (in my opinion) but an illustration of the potential of the algorithm. Also, as explained by the authors in the rebuttal, I understand that the several experiments are heavily linked (as a consequence of the nature of the algorithm, which is a mix of several components) which implies that it is not straightforward to isolate one specific question. I hope that the rearrangements in the experimental section that the authors mention in their rebuttal will help to illustrate the studied points better. For all these reasons, I have changed my score from 4 to 6.


Sparse PCA from Sparse Linear Regression
Sparse Principal Component Analysis (SPCA) and Sparse Linear Regression (SLR) have a wide range of applications and have attracted a tremendous amount of attention in the last two decades as canonical examples of statistical problems in high dimension. A variety of algorithms have been proposed for both SPCA and SLR, but an explicit connection between the two had not been made. We show how to efficiently transform a black-box solver for SLR into an algorithm for SPCA: assuming the SLR solver satisfies prediction error guarantees achieved by existing efficient algorithms such as those based on the Lasso, the SPCA algorithm derived from it achieves near state of the art guarantees for testing and for support recovery for the single spiked covariance model as obtained by the current best polynomialtime algorithms. Our reduction not only highlights the inherent similarity between the two problems, but also, from a practical standpoint, allows one to obtain a collection of algorithms for SPCA directly from known algorithms for SLR. We provide experimental results on simulated data comparing our proposed framework to other algorithms for SPCA.

The paper proposes an approach to reduce solving a special sparse PCA to a sparse linear regression (SLR) problem (treated as a black-box solution). It uses the spiked covariance model [17] and assumes that the number of nonzero components of the direction (u) is known, plus some technical conditions such as a restricted eigenvalue property. The authors propose algorithms for both hypothesis testing and support recovery, as well as provide theoretical performance guarantees for them. Finally, the paper argues that the approach is robust to rescaling and presents some numerical experiments comparing two variants of the method (based on SLR methods FoBa and LASSO) with two alternatives (diagonal thresholding and covariance thresholding).  Strengths:  - The addressed problem (sparse PCA) is interesting and important. - The paper is well-written, the general idea is relatively simple and easy to understand (the implementation of Algorithms 1 and 2 also looks straightforward). - As the underlying sparse linear regression (SLR) approach is treated as a black-box (as long as it satisfies Condition 2.4), the approach has several variants depending on the SLR used. - Both hypothesis testing and support recovery are addressed and the paper provides theoretical performance guarantees for them. - The authors also present experimental comparisons to demonstrate the advantages of the method over some previous solutions.  Weaknesses: - It is obvious that there is a strong connection between PCA and linear regression (see for example [41]), thus the originality of the general idea is limited. - The spiked covariance model for PCA is quite restrictive (for example, Gaussian white noise) and unlikely to be met in practical applications. In fact Gaussianity is crucial for this approach, which makes the results less significant. - The assumption that the number of nonzero components of u are known is strong (though the authors argue that there are adaptive methods to adjust this). - It is not clear how easy it is to check / verify for a particular problem and SLR method that Condition 2.4 holds. It looks like a strong assumptions which is hard to verify. Ideally, the authors should have demonstrated verifying this condition on an example.  - The fact that the underlying SLR method is treated as a black-box might hide the problem of selecting the appropriate SLR method. Which one should be used for various problems?  Minor comments: - The authors argue that though random design matrices for linear regression can arise, it makes no difference to assume that it is deterministic (by conditioning). This argument is a bit misleading, as it is only true if the design matrix (\mathbb{X}) and the noise vector (w) affecting the observations are independent. This is not the case, for example, if the linear regression problem arises from a time-series problem, such as estimating the parameters of an autoregressive model (in which case the design matrix cannot be assumed to be deterministic). - It is not a good practice to cite works that are only available on arXiv (as they did not go through any review process, they could contain unsupported claims, etc.).  Post-rebuttal comments: Thank you for your replies. It is a nice feature that the presented approach can turn a black-box SLR to a SPCA solver. Nevertheless, it requires strong assumptions, for example, the spiked covariance model, knowledge of sparsity, and RE, which limit its theoretical and practical relevance. I understand that these assumptions could be relaxed a bit, for example, the method could be extended to sub-Gaussian variables and unknown sparsity could be handled by binary search (for hypothesis testing). It would be good to discuss these issues in more detail in the revised paper.


Learning to Exploit Stability for 3D Scene Parsing
Human scene understanding uses a variety of visual and non-visual cues to perform inference on object types, poses, and relations. Physics is a rich and universal cue that we exploit to enhance scene understanding. In this paper, we integrate the physical cue of stability into the learning process by looping in a physics engine into bottom-up recognition models, and apply it to the problem of 3D scene parsing. We first show that applying physics supervision to an existing scene understanding model increases performance, produces more stable predictions, and allows training to an equivalent performance level with fewer annotated training examples. We then present a novel architecture for 3D scene parsing named Prim R-CNN, learning to predict bounding boxes as well as their 3D size, translation, and rotation. With physics supervision, Prim R-CNN outperforms existing scene understanding approaches on this problem. Finally, we show that finetuning with physics supervision on unlabeled real images improves real domain transfer of models training on synthetic data.

The goal of this paper is to output a set of 3D bounding boxes and set of dominant planes for a scene depicted in a single image. The key insight is to incorporate stability constraints in the 3D layout, i.e., the reconstructed 3D boxes should not move too far under simulation (in Bullet) with physical forces (gravity, friction). Parameters for 3D boxes are regressed using a modified R-CNN training loss and dominant planes for the walls and floors are regressed via a RNN. A stability criterion is used to update the output 3D scene (via REINFORCE) where the predicted 3D layout is run through Bullet simulator and 3D displacements are checked. Results are shown on synthetic (SUNCG, SceneNet RGB-D) and real (SUN RGB-D) datasets, out-performing the factored 3D approach of [Tulsiani18].  Positives: I like the direction this paper takes towards incorporating physics for this problem. The paper demonstrates an interesting approach that leverages modern recognition components and shows positive results on a hard problem. I also like the analysis performed on synthetic and transfer to real scenes. The paper is well written (particularly given the amount of conveyed information in tight space) and reasonably cites prior work (although I have a couple more reference suggestions below).   Negatives: My comments are mostly minor.   A. Probably the biggest issue is the submission missed a highly relevant paper that also used stability constraints for inferring 3D scene layout:  Abhinav Gupta, Alexei A. Efros and M. Hebert, Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics. In ECCV 2010.  There are big differences between the current submission and the above paper ([Gupta10] infers layout for outdoor scenes, have a different mechanism for reasoning about stability, and use 2010 pre-deep learning tech). I think it would be good to tone down a bit the first contribution in the introduction in light of this paper.  B. The second contribution mentions that “a end to end [sic x2] scene reconstruction network” is proposed. Since the simulator is not differentiable (L172), is this still considered an end-to-end network?  In general, I think this would be a good poster. I like the overall story and the execution looks good. There are many system components, so I think it would be hard to reproduce without source code.   Minor comments:  + L161 “parameterizated”  + Why is there a big bend around ~5x10^-2 in Fig 3(a) and at 10^-1 in Fig 5(a)?  + The paper mentions that the model is trained with unlabeled synthetic data. However, it looks like ground truth bounding box annotations are used (L211). If this is the case, it would be great to rephrase this to avoid confusion since it’s not really unlabeled.  + It may be good to also include some relevant work on higher-order 3D scene layout modeling, e.g.,   Characterizing Structural Relationships in Scenes Using Graph Kernels. Matthew Fisher, Manolis Savva, Pat Hanrahan. SIGGRAPH 2011.


Gradient Sparsification for Communication-Efficient Distributed Optimization
Modern large-scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures. A key bottleneck is the communication overhead for exchanging information such as stochastic gradients among different workers. In this paper, to reduce the communication cost, we propose a convex optimization formulation to minimize the coding length of stochastic gradients. The key idea is to randomly drop out coordinates of the stochastic gradient vectors and amplify the remaining coordinates appropriately to ensure the sparsified gradient to be unbiased. To solve the optimal sparsification efficiently, a simple and fast algorithm is proposed for an approximate solution, with a theoretical guarantee for sparseness. Experiments on 2 -regularized logistic regression, support vector machines and convolutional neural networks validate our sparsification approaches.

After rebuttal:  In my opinion, this paper falls slightly short of the mark for NIPS, for the reasons outlined in my original review, but I believe it's worth communicating, and will not oppose its acceptance. I slightly upgraded my score since I agree to some extent that the algorithmic idea is not the same as QSGD; the ultimate decision lies with the area chair.   ====  There's been recently a lot of work on the communication-variance trade-offs of distributed SGD. The present submission fits squarely into this space, by looking at the problem of sparsifying SGD gradients stochastically, to fit a certain variance budget. Thus, it aims to improve upon recent work like TernGrad and QSGD, which mainly try to reduce the communication budget by reducing bit-width (not necessarily sparsifying) gradients.   The results can be summarized as follows:  - The authors give an optimal scheme for fitting the sparsity in a gradient stochastically to a fixed variance budget. This is done by modeling the problem of setting choice probabilities for gradient entries in stochastic quantization as an optimization problem, for which the authors show that there is a nice closed-form solution. They then give an efficient way of implementing this. - The scheme is shown to provide both sparsity and variance bound guarantees. - The authors provide a relatively wide array of small benchmarks, for convex objectives (regression), non-convex (toy neural nets), and asynchronous parallel SGD. The experiments show that the scheme works well when compared with random and naive sampling approaches, and similarly (or a bit better) when compared with QSGD.   My evaluation is as follows:  On the positive side, the solution given here is reasonable, and appears correct to my reading. I think this is a nice set of results, which should be communicated in some form.   On the negative side, I have serious doubts that this crosses the high bar set by NIPS.   First, in terms of significance and improvement over previous work, the contribution here is, in my opinion, limited. The fact that stochastic quantization induces sparsity for low target bit width is not new. A basic version of this observation appears to be given in the QSGD paper: if we take their version with two quantization levels (0 and 1) it seems to me we get a version of this sparsification method with sqrt n density and sqrt n variance. The authors' contribution here is to set the question of choosing sampling probabilities efficiently given a specific variance budget as an optimization problem, which they solve efficiently. However, their optimal solution is quite similar to the QSGD scheme for a single non-zero quantization level. Thus, simplistically, it would seem to me like the contribution here is to determine the optimal scaling parameter for the sampling probabilities?  Moreover, there are other related non-stochastic delay-based sparsification schemes, e.g. [1]. To my knowledge, these schemes do not have convergence guarantees, but they seem to ensure extremely high sparsity in practice for neural nets (an order of magnitude larger than what is considered here in the experimental section). Yet, there is minimal comparison with these, even just at the experimental level. Is the authors' claim that such schemes do not converge for convex objectives? This point is left ambiguous.   Second, the experimental section is at the level of toy synthetic experiments, and therefore leaves much to be desired. I believe that validation on convex objectives is nice, and I haven't seen it in other papers in this area. But this is because communication-compression for regression problems is not really a critical performance problem these days, at least not at the scale shown here.  The non-convex validation is done on toy CNNs, which would have no problem scaling on any decent setup. By contrast, the standard in the area is ImageNet or large-scale speech datasets, for which scaling is indeed a problem. Such experiments are not given here.  The asynchronous experiment is more intriguing, but ultimately flawed. Basically, the idea here is that sparsity will help if threads compete on a bunch of memory locations (corresponding to the parameter). To boost contention, the authors *lock* each individual location (to my understanding). But why is this necessary(other than making the experiments look more impressive)? Modern computer architectures have compare&swap or fetch-and-add operations which could be used here, which wouldn't block access to those locations like locking does. These should be much faster in practice, and I believe standard analyses cover such asynchronous variants as well. Have the authors actually tried this?  Another thing which the authors might want to keep in mind here is that, even though they ensure non-trivial sparsity (e.g. 70%) at each node, collectively the summed updates (gradients) may well become dense for reasonable node count. Hence this technique will not be very useful in any of the cases considered unless #dimension / #nodes >> sparsity.    Summing up, while I have enjoyed reading this paper, I believe this is clearly below the threshold for NIPS, both on the theory and on the experimental side.   Some minor comments:  - sparsification technology -> "technique" is probably better here - you keep changing between [] and () for expectation - "The algorithm is much easier to implement, and computationally more efficient on parallel computing architecture." Why? - "after sparsified" -> please rephrase - "baseline of uniform sampling approach" -> please rephrase - Figure 2: overwriting the experimental results with the legend seems like a bad idea to me. - "converges much faster" -> what is "much" here? please be more precise - "employs a locked read" please be clearer with your experimental setup here. Are these reader-writer locks? phtread locks? test&set locks? The whole section should be re-written for clarity.