Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
We present a framework for efficient inference in structured image models that explicitly reason about objects. We achieve this by performing probabilistic inference using a recurrent neural network that attends to scene elements and processes them one at a time. Crucially, the model itself learns to choose the appropriate number of inference steps. We use this scheme to learn to perform inference in partially specified 2D models (variable-sized variational auto-encoders) and fully specified 3D models (probabilistic renderers). We show that such models learn to identify multiple objects -counting, locating and classifying the elements of a scenewithout any supervision, e.g., decomposing 3D images with various numbers of objects in a single forward pass of a neural network at unprecedented speed. We further show that the networks produce accurate inferences when compared to supervised counterparts, and that their structure leads to improved generalization.

This paper presents a framework for "vision as inverse graphics" that combines structured generative models, inference networks, and end-to-end learning. A key element of the framework is the use of a trans-dimensional generative model that allows for the generation of a variable number of objects in the scene. The inference network that enables learning in the model uses a recurrent network to parse the scene one object at a time. This mechanism simultaneously parses the current object in terms of what/where information and predicts whether there is another object in the scene that remains to be parsed. The evaluation considers a number of 2D and 3D instances of the proposed model including a 2D model that directly learns object appearances and a 3D model that integrates a full 3D renderer. The representations learned by the framework are used for a variety of down-stream tasks including counting objects, where the performance is shown to outperform DRAW networks.
Review Update: Tank you for the additional information on computation time as well as clarifying the issue with repetitions.  Technical quality: The technical contribution of the paper is defined at the level of a framework with modular parts and so is quite high-level as a result. The main components are the generative model (Eq 1), the recurrent inference network, and the use of variational learning. To the extent that technical details are provided for these components, they are correct.   The bulk of the paper focuses on the construction of experiments and the analysis of the results. In general, the data sets and tasks are well designed in both the 2D and 3D cases. The results in terms of object counts are promising. One aspect that could have been better evaluated in the 2D experiments is the accuracy of the pose (where) parameters inferred. This information is only presented qualitatively in terms of the reconstructions (although these do look good in the examples shown). In the case of 3D scenes, the results are also promising. However, the reconstruction accuracy metric appears not to be defined. The discussion of the problem caused by repetitions in the data sets is also not clear and should be explained further.   A final important point regarding the experiments is that the paper claims to enable scene understanding at unprecedented speeds, but there are no timing results of any kind in the paper. The video in the supplemental material appears to be showing real-time scene parsing for multiple digits, but the speed results need to be directly substantiated in the main paper.  Novelty/originality: As mentioned already, the paper describes a framework that combines several existing components: generative scene models, recurrent networks for inference, and variational learning. The type of generative model described has been studied in computer vision extensively in the vision as inverse graphics literature. This paper introduces a specific form of such a model that allows for inference of the number of objects and combines it with a recurrent inference network that can also reason about variable numbers of objects, which does appear to be a novel combination. The main novel claim made in the paper is enabling fast inference in this type of model, which has been the main issue in past work. However, as mentioned above, direct results substantiating the speed of the proposed framework are not included in the paper.  Impact: As a framework, this work is also very extensible in different directions. The use of graphical models combined with RNN inference networks to enable parsing scenes with multiple objects is a nice idea and is likely to be adopted by others if the speed claims are suitably substantiated.   Clarity and presentation: The paper is very well written. There are no issues with grammar or language. The paper is clear, although the descriptions are quite high-level for the most part. The figures are well done although most of the line graphs could benefit from being larger.


Composing graphical models with neural networks for structured representations and fast inference
We propose a general modeling and inference framework that combines the complementary strengths of probabilistic graphical models and deep learning methods. Our model family composes latent graphical models with neural network observation likelihoods. For inference, we use recognition networks to produce local evidence potentials, then combine them with the model distribution using efficient message-passing algorithms. All components are trained simultaneously with a single stochastic variational inference objective. We illustrate this framework by automatically segmenting and categorizing mouse behavior from raw depth video, and demonstrate several other example models.

This paper presents a general modeling and inference framework that combines the strengths of probabilistic graphical models and the flexibility of neural network models. Graphical models are used to represent structured probability distributions and variational autoencoder ideas are used to learn the nonlinear data manifold in an approach that is referred to as the structured variational autoencoder (SVAE).  A major challenge of hybrid models is that inference is often difficult. The main idea of this paper is to learn recognition networks that output conjugate graphical model potentials, allowing for the use of tractable graphical model inference algorithms. They outline an algorithm for doing inference in the SVAE model.  The SVAE model is then applied to both a synthetic dataset and a real dataset. In the synthetic data experiment, a latent linear dynamical system (LDS) SVAE is trained to predict the trajectory of a dot that is bouncing back and forth in a sequence of images. The model seems to do a very good job on this relatively simple task.  Next, they apply a LDS SVAE to model depth video recordings of mouse behavior, where the model seems to predict future frames relatively well. Finally, they apply a latent switching linear dynamical system (SLDS) SVAE model to try to learn discrete states corresponding to natural behavioral units.
Combining PGMs and deep learning/neural networks is a very active and promising area of research. This is an interesting paper, with it’s main contribution to the existing literature being that it presents a model that can account for discrete latent variables. This new capability suggests that it could be used in a variety of interesting applications, including the type of behavior representation modeling shown in one of the experiments.  Overall, the organization of the paper is excellent, and the writing is clear. The technical part is however quite dense. Algorithm 1 in Section 4 is hard to follow. Some of the symbols in the algorithm don’t seem to be defined anywhere (and if they are, they’re too hard to find).   One of the strengths of the paper is that the approach is quite general. The very general description of the method might make it hard to figure out how to apply the algorithm to a specific new model. It might be helpful to include in the appendix a detailed description of the algorithm applied to the examples in Figure 3.  My main criticism is that there is no comparison to existing baselines, except for the "toy" example in Figure 1. I do believe that there are advantages to "structured" representations (at the very least, in terms of interpretability), but it would have been nice to see the results achieved by some baseline (existing) method. E.g., in Figure 5, it’s hard for someone who is unfamiliar with the data to determine if the model is making good predictions without having something to compare to.   Section 6.3 and Figure 6 should be explained in more detail. It seems that one of the more interesting contributions of this approach is that the latent switching linear dynamical system (SLDS) model can identify discrete latent behavioral states that influence the observed dynamics. Without more detail, it’s very hard to tell what is going on. For example, Section 6.3 states that there are 30 discrete states, but Figure 6 only shows two of these - are the other states semantically meaningful as well, or were these two picked because they were the only useful ones?


The Generalized Reparameterization Gradient
The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterization gradient fit Gaussian distributions. In this paper, we introduce the generalized reparameterization gradient, a method that extends the reparameterization gradient to a wider class of variational distributions. Generalized reparameterizations use invertible transformations of the latent variables which lead to transformed distributions that weakly depend on the variational parameters. This results in new Monte Carlo gradients that combine reparameterization gradients and score function gradients. We demonstrate our approach on variational inference for two complex probabilistic models. The generalized reparameterization is effective: even a single sample from the variational distribution is enough to obtain a low-variance gradient.

This works studies stochastic variational inference for approximate posterior inference. The authors propose a unified framework for black-box variational inference based on the log-derivative trick (or score function method) and the reparametrization trick. More specifically, they generalise the latter to non-Gaussian posteriors and discuss three other popular cases (Gamma, Beta and log-normal). The report good experimental results.
This paper was a pleasure to read. It was very well-written and provided an excellent and comprehensive synthesis of recent advances that make stochastic variational inference more generic.  The proposed formulation unifies black-box variational inference based on the score function and the reparametrization trick (used in the variational auto-encoder). The authors introduce the so-called generalized reparameterization gradient, a method that extends the reparameterization gradient to a wider class of variational distributions by allowing for weak dependency. The main idea is to define an invertible transformation of the latent variables such that the distribution of the transformed variables is only weakly governed by the variational parameters. This essentially means that at least the first moments of the auxiliary variable do not depend on the original variational parameters. While being presented as general, it is still required in practice to identify an appropriate transformation. Also, the considered cases are in the same class as the one covered by ADVI, but results are encouraging. The general reformulation is also a step forward in the direction of handling non-conjugate cases.  Could the authors discuss the choice of the denominator in the case of the beta into more detail? From the appendix, it was not clear to me why these choices are working well.  While the authors indicate that it is sufficient to consider only one sample, could they comment on how the results improve as a function of the number of samples?


A Locally Adaptive Normal Distribution
The multivariate normal density is a monotonic function of the distance to the mean, and its ellipsoidal shape is due to the underlying Euclidean metric. We suggest to replace this metric with a locally adaptive, smoothly changing (Riemannian) metric that favors regions of high local density. The resulting locally adaptive normal distribution (LAND) is a generalization of the normal distribution to the "manifold" setting, where data is assumed to lie near a potentially low-dimensional manifold embedded in R D . The LAND is parametric, depending only on a mean and a covariance, and is the maximum entropy distribution under the given metric. The underlying metric is, however, non-parametric. We develop a maximum likelihood algorithm to infer the distribution parameters that relies on a combination of gradient descent and Monte Carlo integration. We further extend the LAND to mixture models, and provide the corresponding EM algorithm. We demonstrate the efficiency of the LAND to fit non-trivial probability distributions over both synthetic data, and EEG measurements of human sleep.

In this paper the authors present a locally adaptive normal distribution. Instead of measuring distances via a Euclidean distance they define a Riemannian metric M(x) which varies smoothly across the data set. Given the R. Metric M(x) geodesic distances along the manifold can be calculated via the exponential and logarithmic map. The authors use the Euler-Lagrange equations which define a set of ordinary differential equations for estimating the geodesic distances. The logarithm map takes points from the manifold (y) into the tangent plane (at x) such that the length of the resulting vector is the geodesic distance from x to y along the manifold.   The authors then apply a normal distribution computing a Mahalanobis distance on points mapped by the logarithm map. This inner product is used to compute distances from a point x to the mean instead of Euclidean distances. The authors then derive estimators for the mean and covariance via maximum likelihood. Since the solutions are not available in closed form they present a steepest descent algorithm for estimating them. Finally, since the normalization constant is also not available in closed form the authors present a Monte-Carlo approximation method.  They apply this method to synthetic data generated on a manifold as well as in a clustering task. They compare to the standard mixture of Gaussians.
Overall I feel that the paper provides an interesting and useful extension of the normal distribution which adapts to local manifold structure. The authors combine work in metric estimation, computing geodesic distances, and learning distributions. I believe this work is worthy of publication. I have a few comments on my scores for the above:  Technical quality Comments:  The manifold techniques (using Euler-Lagrange equations, estimating logarithm maps) are sound and well explained. The Monte-Carlo estimation of the normalization constant and steepest descent algorithms are clever and well described. Analysis of the applications is lacking (sigma parameter appears to be picked by optimizing on training data?). Explanation of the usefulness of the learned distribution is also somewhat lacking.  Novelty Comments:  It appears the main contribution of this paper is combining previous work (using Kernels for a metric M(x), estimating geodesic distances via Euler-Lagrange, applying a normal distribution) and providing an algorithm for estimating the parameters (mean, covariance, normalization constant).      Potential Impact Comments:  The authors provide a fair evaluation of the GMM and LAND in the synthetic data (looking at log-likelihood on data generated by the learned distributions) but it's not clear how they chose the sigma parameter. The authors show that the LAND fits well on data generated along a manifold but don't show how well it compares to a GMM when data is generated according to a single normal distribution or GMM.   In the real life (clustering) data they appear to choose the sigma value for the LAND via optimizing the F measure and then present that same F measure. What they should have done (or if they did made this clearer) is provide the F measure on a held-out test set to be fair. It is, therefore, hard to see if there is or is not a problem with over-fitting. It also would have been nice to examine very high dimensional data.  One potentially useful application that the authors present is that the LAND could be used as a generative model for manifold data, however, they make the rather unusual manifold assumption that the data X is observed in D dimensions *and* the manifold dimension is also D (that is, points on the tangent plane are D-dimensional and the Metric M(x) is full rank). In manifold learning we're usually interested in data that lie on a d-dimensional manifold embedded into a D-dimensional Euclidean space where D is much larger than d.          Clarity Comments:  There's one particular notation point I found confusing: the overloading of the Sigma notation. In particular they say M = Sigma^-1 but then estimate it via a (local) diagonal Kernel matrix.  Is this Simga also the same as the (fitted) covariance matrix? If so, is the kernel just an initialization? This would be strange since M(x) should vary with x but the fitted covariance matrix is a single parameter. In the proof of the steepest decent they explicitly say say that "in our case M = Sigma^-1" and use that substitution. Clarification here would be nice.  In Sec 4.2 they don't state but I assume cluster membership was chosen by picking 2 clusters and computing closest (geodesic?) distance to the cluster means.    Overall I felt the manifold introduction was well explained (at least for someone familiar with Riemannian manifolds).


Structure-Blind Signal Recovery
We consider the problem of recovering a signal observed in Gaussian noise. If the set of signals is convex and compact, and can be specified beforehand, one can use classical linear estimators that achieve a risk within a constant factor of the minimax risk. However, when the set is unspecified, designing an estimator that is blind to the hidden structure of the signal remains a challenging problem. We propose a new family of estimators to recover signals observed in Gaussian noise. Instead of specifying the set where the signal lives, we assume the existence of a well-performing linear estimator. Proposed estimators enjoy exact oracle inequalities and can be efficiently computed through convex optimization. We present several numerical illustrations that show the potential of the approach.

The paper studies theoretically linear filters for signal denoising. An interesting formulation of the problem is obtained under the assumption that the signal has a regular component in an unknown shift-invariant linear subspace, a bounded free component and an i.i.d. Gaussian noise component. It is proposed that the recovery formulation that constrains or penalizes l1 norm in the Fourier space, under some technical conditions, possess guaranteed performance bounds. The goal here is to obtain an estimator achieving a risk within a constant additive bound (called the price of adaptation) of that of the optimal linear estimator. The paper improves upon previous works [10,5] providing tighter additive bounds. Penalized recovery version does not need to know the filter norm parameter in advance.
I’ve indicated below that I am not confident in this topic, please take my questions with ease. Clarity A general question about blind filtering: Is it indeed the case that the knowledge about the set of signals X cannot be obtained / learned in some applications or it is more that it is difficult to describe? In the case of shift-invariant subspaces the description does not seem to be a problem, it is given by equations (6). Small-error linear filters. It was unclear why the signal is restricted to the support interval [0 n] but its recovery is requested on the interval [-m 2m]. Does it mean in particular that there are m samples of pure noise available? How does this support assumption combine with shift-invariance equations (6)? The simplicity assumption is designed such that the total recovery error (5) does not depend on the length of the sequence, and the filter size must grow proportionally to the sequence support. It does not seem realistic.  While the main theoretical contributions were presented for the “sequential” filters – only signals to the left on the axis are participating, the experimental results are presented in the different setting – full 2D linear filters. Is this difference not essential? Do the bounds continue to hold? There are many image denoising techniques in the literature. If we compare to TV-denoising, the prior is imposed on the reconstructed signal, whereas in the proposed method it is imposed on the filter. Can the methods be theoretically / experimentally compared? Since the reconstruction requires to solve an optimization problem, is the special structure of requesting a linear filter is still well-justified / beneficial in practice? While the discussion section dives deeper in a specific direction, a concluding section discussing the theoretical results and experiments overall is missing. Technical Line 56: a new notation seems to be introduced, in which case “may be written as” is not appropriate. Line 75: \phi^0 Line 93: \Delta^\tau is not defined – compositional power?  ***************************  I would recommend to the authors to improve on accessibility of their work. While presented results are really theoretically interesting, understanding them is not easy. In particular the assumptions made for the theory to hold are by far non-trivial. Try to show exemplar cases where these assumptions do hold and if possible explain more their meaning. Do these assumptions translate to something that can be calculated post-optimization so that a provable bound for a quality of reconstruction of a given instance can be computed? It would be also beneficial if the handling of different kinds of filters w.r.t. time axis was more uniform across the paper. Despite the authors claim the results are easily extendible, it is not straightforward to see how the assumptions and the bounds should extend to 2D full filters. The question regarding optimization-based recovery of linear filter versus optimization-based recovery of the signal directly was not fully answered in the rebuttal. In particular problem (8) can be rewritten with a substitute variable \hat x = \varphi * y, which will result in a special regularized on \hat x. While the method does not need to know the structure (the shift-invariant linear subspace) of the signal, in many applications such structure, e.g. appropriate basis can be learned from the training data. The advantage of doing blind recovery in this setting is not of that high practical interest in my opinion.


Dynamic Mode Decomposition with Reproducing Kernels for Koopman Spectral Analysis
A spectral analysis of the Koopman operator, which is an infinite dimensional linear operator on an observable, gives a (modal) description of the global behavior of a nonlinear dynamical system without any explicit prior knowledge of its governing equations. In this paper, we consider a spectral analysis of the Koopman operator in a reproducing kernel Hilbert space (RKHS). We propose a modal decomposition algorithm to perform the analysis using finite-length data sequences generated from a nonlinear system. The algorithm is in essence reduced to the calculation of a set of orthogonal bases for the Krylov matrix in RKHS and the eigendecomposition of the projection of the Koopman operator onto the subspace spanned by the bases. The algorithm returns a decomposition of the dynamics into a finite number of modes, and thus it can be thought of as a feature extraction procedure for a nonlinear dynamical system. Therefore, we further consider applications in machine learning using extracted features with the presented analysis. We illustrate the method on the applications using synthetic and real-world data.

The manuscript, entitled “Koopman Spectral Analysis of Nonlinear Dynamical Sys- tems with Reproducing Kernels” extended the dynamical mode decomposition (DMD) to reproducing kernel Hilbert space (RKHS). The resulting algorithm (KDMD) is then applied to a few toy problems. It hence allows to use features map as definition of the kernel, in order to link more the DMD to specific features of interest; it is then a powerful generalization of DMD. My main concern is that the paper is really hard to read. Even for a few simple notions, the authors emphasize on the mathematical writings when short sentences would have been more than enough. Some sections, on the other hand, are actually verbose. It might make it difficult to understand even for specialists, and a few points should be clarified. It is too cryptic for publication in the Neural Information Processing Systems conference proceedings, but I strongly encourage the authors to rewrite it in a more pedagogic way; the resulting article should actually be highly cited.
Referee’s report on: Koopman Spectral Analysis of Nonlinear Dynamical Systems with Reproducing Kernels   The manuscript, entitled “Koopman Spectral Analysis of Nonlinear Dynamical Sys- tems with Reproducing Kernels” extended the dynamical mode decomposition (DMD) to reproducing kernel Hilbert space (RKHS).  The resulting algorithm (KDMD) is then applied to a few toy problems. It hence allows to use features map as definition of the kernel, in order to link more the DMD to specific features of interest; it is then a powerful generalization of DMD. My main concern is that the paper is really hard to read. Even for a few simple notions, they emphasize on the mathematical writings when short sentences would have been more than enough. Some sections, on the other hand, are actually verbose. It might make it difficult to understand even for specialists, and a few points should be clarified. It is too cryptic for publication in the Neural Information Processing Systems conference proceedings, but I strongly encourage the authors to rewrite it in a more pedagogic way; the resulting article should actually be highly cited.  I. MAJOR REMARKS Please find my major remarks in the following: 1. The presentation is really verbose when it could be simplified. For instance: “Con- ceptually, DMD can be considered as producing an approximation of the Koopman eigenfunctions using a set of linear monomials of observables as basis functions, which is analogous to a one-term Taylor expansion at each point.” can be easily summarized in one short and simple sentence. 2. Sec. 4 is actually useless in a short paper and was (partially) treated in the introduc- tion. 3. Sec. 5 should have been illustrated, or be summarized in the conclusion. 4. Sec. 6 should have been developed in the way of Sec. 5! It has been condensed too much. II. MINOR REMARKS My other minor remarks are: 1. there is probably a typo in the description of the toy NL case, as it is actually linear. 2. first two problems are actually polynomial. It is were the KDMD should shine - when the kernel can be guessed, as in 1 . More details should be given, as actually in Sec. 5.  REFERENCES 1 Steven L. Brunton, Bingni W. Brunton, Joshua L. Proctor, J. Nathan Kutz, “Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control,” http://arxiv.org/abs/1510.03007v2, (2015)


Single Pass PCA of Matrix Products
In this paper we present a new algorithm for computing a low rank approximation of the product A T B by taking only a single pass of the two matrices A and B. The straightforward way to do this is to (a) first sketch A and B individually, and then (b) find the top components using PCA on the sketch. Our algorithm in contrast retains additional summary information about A, B (e.g. row and column norms etc.) and uses this additional information to obtain an improved approximation from the sketches. Our main analytical result establishes a comparable spectral norm guarantee to existing two-pass methods; in addition we also provide results from an Apache Spark implementation 1 that shows better computational and statistical performance on real-world and synthetic evaluation datasets.

The authors have proposed a novel single pass PCA algorithm for computing the low-rank approximation of the inner product of two matrices A and B. Therefore, two smaller representations of the full-state matrices are computed using matrix sketching. Then, based on the sketched matrices, an estimator for the inner product is computed. The key innovation of the algorithm is the rescaled Johnson–Lindenstrauss (JL) embedding, used to estimate the important entries of the inner product. Finally, a matrix completion algorithm is used to obtain the desired rank-r approximation. In addition to theoretical results, the authors provide an Apache Spark implementation and empirical evaluations using both synthetic and real data.
The paper is well written, and the theoretically developments seem sound and interesting, though I checked the proofs in the appendix only briefly. The provided code is structured and well readable.  Technical quality: Overall the technical quality is very good. My main concern is with the experiments, specifically the algorithms run-time, shown in Figure (3a). The evaluation seems somewhat fuzzy for two reasons. (1) The sketch size 'k=10' is used, while in Table (1) the sketch size 'k=4000,8000' are shown. (2) The authors excluding the time for executing alternating least squares (ALS), because they claim that this is a common computational step. However, ALS is an iterative algorithms and the time to converge depends considerably on the initial input. Hence, I could assume that the proposed algorithms requires several more iterations to converge. Thus, the algorithm might not be that much faster in practice. I hope this issue can be clarified by the authors. Moreover, a Figure which shows the run-time vs. sketch size and in addition error vs. sketch size for the same data-set would be insightful.   Originality: The theoretical results seem to be different from prior work. In particular the work on rescaled JL embedding is interesting.     Potential impact or usefulness: The usefulness of the proposed algorithm relies mainly on the compute time. However as stated above, the computational gain (in practice) seems somewhat fuzzy. A better illustration of the trade-off between time and error would be useful.   Clarity and presentation: Overall the presentation is very good. The paper is well written. However some of the Figures are non vector graphics, e.g., 1 and 3a. Hence, the Figures look a bit blurry on the printed document or when zooming in.


Human Decision-Making under Limited Time
Subjective expected utility theory assumes that decision-makers possess unlimited computational resources to reason about their choices; however, virtually all decisions in everyday life are made under resource constraints-i.e. decision-makers are bounded in their rationality. Here we experimentally tested the predictions made by a formalization of bounded rationality based on ideas from statistical mechanics and information-theory. We systematically tested human subjects in their ability to solve combinatorial puzzles under different time limitations. We found that our bounded-rational model accounts well for the data. The decomposition of the fitted model parameter into the subjects' expected utility function and resource parameter provide interesting insight into the subjects' information capacity limits. Our results confirm that humans gradually fall back on their learned prior choice patterns when confronted with increasing resource limitations.

This paper proposes a model of bounded-rational decision-making using ideas from statistical mechanics and information-theory, in particular assuming that humans adopt a choice policy which maximizes expected subjective utility while minimizing KL divergence from a prior choice distribution, with the trade-off determined by a temperature parameter that is related to the amount of time/computational resources available to the individual for deliberation.  The model is tested on a set of data obtained from human subjects solving a pictorial version of the conunctive normal form (CNF) problem, and finds, based on the results, that subjects' prior choice distributions are similar to the true distribution over solutions, that the estimated temperature parameter varies monotonically with deliberation time allowed, and the estimated subjective utility peaks at the correct solution for well-trained puzzles but rather flat for an untrained puzzle.
Overall, this seems to be an interesting and mathematically sophisticated piece of work, bringing together quantitative tools from diverse feeds to bear on the modeling of bounded-rational decision making.   I feel the conclusions/framing of the paper is somewhat over-reaching, because the model of choice (primarily Eq. 1) is not really a model of decision-making, nor of learning.  What I mean is that the framework allows one to infer an individual's "subjective utility" (which from a normative perspective should be the distribution of correct answers for the trained puzzles), but does not say anything about how those utility preferences get computed, nor explain how these computations are learned or improved over time as the subject gets more experience with a particular puzzle.  The application to the CNF pictorial puzzle data set seems original, but with the way the task is designed (often very short viewing time), the task really seems more like a memory task than a decision-making task. The estimated utility function being flat for an untrained puzzle makes it clear that nontrivial computations & learning are indeed involved. In fact, it's clear from looking at Eq. 1, that if a subject really had access to the utility function Ux(y), then he should just pick the maximal utility response every single time for a given stimulus x, as that would yield better empirical utility (accuracy rate) than maximizing the full Eq. 1.  So Eq. 1 cannot be taken very literally as a model of how subjects make choices on each given trial, but rather is a descriptive model of how the distribution of choices might arise over a set of stimuli and presentations. The authors' feedback on this point did not assuage my concerns. Also, in terms of related work, there was a previous NIPS paper (Frazier & Yu, 2008) entitled "Sequential hypothesis testing under stochastic deadlines" that treated the problem of human decision-making under time pressure. While that took a very different approach at multiple levels, it is surely worth including as a reference in discussing how the current work situates in the ecology of human decision-making modeling literature.  Unfortunately, the actual findings are not very surprising/interesting at a scientific level. That the inferred "subjective utility" should peak at the correct solution for learned puzzles is hardly surprising, if subjects are indeed capable of learning to do this task through repeated trials. However, it does serve as a useful method validation step, to establish for a problem where the "correct" utility is known, that the utility function+beta+prior estimation procedure indeed performs as desired.  I would have liked to see an alternative model being compared to. In general it is good to have a baseline model as comparison so as to give some additional insight to the mechanism, correctness, and generality of the proposed model. In this particular case, because the way the decision-making model set up is so unusual compared to conventional practice, I think the readers would especially benefit from a comparison to (perhaps simpler) alternatives. The authors' feedback on this point did make me feel more reassured.  However, the 80% subject dropout rate mentioned in the rebuttal is extremely high! This would suggest that perhaps the experimental procedure resulted in substantial "subject filtering" to get a good model fit, instead of taking the more sensible "model filtering." AIC was better for the B-R model in 21 out of 30 included subjects, but really it could have actually been only out of a total 120 subjects. That hardly makes this seem like a persuasive model of human decision-making.  My final scores reflect the appearance/persistence of the various concerns that crystalized during the rebuttal phase.  Other comments: - Sec. 2 is rather dense with math, and it is not always clear whether a mathematical statement is an assumption, a derivation, an explanation, or an interesting aside.


PAC-Bayesian Theory Meets Bayesian Inference
We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by an i.i.d. distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.

The paper presents (long desired) links between PAC-Bayesian theory and Bayesian inference, based on treating the negative log-likelihood as a loss function and the resultant bounds show a trade-off between the KL-divergence between the prior and posterior and the NLL. Technically this involves extending the normal analysis that only applies to bounded losses to the real line. Secondly, it is shown that Bayesian model selection through the model evidence can be linked to minimizing the PAC-Bayes bound. They show how this is applied to the special case of Bayesian linear regression.
The paper is well written and theoretically strong. It's been conjectured in the past that there should be links between PAC-Bayes theory and Bayesian inference, but to my knowledge this is the first theoretically complete demonstration of such links.  Some comments:  - In eq(8) (and above) the notion of a prior with bounded likelihood is introduced. Am I right in thinking that this is a data-dependent prior, since it can only be known if the likelihood will be bounded for a given prior after observing the data? If this is not the case can you explain how such a prior is possible?  - I note that Theorem 3 is from unpublished work, although the proof is given as A.1. However I'm not sure the proof in A.1 is complete: Firstly \zeta_\pi and \Phi_{\ell, \pi, D} are not defined. Secondly, what are the requirements on X' and Y'?   - in the proof of 14/15: -- where does the first inequality come from? -- it would be nice to state that you used the Chi-squared MGF during the proof, as it took me a while to work this out -- I couldn't quite work out why the equality was only valid for \lambda < \frac{1}{c}. Again I think this relates to the chi-squared MGF?   Minor comments: - I'm not sure it helps to have so many references in the footnotes on p1. Perhaps if a numbered citation format was used rather than author-year, they could be put into the text - L22  do you mean implicitly rather than explicitly? - L95  Theorems 1 -> Theorem 1 - L185  mode -> model - L203  bounds -> bound - L206  is this L now the number of hyperparameters rather than the number of models? - L229 can be express by -> can be expressed as - L247 to a Gaussian -> by a Gaussian - Figure 1 panel (b) - legend is hard to read


Low-Rank Regression with Tensor Responses
This paper proposes an efficient algorithm (HOLRR) to handle regression tasks where the outputs have a tensor structure. We formulate the regression problem as the minimization of a least square criterion under a multilinear rank constraint, a difficult non convex problem. HOLRR computes efficiently an approximate solution of this problem, with solid theoretical guarantees. A kernel extension is also presented. Experiments on synthetic and real data show that HOLRR computes accurate solutions while being computationally very competitive.

This paper proposes a tensor-variate multilinear regression model called HOLRR for the prediction of a tensor output from vector input. They do this by formulating the problem as the minimization of a least squares subject to a multilinear rank constraint on the regression tensor. A kernelized extension of HOLRR for nonlinear setting is also introduced.
Strength: --The paper provides the theoretical analysis of approximation guarantees and a generalization bound for the class of tensor-valued regression functions.  --The paper is clearly written for readers to understand and appreciate the proposed algorithm.   Weakness: --A major drawback is that the novelty and contribution is rather limited.   The key idea and the model of this paper is actually equivalent to the HOPLS in the following paper: [Zhao et. al 2013a] "Higher order partial least squares (HOPLS): a generalized multilinear regression method. PAMI".  In HOPLS, it assumes the tensor input has low-rank structure and also the tensor output has low-rank structure, and the link of them is established in the common latent space. Specifically, HOPLS projects both input tensor and output tensor simultaneously onto a common latent space by using Tucker models. And then follows a regression step against the projected latent variables.   In HOLRR, this equivalence can be seen by taking look at the final regression function f^star in the paragraph "low-rank regression function" of Section 3.  For the input vector x: the term x^T*U_0 in f^star is in fact the latent variables, where the factor matrix U_0 projects the input vector x onto a common latent space vector.  For the output tensor Y: the authors assume tensor Y has a low-rank Tucker model structure, where G is in fact the core tensor of Y and U_1 to U_p are the loadings.  The regression from x to Y is linked by the latent vector as follows: f^star = (x^T*U_0)*G_(1) (U_p kron ... kron U_1), which is essentially the same as HOPLS, the correspondence can be seen clearly in Section 3.4 in the paper [Zhao et. al 2013a].  Thus, the model HOLRR is just a special case of model HOPLS where the input takes form of matrix (stacked vector samples) and the output assume to be a low-rank tensor.  The kernelized version is very similar to the Kernel-based tensor PLS regression which can be found in the paper: [Zhao et. al 2013b] "Kernelization of tensor-based models for multiway data analysis: Processing of multidimensional structured data".    --This paper only considers the correlations in the output tensor, and hence ignores (destroys) the input-side tensor structure by the vectorization of input data into a vector form, which may suffer the similar limitations as those ignoring the output-side structure.   In practice, most real-world tensor-variate regression applications involve tensor covariates input, which may limit the applicability of this model.   --For the algorithms, the authors simply apply the truncated HOSVD to find the projection matrices U_p for each mode, and HOSVD is usually used as a initialization point for other algorithms. I doubt that HOSVD is not as accurate as ALS-based HOOI that is used in algorithms such as HOPLS.   --For real-data experiments, the HOLRR is mainly tested on the spatio-temporal task against ADMM and GREEDY. The spatio-temporal data is a special case of general tensor data. I think some tasks involving general tonsorial input and output data should be tested.   In order to show the superiority of the proposed HOLRR, I think HOPLS [Zhao et. al 2013a] and the corresponding Kernel-based tensor PLS regression [Zhao et. al 2013b] should also be included in comparison on the general data.


Stochastic Structured Prediction under Bandit Feedback
Stochastic structured prediction under bandit feedback follows a learning protocol where on each of a sequence of iterations, the learner receives an input, predicts an output structure, and receives partial feedback in form of a task loss evaluation of the predicted structure. We present applications of this learning scenario to convex and non-convex objectives for structured prediction and analyze them as stochastic first-order methods. We present an experimental evaluation on problems of natural language processing over exponential output spaces, and compare convergence speed across different objectives under the practical criterion of optimal task performance on development data and the optimization-theoretic criterion of minimal squared gradient norm. Best results under both criteria are obtained for a non-convex objective for pairwise preference learning under bandit feedback.

This paper applies stochastic approximation methods in training log linear probabilistic structured prediction models under the bandit setting. It differs from the full information setting in that the learner only receives partial feedback for every prediction in the form of task loss evaluation, rather than a ground truth output with which to validate its prediction. The author(s)' stated main contribution of the paper is to apply stochastic first order (SFO) methods to learning with pairwise comparison feedback, and showing that it achieves the fastest convergence over two other settings (loss and cross-entropy minimization) based on empirical evaluations. The convergence properties of all three methods are analyzed based on existing results in the literature, which assume smoothness and boundedness conditions. Despite worst-case theoretical guarantees that favor the strongly convex cross-entropy objective, the author(s) reason that pairwise learning performed better due to smaller constant factors such as the variance of stochastic gradient updates. This conclusion is drawn from experimental evaluations.
A main result of this paper is that under bandit feedback, applying SFO methods in pairwise preference ranking (PR) and expected loss minimization (EL) can lead to better empirical convergence rates than in cross-entropy (CE) minimization, despite theoretical results that might suggest otherwise. The author(s) explained this result by estimating the constant factors of the bounds, such as the variance of stochastic updates, showing that they are orders of magnitude smaller for PR and CE. This leads to a statement in the conclusion that '... constants such as the variance of the stochastic gradient update can offset possible theoretical gains in convergence speed.'  Applying SFO methods in PR is interesting and has potential applications, but I see several issues with drawing the conclusion above from empirical evaluations, 1. The learning rates for all three methods were fixed at 1e^-6: the authors stated in line 183-184 that theoretical convergence for strongly convex functions (which applies to CE) were analyzed under diminishing learning rates, but used a constant rate in the experiments. I am not aware of similar bounds that hold under constant learning rate in the strongly convex case. How can one be certain that the apparent advantages of PR and EL over CE are not due to suboptimal choices of learning rates? 2. There is no justification of why 1e^-6 is chosen. Does the conclusion still hold under different values? 3. The constant factor estimates are based on averaging the results of 3 runs of each algorithm; I think the sample may be too small.  Another issue of this paper is that a number statements were presented without proper justification or context, which makes it difficult to assess their correctness. This is problematic because several of them are crucial for the paper's main results to hold. For example, 1. line 158-159, 'Note that multiplying the gradient by ...' -- Isn't this multiplication by 1? It is not clear how the sampling is done. 2. line 159-162, 'While minimization of this objective ... This further increases the variance ...' -- This explanation is unclear to me. Since this relates to a central claim of the paper, it would be helpful to clarify it. 3. line 175-177 -- These assumptions are crucial and it would help to be more precise here, e.g., by deriving the bounds on the variance explicitly under all 3 objectives. 4. line 187-189, For constant learning rates, even faster convergence can be shown ... -- What is the implication of this and how is 'faster' characterized? 5. line 193-194, 'Second, the standard asymptotic complexity bounds hide the constants ...' -- It would be helpful to state the bounds precisely with the inclusion of epsilon and all these constants, along with any other factors that may affect convergence rates (e.g, dimension of w and learning rate choices).  It would also help to provide more context in Related Work section. Instead of generic references to reinforcement learning and bandit learning (which are too broad), the author(s) could survey existing approaches to probabilistic structured prediction, going into more depth on how SA methods have been applied to closely related problems, and what this paper does differently.  Overall, I find the evidence presented in this paper and its explanations inadequate to support the conclusions. The paper could also use improvements in clarity and rigor of writing, especially when presenting theoretical results of convergence rates and the factors that influence them. For these reasons, I am unable to recommend it for publication.


Hierarchical Object Representation for Open-Ended Object Category Learning and Recognition
Most robots lack the ability to learn new objects from past experiences. To migrate a robot to a new environment one must often completely re-generate the knowledgebase that it is running with. Since in open-ended domains the set of categories to be learned is not predefined, it is not feasible to assume that one can pre-program all object categories required by robots. Therefore, autonomous robots must have the ability to continuously execute learning and recognition in a concurrent and interleaved fashion. This paper proposes an open-ended 3D object recognition system which concurrently learns both the object categories and the statistical features for encoding objects. In particular, we propose an extension of Latent Dirichlet Allocation to learn structural semantic features (i.e. topics) from low-level feature co-occurrences for each category independently. Moreover, topics in each category are discovered in an unsupervised fashion and are updated incrementally using new object views. The approach contains similarities with the organization of the visual cortex and builds a hierarchy of increasingly sophisticated representations. Results show the fulfilling performance of this approach on different types of objects. Moreover, this system demonstrates the capability of learning from few training examples and competes with state-of-the-art systems.

In the “Introduction” section, the authors point out the neurophysiological evidences that a human brain has a hierarchical structure for the object recognition, and that the learning and recognition of objects occurs concurrently in a human brain. Then, they briefly explained how they built their 3D object recognition model that concurrently learns and recognizes the objects that works in the similar way to the human brain.  The authors talk about conventional approaches for the object recognition problem in the “Related work” section. They mention about the probabilistic latent semantic indexing (pLSI), latent Dirichlet allocation (LDA) and its variations, etc. And they point out that none of these conventional models can learn and recognize objects in an open-ended manner. It was explained in the section, that an open-ended system is a system that can concurrently learn and recognize objects. They also mentioned that the number of classes for an open-ended system is not predefined and can grow continuously.  For section 3 to 6, the authors explained their proposed model. The model is designed to use the LDA in recognizing as well as in learning both categories and their features in an open-ended manner. To do this, they used 5 layers to build their system: feature layer, Bag-of-Words (BoW) layer, topic layer, view layer, category layer. The feature layer computes key points from spin-images. Then the BoW layer assigns a visual word for each spin-images. The topic layer uses LDA to discover abstract topics. Then the view layer stores the random mixture over latent topics of categories. Finally, the categorical output is made in the category layer.  The authors show their 3 experimental results at section 7. Their model was tested with a dataset using 10-fold cross validation method. The results showed that their topic layer, which develops independent topics for each category, showed better performance than the conventional approach (supervised LDA, class LDA) of sharing topics. And also showed that the test accuracy obtained from BoW layer is higher than the feature layer. Second experiment was conducted in an open-ended manner and showed a good performance of the model in recognizing and learning object categories concurrently. Finally, in the last experiment, the authors demonstrated how the model can work for robots with the aid of human instructors.
For the first experiment, the authors should have compared their model with a variety of models. There are many deep learning models for 3D object recognition, yet, their performances were not compared with that of the authors’ model when their model was tested in an off-line condition. Therefore I evaluated the technical quality of this paper as “Poster level”. In the paper, the authors used abbreviations such as PCL, OCD without explaining what they stand for. Therefore I gave “Poster level” for the technical quality. The proposed model has novelty in that it recognizes and learns the category of 3D objects concurrently using latent Dirichlet allocation. By using the model, the tedious work of making dataset, and training a robot with it to adapt to its working environment may be alleviated for the researchers in the robotics field. Therefore, I believe that this study has made an advance in the field of vison processing for the robotics. Therefore I evaluated both novelty and potential usefulness as “Oral level”.


Learning Bayesian networks with ancestral constraints
We consider the problem of learning Bayesian networks optimally, when subject to background knowledge in the form of ancestral constraints. Our approach is based on a recently proposed framework for optimal structure learning based on non-decomposable scores, which is general enough to accommodate ancestral constraints. The proposed framework exploits oracles for learning structures using decomposable scores, which cannot accommodate ancestral constraints since they are non-decomposable. We show how to empower these oracles by passing them decomposable constraints that they can handle, which are inferred from ancestral constraints that they cannot handle. Empirically, we demonstrate that our approach can be orders-of-magnitude more efficient than alternative frameworks, such as those based on integer linear programming.

This paper concerns the constrained optimisation problem of learning Bayesian networks from local scores with ancestral constraints. The search is branch-and-bound where the nodes in the search tree are CPDAGs and we 'move down' from CPDAG1 to CPDAG2 by (roughly) choosing a parent set for a BN variable which is not a node in CPDAG1. Sensible symmetry breaking is effected. The search is A* where the URLearning system is used to provide a heuristic. (Details of how this happens are not given here but are available in an earlier paper).
Given ancestral constraints, some pruning of the search tree is possible. Lemma 3 (supplementary material) is the key result here. I believe it to be true, but I don't understand the proof. The phrase "By the EC tree edge generation rules, G_k also contains edge Z - > W" needs more explanation.   In addition there are implied constraints ( "implied constraints" is the standard terminology, here they are called "projected constraints"). A bunch of ancestral constraints will imply constraints on edges which can be used for pruning.  Constraints on permissible topological orderings are also inferred ("ordering constraints"). The approach to doing this is not properly explained. We have the statement that we "can infer Y < Z from Z not an ancestor of Y". The issue is the word "can". Clearly there are DAGs with Z is not an ancestor of Y but Y < Z is a consistent order: the graph with vertices Z and Y and no edges, for example. The authors, of course, know this! What they mean to say is that if "Z not an ancestor of Y" then we can *choose* to add the constraint Y < Z. So the idea is to 'infer' as many consistent ordering constraints as possible (not just those which are entailed). The authors are careful not to 'infer' too many and use a MAXSAT approach to do the inference.  Experiments are conducted where the number of variables, datapoints and ancestral constraints are varied. There is a comparison with the GOBNILP system where the current system is shown to be far faster than GOBNILP. As the authors note, without ancestral constraints, GOBNILP performs worse than A* approaches when there is no limit on parent set cardinality. In the current experiments there is no such limit, so it is difficult to tell whether GOBNILP's poor performance here is due to its 'normal' failure to deal with many parent sets or is more specifically down to dealing with ancestral constraints poorly.  Also the encoding of ancestral constraints ( given in the supplementary material) is based on a MAXSAT encoding from [Cussens, 2008] rather than, say, the ILP encoding found in Section 3.1.1 of [Cussens, 2010]. And this MAXSAT encoding is a poor formulation for an ILP system. Why not just have the A(X,Y) binary variables in addition to the I(Z,U) variables and omit the (cubic number of!) E(X,Y,Z) variables? I think all that is then needed is the transitivity constraints on the A(X,Y) variables and the following constraints linking A and I variables:  A(X,Y) \geq \sum_{U:X \in U} I(Y,U)  In other words if X is a parent of Y then X is an ancestor of Y. This formulation provides a far tighter linear relaxation and should provide much better performance. This is the approach of Section 3.1.1 of [Cussens, 2010] for a total ordering on variables but with a constraint dropped since ancestor is a partial, not total oder.  I am confident that even with a sensible ILP formulation of ancestral constraints the current system will outperform GOBNILP, but the paper is weakened by comparing to a poor formulation rather than a sensible one.   It would be interesting to compare the current method to that of van Beek and Hoffmann, particularly since it often outperforms GOBNILP. I don't think it would be too hard to adapt that method to allow ancestral constraints to be posted. The paper by van Beek and Hoffmann is not mentioned by the current authors.   [Cussens, 2010] James Cussens. Maximum likelihood pedigree reconstruction using integer programming". WCB'10.  Peter van Beek and Hella-Franziska Hoffmann. Machine learning of Bayesian networks using constraint programming. Proceedings of CP 2015, Cork, Ireland, August, 2015.


Relevant sparse codes with variational information bottleneck
In many applications, it is desirable to extract only the relevant aspects of data. A principled way to do this is the information bottleneck (IB) method, where one seeks a code that maximizes information about a 'relevance' variable, Y , while constraining the information encoded about the original data, X. Unfortunately however, the IB method is computationally demanding when data are high-dimensional and/or non-gaussian. Here we propose an approximate variational scheme for maximizing a lower bound on the IB objective, analogous to variational EM. Using this method, we derive an IB algorithm to recover features that are both relevant and sparse. Finally, we demonstrate how kernelized versions of the algorithm can be used to address a broad range of problems with non-linear relation between X and Y .

This paper combines the information bottleneck method (IB) approach with sparse coding. It proposes two instantiations of IB: first one is using sparse priors on the internal representation, and the second one is a kernel extension of the first model. A variational algorithm for learning the model is proposed, and the model is evaluated on simulated and real data (handwritten digits).
I find the paper novel and interesting. To my knowledge the algorithm is original and it adds to the existing tollbox of IB based approaches. The proposed method seems to outperform Gaussian IB on denoising and occlusion/inpaiting tasks on simulated and real data. It also provides new analysis tools for sparse representations in the form of IB information curves. Overall I think this work has many promising applications in machine learning and neuroscience and would be of interest to the NIPS audience.  Below I list a few questions / suggestions which could help authors to further improve the quality/readability of the paper:  1) As I understand, encoding/decoding dictionaries $W, U$ are trained for a fixed constraint $\gamma$. What is the influence of $\gamma$ on trained dictionaries? Do different receptive fields emerge under different constraints?  2) The authors validate the model using simulated data and handwritten digits. While these are necessary tests which demonstrate the correctness of the method, used datasets are somewhat constrained. It would be interesting to see the algorithm validated on a more diverse set of stimuli e.g. natural image patches.  Alternatively, I think that the authors should provide a brief justification for their choice of data (handwritten digits).   3) Perhaps it would be good to include a brief justification for the choice of the sparse prior. Why did the authors choose Student-t distribution? Is it analytical convenience, or are there some other reasons? Why not a generalized Gaussian for instance? Different priors have different entropies, which as I understand should affect the performance of the model.  4) I like the discussion which relates the proposed model to previous infomax/sparse coding approaches. Can we understand them as special cases of the sparse IB model?   Minor comments:  While overall the paper is very clearly written I have a feeling that in a few cases an additional line of explanation would substantially improve reception. For instance in equation (5) - it is not immediately obvious what is the relationship between the upper bound on I(X;R) and the log-likelihood of R.  Line 38 - the objective is to minimize the Lagrangian L (in the form the authors present it), not maximize it  Line 44 - I do not understand that sentence - what does "and" refer to?, Also - the inequality within the sentence could be given some more context for clarity.  Figure 2 D - as I understand "stimuli" on that panel means Y, not X. Perhaps you could clarify that.


Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gradient method with constant minibatch converges to a stationary point. To tackle this issue, we develop fast stochastic algorithms that provably converge to a stationary point for constant minibatches. Furthermore, using a variant of these algorithms, we obtain provably faster convergence than batch proximal gradient descent. Our results are based on the recent variance reduction techniques for convex optimization but with a novel analysis for handling nonconvex and nonsmooth functions. We also prove global linear convergence rate for an interesting subclass of nonsmooth nonconvex functions, which subsumes several recent works.

The paper studies stochastic algortihms for minimizing a regularized finite sum problem, where the regularizer is nonsmoth but convex; and the functions in the finite sum are smooth but possibly nonconvex. This problem is not well studied and hence understood from a complexity point of view. Previous known results give complexity bounds (to a stationary point defined using the notion of the proximal mapping) for a method with increasing minibatch sizes. The present paper sets out to give the first results which do not require growing minibatches. Linear convergence results are obtained under a Polyak-Łojasiewicz condition. Minibatch proximal versions of the SAGA and SVRG methods are proposed and studied.
The paper is a pleasure to read – excellent writing. The ideas are well explained, the paper proceeds smoothly. The paper is well set in the context of existing results and literature (one small issue is the omission to mention that a minibatch version of Prox SVRG was analysed before in the case of smooth losses – and is known as the mS2GD method; see line 124).   The complexity (in terms of number of stochastic gradient evaluations; IFO) for both is shown to be O(n + n^{2/3}/eps); which is better than O(1/eps^2) rate of ProxSGD and O(n/eps) rate of ProxGD. The complexity is also improved for PL functions. The new results are a good addition to our knowledge of complexity of stochastic methods for  an important class of nonconvex problems relevant to ML. Some previous results are included as special cases of this approach (e.g., [27], [34]).  Experiments with non-negative PCA show these methods work about the same as much better than ProxSGD.  Q: Are any lower bounds known for the IFO complexity for this problem?   Remark: The word “fast” in the title evokes the use of Nesterov’s “acceleration” – but this is not the case. It is tempting to call a method fast, but if this is to be justified, either extensive computational experiments should be performed against all or most competing methods; or the word should have a special  meaning (such as Nesterov’s acceleration). I suggest the word be dropped from the title and replaced by “Complexity of”.   Small issues:  a) Line 40: to -> to be b) Table 1 contains complexity results for proxSGD under PL assumption. Where have these been proved (citation is missing)? Confusingly, the caption says that no specific complexity results exist. Do they exist or not? c) Line 151: For the -> For d) Line 154: is a -> be a e) Line 236: worser -> worse f) References: [9] : capitalize P and L in PL; [22] appeared in ICML 2015; [25] – capitalize journal name; [26] appeared in Math Prog 2016; [33] appeared; [34] – write SVRG and not svrg.   ----- post-rebuttal comments -----  I have read the rebuttal. I have the same opinion of the paper as before and keep my scores.


Variance Reduction in Stochastic Gradient Langevin Dynamics
Stochastic gradient-based Monte Carlo methods such as stochastic gradient Langevin dynamics are useful tools for posterior inference on large scale datasets in many machine learning applications. These methods scale to large datasets by using noisy gradients calculated using a mini-batch or subset of the dataset. However, the high variance inherent in these noisy gradients degrades performance and leads to slower mixing. In this paper, we present techniques for reducing variance in stochastic gradient Langevin dynamics, yielding novel stochastic Monte Carlo methods that improve performance by reducing the variance in the stochastic gradient. We show that our proposed method has better theoretical guarantees on convergence rate than stochastic Langevin dynamics. This is complemented by impressive empirical results obtained on a variety of real world datasets, and on four different machine learning tasks (regression, classification, independent component analysis and mixture modeling). These theoretical and empirical contributions combine to make a compelling case for using variance reduction in stochastic Monte Carlo methods.

This paper proposes two stochastic gradient sampling methods, motivated by combining the approach of Stochastic Gradient Langevin Dynamics with recent variance reduction techniques from the stochastic approximation literature.
The overall structure of the paper is reasonably clear, and the algorithms themselves are stated reasonably precisely (although there are inconsistencies in the notation used within the algorithm boxes), with clear motivation from the introductory material.  There are some typos (see minor points below), and quite a few instances where notation is used without being defined – in some circumstances it is clear what is meant, but in several circumstances it impeded my understanding of the paper. For example, the quantities \bar{\phi} and \hat{\phi} that appear in Theorems 1, 2, and 3 are not defined.  An important point which I don’t believe has been picked up in the paper is that the newly proposed methods are not MCMC methods as claimed in the abstract - the fact that gradients are stored between timesteps breaks the Markovian structure of the process.  In relation to the “Analysis” section of the paper, the two main theoretical results supporting the algorithms are clearly stated (although as noted above some notation is undefined).  There are some particularly strong conditions in assumption [A2], particularly the condition on uniform boundedness of the gradient log-prior and log-likelihoods, which restricts the applicability of Theorems 2 and 3. The condition rules out (for example) Gaussian priors or likelihoods, and therefore the theorems subsequently proven do not apply to any of the models in the experiments. It would be useful to explain what kinds of models the theoretical results can be applied to, and what kinds of models are excluded by these strong conditions.  I found the proofs of Theorems 2 and 3 in the appendix difficult to follow, due to undefined notation (e.g. C^\prime appearing in Eqn (16)), and missing notation (presumably the norms appearing in the displayed equations between lines 329 and 330 should have expectations taken over them).  Additionally, there appears to be a mistake in the displayed equations between lines 329 and 330; the expected value of the squared norm of eta_j, a d-dimensional multivariate normal random variable (with mean vector 0 and covariance matrix \sqrt{h} I), appears to be bounded by h. In fact, the expected value is exactly hd, where d is the dimension of the variable.  A general point about the final paragraph of Section 4 is that the bounds obtained in Theorems 1, 2, and 3 all have 1/(Th) and h^2 terms in common - would this mean that the bounds all behave similarly asymptotically, even if the first term in the bounds of Theorems 2 and 3 decays quicker than that of Theorem 1? If this is the case, then the claim that convergence of the MSE of SAGA-LD and SVRG-LD is significantly faster than that of SGLD might not hold.  In relation to the “Experiments” section of the paper, there is a good range of experiments (Bayesian linear regression, logistic regression, and ICA) which clearly demonstrate that SAGA-LD performs well in comparison to SGLD in several circumstances. It is a shame that no experiments relating to the SVRG-LD method were included.  The SAGA-LD algorithm begins with a full pass through the data to calculate the gradient log posterior at the initial parameter value - would this materially affect the comparisons drawn for the experiments if the methods were compared on a wall-clock basis, rather than number of passes through the data?  As a minor note, it appears as though in several of the regression and classification experiments, SAGA-LD and SGLD reach a similar level of MSE after several passes through the data set. It would be interesting to see whether initialising both algorithms at the MAP estimate of the posterior (say) would reduce the difference in performance between the two methods. Comparing the performance of the sampling schemes with pure optimisation algorithms such as SGD/SAGA could also provide a useful benchmark for the results obtained.  The mixture model experiment only shows results for SGLD (although perhaps this is a typo in line 258?), so it is not clear to me why it has been included in the paper.  Minor points: Throughout: “Robbins-Monroe” should be “Robbins-Monro” Line 23: the word “method” is repeated Line 50: “success” should be “successes” Line 76: U is the unnormalized *negative* log posterior Eqn (3): In the definition of the generator, the expectation should be conditioned on \theta_0 = \theta References: The 1951 Robbins-Monro paper appears twice in the list of references Appendix Section E: the “protein” graph appears twice


Solving Random Systems of Quadratic Equations via Truncated Generalized Gradient Flow
This paper puts forth a novel algorithm, termed truncated generalized gradient flow (TGGF), to solve for x ∈ R n /C n a system of m quadratic equations
random is known to be NP-hard in general. We prove that as soon as the number of equations m is on the order of the number of unknowns n, TGGF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with the time required to read the data {(a i ;
. Specifically, TGGF proceeds in two stages: s1) A novel orthogonality-promoting initialization that is obtained with simple power iterations; and, s2) a refinement of the initial estimate by successive updates of scalable truncated generalized gradient iterations. The former is in sharp contrast to the existing spectral initializations, while the latter handles the rather challenging nonconvex and nonsmooth amplitude-based cost function. Empirical results demonstrate that: i) The novel orthogonalitypromoting initialization method returns more accurate and robust estimates relative to its spectral counterparts; and, ii) even with the same initialization, our refinement/truncation outperforms Wirtinger-based alternatives, all corroborating the superior performance of TGGF over state-of-the-art algorithms.

The problem of recovering a vector from quadratic measurements (a.k.a. phase retrieval) is studied. This paper follows the now-standard recipe of formulating a (non-convex) optimization problem, which is solved by a gradient-descent-like method with suitable initialization. However, key changes are made to both the initialization and the gradient-descent stage: 1) As in existing works, the initialization is obtained as an eigenvector of a matrix. However, the matrix proposed in this paper is quite different from that used in existing methods. The improved sample complexity due to this change is demonstrated clearly in the numerical experiments (Figs. 3 and 4). 2) The refinement post-initialization is essentially gradient-descent, but ignoring a few undesirable data points at each iteration. The novelty here compared to existing works is that both the objective function and the truncation rule are new. The improved sample complexity due to this change is depicted in Fig. 1.  The overall advantages of the new method in terms of sample complexity are depicted in Fig. 5. The computational complexity is not significantly better than existing methods.
1) Figure 1 provides a comparison of different algorithms which are all initialized using the truncated spectral initialization. Here, the success rate of TWF is significantly worse than TGGF. It would be very interesting to see what happens if the initialization is replaced by the new initialization proposed in this paper. Is the success rate of TWF still worse than TGGF? If not, then it will tell us that the improvement of TGGF can be essentially attributed to the improved initialization. If yes, then it increases the strength of this paper by giving evidence for improvement which is due to the second stage. 2) The TWF paper also provides some experiments using structured sensing vectors (coded diffraction patterns). How does TGGF perform if the sensing vectors are structured and not random? 3) While the empirical results in the noisy case are very good, theoretical results for the noisy case are missing. This is not a major concern about this paper, because the empirical results for both the noiseless and noisy case, and the theoretical results for the noiseless case are already good. 4) For the noisy case, the measurement model is considered to be |a_i^Tx + eta_i| instead of |a_i^Tx|^2 + eta_i as in some earlier papers. Also, eta_i is assumed to be Gaussian, not Poisson or arbitrary. Since the paper anyway does not provide theoretical results for the noisy case, can numerical experiments for the case of Poisson noise be provided, which is a good model for imaging applications? 5) Is the improvement of TGGF over TWF purely in terms of the sample complexity? Say n=1000 and m=8000. How does the rate of convergence of TGGF compare with that of TWF?


Kronecker Determinantal Point Processes
Determinantal Point Processes (DPPs) are probabilistic models over all subsets a ground set of N items. They have recently gained prominence in several applications that rely on "diverse" subsets. However, their applicability to large problems is still limited due to O(N 3 ) complexity of core tasks such as sampling and learning. We enable efficient sampling and learning for DPPs by introducing KRONDPP, a DPP model whose kernel matrix decomposes as a tensor product of multiple smaller kernel matrices. This decomposition immediately enables fast exact sampling. But contrary to what one may expect, leveraging the Kronecker product structure for speeding up DPP learning turns out to be more difficult. We overcome this challenge, and derive batch and stochastic optimization algorithms for efficiently learning the parameters of a KRONDPP.

The authors propose two efficient algorithms (learning and sampling) for determinantal point processes (DPP) in which the DPP-kernel matrix factors as the Kronecker product of smaller positive definite matrices. The first proposed algorithm learns a DPP-kernel which can be factored as the Kronecker product of two positive definite matrices in time quadratic in the number of items. This is an improvement over the existing algorithm [25] for learning a DPP-kernel matrix which runs in time cubic in the number of items -- however, the existing algorithm [25] works without assuming the Kronecker structure of the DPP-kernel. The stochastic version of the algorithm is even faster and requires time O(N^{1.5}), with N denoting the number of items. The second proposed algorithm is for sampling from a DPP with the DPP-kernel having the Kronecker structure. This algorithm runs in time O(N^{1.5}) and represents an improvement over more general DPP-sampling algorithms which do not assume the Kronecker structure of the DPP-kernel and require either cubic time in the number of items or a MCMC sampling with an impractical mixing bound.
The paper can be split into three major components: i) learning algorithm, ii) sampling algorithm, iii) experiments.  i) Learning algorithm: The algorithm works by maximizing the log-likelihood of the model over the space of positive definite matrices which factor as the Kronecker product of smaller positive definite matrices. It represents an extension of the algorithm for learning determinantal point processes proposed in [25] and which does not assume that the DPP-kernel has the Kronecker structure. The authors overcome the issue with the additional constraint on the DPP-kernel by performing the block-coordinate ascent. In Theorem 3.2 they show that the proposed optimization method makes the objective non-decreasing (a similar result was also shown in [25] for the algorithm which does not assume the Kronecker structure of the DPP-kernel). The authors also give guidelines how to generalize the algorithm for DPP-kernels which can be represented with the Kronecker product of more than two matrices. However, for the case of more than two matrices in the Kronecker product the authors do not show the computational gains over [25] and learning without assuming the Kronecker structure (see Theorem 3.3).  @LA1: The maximization of the log-likelihood is non-convex problem and while the proposed version of the block-coordinate ascent is guaranteed to result in non-decreasing objective there is no guarantee that the learned model is a globally optimal one. As this is a non-convex problem and, hence, highly dependent on the initial solution, it would be nice to have a heuristic or guidelines for initializing the solver in Algorithm 1. Has this aspect been tested thoroughly while empirically evaluating Kronecker DPPs? @LA2: It is not clear how constraining is the assumption that the DPP-kernel factors as the Kronecker product of two positive definite matrices (for this case, the authors give an efficient implementation). How does this assumption affect the capacity/properties of such DPP models?  ii) Sampling algorithm: The algorithm exploits the fact that the eigen-decomposition of DPP-kernels which factor as Kronecker products of smaller positive definite matrices can be performed by decomposing the smaller matrices (see Corollary 2.2).  @SA1: The proposed sampling algorithm is the standard DPP-sampling algorithm with an efficient eigen-decomposition which is possible due to the assumed Kronecker structure of the DPP-kernel.   iii) Experiments: The main focus of the experiments is to compare the learning algorithm to [25]. This is achieved via synthetic experiments and experiments on two real-world data sets. In one of the experiments with a real-world data set, the authors confirm that the Kronecker DPPs are less expressive than the DPPs with kernels not assuming such structure. The second experiment with a real world data set, shows a clear reduction in the training time of Algorithm 1 for Kronecker DPPs over [25], designed for standard DPP-learning.  @E1: It is very difficult to distinguish the plots when printed black-and-white. It would be nice if the authors could assign a different marker to each of the plots (e.g., circle, square etc.). @E2: Given the fact that the problem is non-convex it is rather difficult to empirically assess the loss in the expressiveness which comes as a result of the Kronecker structure assumption. However, in my opinion this is an important aspect of the Kronecker DPPs that was not addressed properly by the authors in the experiments.  Minor comments: -- line 40: Y \in \mathcal{Y} is just an element of the set of items and (1) can be misinterpreted (perhaps, to use the power set instead) -- line 43: L_Y is not introduced in the text around (2) -- line 85: bracket missing on Tr_1 -- line 112: notation with the composition of f and the Kronecker product looks suboptimal -- line 133: typo on `Generalization' -- lines 137-141: it would be nice to add more details to the appendix -- line 166: `by from' -- line 200: `true' kernel is not clear enough -- Appendix: Algorithm 3, `end forreturn ...' ------------------------------------------------------------------------------------  The paper is very well organized and extremely easy to follow. All the proofs are clear enough and the compressed parts can be derived easily. It is a solid paper with a very elegant theoretical part. Also, a very interesting observation was given in Section 6, where the authors observe that one of the two parts of the log-likelihood function is geodesically convex over the Riemannian manifold of positive definite matrices and it will be interesting to see how this idea develops in the future.


Learning Supervised PageRank with Gradient-Based and Gradient-Free Optimization Methods
In this paper, we consider a non-convex loss-minimization problem of learning Supervised PageRank models, which can account for features of nodes and edges. We propose gradient-based and random gradient-free methods to solve this problem. Our algorithms are based on the concept of an inexact oracle and unlike the state-ofthe-art gradient-based method we manage to provide theoretically the convergence rate guarantees for both of them. Finally, we compare the performance of the proposed optimization methods with the state of the art applied to a ranking task.

The authors motivate pagerank and then bring up a framework which has node and edge weights from a feature model. They pointed out that a previous algorithm for learning the parameters of the model had no performance guarantees. They plan to solve the methods by combining a previous method for approximating the stationary distribution of a Markov random walk with an adaptation of a gradient free method used for optimization when the loss function can only be estimated. They reproduce a framework under a different name and notation from [21]. They discuss a number of technical assumptions and a loss function and explain why the problem is hard to solve. Some lemma bounding the approximate gradient are stated. A general gradient free method is discussed, using a biased gradient-free oracle, and state a theorem on its performance. They make some more assumptions and construct an algorithm for the learning problem and adapt the gradient free methods to their problem, stating some more bounds. They outline some details of experimental results on a web graph used in related work, but detailed reporting of results and analysis are relegated to the appendix. Some parameters are not reported, and the novel algorithm is proclaimed to have outperformed other algorithms, but there is no discussion of the extent or implication of the improvement. It is unclear whether a diverse set of graphs were considered for testing.
The writing is littered with grammatical errors. Citations are sloppy, the problem is called Supervised PageRank with a cite to [21], but only semi-supervised page rank appears there. I am not convinced that the problem solved is impactful.    Your bounds are strong, but I have doubts about whether the technical assumptions stated will hold in emprical data. Some assumptions seem limiting, e.g. your constriction of the search space of phi to an R-ball around some chosen phi_0may greatly restrict the efficacy/applicability of your algorithm, which should be discussed.   My biggest issue with the paper is that it all the meat appears in the appendix. You neither provide proof outlines nor full empirical results. How did you choose phi_0 and R in the empirical results section? Did you run tests on more than one graph? You do not report any tables or specific results in the main body of the paper. What are the practical consequences of your improvements?  Your bounds and analysis and algorithm may be powerful, but the content in the paper itself fails to convince me.


Exponential Family Embeddings
Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data. As examples, we studied neural data with real-valued observations, count data from a market basket analysis, and ratings data from a movie recommendation system. The main idea is to model each observation conditioned on a set of other observations. This set is called the context, and the way the context is defined is a modeling choice that depends on the problem. In language the context is the surrounding words; in neuroscience the context is close-by neurons; in market basket data the context is other items in the shopping cart. Each type of embedding model defines the context, the exponential family of conditional distributions, and how the latent embedding vectors are shared across data. We infer the embeddings with a scalable algorithm based on stochastic gradient descent. On all three applications-neural activity of zebrafish, users' shopping behavior, and movie ratings-we found exponential family embedding models to be more effective than other types of dimension reduction. They better reconstruct held-out data and find interesting qualitative structure.

The paper deals with the general problem of finding latent embeddings for a variety of problems and places the problem in a Bayesian framework by showing that embedding problems from several different applications can be expressed as inference in an exponential family scenario. In this case each observation has both an embedding and a context vector. This framework is defined by three main choices (1) defining the context of an observation, (2) deciding how the conditional probability of an observed data point is linked to its context and (3) how the latent embeddings and context vectors are shared across different observations. Inferring the embeddings and context vector can then be done via stochastic gradient descent fairly simply due to properties of the gradient when dealing with exponential families The generality of this framework is then demonstrating that four different real-life embeddings problems can be formulated in this way. The three choices are addressed for each of the four and a gradient formula is derived for each. In the case of two of the scenarios (real valued neural data and Poisson observations for shopping/movies) the EF-EMB method is applied to some real life data sets and empirically strong results are shown in terms of likelihood estimations/errors. Moreover, some quantitative results on these datasets are given to show how having both a context vector and latent embedding can help in the interpretation of results in these contexts.
Technical quality: 3   The experimental method was well motivated and correctly seemed to generalize to the scenarios given. The derived gradients were correct (if a little too succiently explained for my taste). The experimental results themselves were strong (though I am not an expert in any of the areas so hard to say whether they picked the exact best things to compare against) and the additional quantitate analysis was a good decision since it shows there will be a consistent way to interpret results. I think the method seems to actually produce good results, has a good intuition behind it and could actually be useful which in my mind is much more important than excessively long proofs. There were a few questions about the experiment analysis that should have been addressed. In table 2 it’s odd for example that both P-EMB and AP-EMB both perform worse from K=50 to K=100. Since this measures purely the likelihood and does not include the prior theoretically increasing the size of K should only ever improve the score. In the worst case, letting the last 50 embedding and context vector values always be zero and letting the first 50 be the same as the optimal result for K=50 should give you a score as before. This seems to suggest that the SGD has some problems converging for higher dimensions so it would have been nice to have this addressed (e.g: was the convergence stopped due to time constraints). Generally mentioning the running times of EF-EMB methods versus the alternatives would have also been informative. One aspect I found confusing is that the link function introduced in equation 2 was generally assumed to be the identity function. However this runs into problems when the size of the context for each observation point is not constant (as was noted in the experimental analysis of the movie ratings). It seems like taking the dot product of the latent embedding and the weighted average of the context vectors is more generally the correct option.    Novelty/originality: 3  There is no new algorithm here but the idea of putting many different embeddings under the same sort of framework is new. The main novel contribution is noting that this could be done and esthablishing which key steps are required to do this. Moreover, each of the 4 exponential family versions of the embeddings for the difference scenarios are a novel contribution and appear to be empirically useful formulations. The experimental methods are fairly standard but seem fully appropriate for the problem of comparing old methods with EF-EMB.  Potential Impact: 4  I feel the authors have esthablished several key things about this framework. Firstly they have shown that it is fairly general through good use of 4 very different examples which seem well served by the method. Secondly, they have shown that the method is fairly easy to apply to a problem if you are familiar with the problem and exponential families. Once I review EF it was fairly easy to follow how they derived these new models and this ease of understanding makes it likely to use. Finally, they have shown that this is useful both in terms of the quantitate results but also in terms of giving us some examples of how this can be used to interpret results for exploratory reasons. Though this third part is perhaps the one I would like to see improved the most since they have not talked about how this fares for problem where the dimensionality of the embedding is very high (K was limited to just 100 in the examples) nor shown results for the more well studied problem of word embeddings. As previously mentioned there is no mention of how long it takes to run compared to other options which is also something that would help us understand how applicable the method might be.  Clarity and presentation: 3  Disclaimer, I last looked at exponential families 5 or so years ago so had to refresh myself on this several times so keep that in mind when reading this section. In my mind this was the weakest area of the paper and also the least balanced. I though the paper was very well motivated as we moved through sections 1 and 2: I was following the idea well, understood the idea of what needs to be defined to make an EF-EMB etc. Things starting getting a little harder to follow around 2.1. I think the authors presume a fair amount of familiarity with exponential families and would have benefitted from just stating things a little more explicitely. In example 1 of 2.1 for example there is discussion of a Gaussian with mean esthablished by the inner product but no mention of the standard deviation: it was only after a bit of reading that I saw this is replaced by lambda in the case of an exponential family prior-posterior case. The hardest aprt to follow was 2.2. Without referencing the canonical way to set up an exponential family conditional ( f(x| theta ) = h(x)exp(eta(theta). T(x) – A(theta) ) its not clear what terms like eta represent or to even find out what a(eta) is (which appears without warning in equation three first). Obviously its not a huge flaw but it does add a significant amount of effort to what was otherwise a pretty smooth read beforehand.  I was also a little confused by the use of “regularizer” in the objective function. From what I can understand this could more properly defined as the priors of the latent embeddings and context vectors and ensures that the objective function is seeking to maximize the entire posterior. Some of the equations’ derivations were a little terse. I can understand that in the interest of space not mentioning them in the main paper but it would have been nice to have a little discussion of what the prior is and how the chain rule is used in appendix A.1 for example. Again, nothing that can’t be worked out without a little extra research/calculations but neither is it something desirable. The experimental analysis was fairly easy to follow but I would have liked a little more detail on exactly what kind of factor analysis was being used for the first set of experiments as well as an explaination of why other parameter choices (such as higher values of k) were not explored.  Summary  Overall though I rather enjoyed reading this paper and think it is a great piece of work. It provides a framework for several ideas that is theoretically justified, useful and somewhat interpretable. The fact that it is fairly to simple to understand and does not rely on advanced mathematical machinery is yet another advantage. The generalizations seem generally good though there is certainly space for more investigation (e.g: can it be modified to admit cases where context data points can be weighted, e.g weighing the context word right next to the target word more than that five places to the right). The experimental work certainbly looks promising but I would like to see that get filled in with more analysis: especially on more computationally intensive scenarios.