Automated Variational Inference for Gaussian Process Models
We develop an automated variational method for approximate inference in Gaussian process (GP) models whose posteriors are often intractable. Using a mixture of Gaussians as the variational distribution, we show that (i) the variational objective and its gradients can be approximated efficiently via sampling from univariate Gaussian distributions and (ii) the gradients wrt the GP hyperparameters can be obtained analytically regardless of the model likelihood. We further propose two instances of the variational distribution whose covariance matrices can be parametrized linearly in the number of observations. These results allow gradientbased optimization to be done efficiently in a black-box manner. Our approach is thoroughly verified on five models using six benchmark datasets, performing as well as the exact or hard-coded implementations while running orders of magnitude faster than the alternative MCMC sampling approaches. Our method can be a valuable tool for practitioners and researchers to investigate new models with minimal effort in deriving model-specific inference algorithms.

I have read the authors feedback.Summary of the Paper:The paper describes a black-box variational inference method that is specific to models based on Gaussian processes. The method works by applying variational inference as the approximation method and by using a Gaussian mixture as the approximating distribution. The authors show that the lower bound in variational inference can be further lower bounded as the sum of a term that approximates the entropy of a Gaussian mixture and a cross-entropy term between the prior and the posterior approximation. A last term remains which is an expectation of the log likelihood of the model with respect to the posterior approximation. The nice thing is that the gradients of all terms except the one that depends on the likelihood can be computed in closed form. The gradient of the remaining term can be specified in terms of an expectation with respect to the posterior approximation and they can be stochastically approximated by using samples from a Gaussian distribution. In summary, the authors describe a method to carry out variational inference without the need to computed closed form expectations that may not be analytically tractable. The advantage of the method is that it computes gradients with respect to the hyper-parameters very easily. The method is compared in several experiments with deterministic VI and Markov chain Monte Carlo methods.Clarity:The paper is very well written and the use of the English language is correct. However, a few points in the paper should be further clarified. I am wondering what happens when the log of the likelihood cannot be computed. For example, this may happen when the likelihood factors are Heaviside functions of the latent functions. I also do not understand the paragraph between lines 251 and 253. It seems that the authors say that they need not use stochastic optimization methods.  However, the do approximate stochastically the gradient of the expected log likelihood. I find this statement a bit contradictory. Another point is that the authors say that they iterate until the changes in the ELBO are less than 1e-5 or the changes in the variational parameters are smaller than 1e-3. Are they evaluating the ELBO always? I think their method can be applied when this is not possible. Originality:The method described can be understood as the application of the more general method described in [16] to the specific case of Gaussian process models. The method in [16] is more general and can be applied to a range of broader models. However, for this same reason, it does not exploit the particularities of Gaussian process models. Thus, I believe the work presented here is original.Besides [16] the authors could also mention a recently published work. Namely,Michalis K. Titsias and Miguel Lazaro-Gredilla. Doubly Stochastic Variational Bayes for non-Conjugate Inference. Proc of the 31st International Conference on Machine Learning (ICML 2014)that is based on similar principles.Quality:I believe the quality of the paper is good in general and the experiments are relevant and representative for the method proposed. The same can be said about the methods the authors compare with. One thing that could be criticized is that only 5 runs are carried out and this might be insufficient to extract more conclusions from the experiments. Another thing that puzzles me is that there seems to be no clear advantage in the experiments for using a mixture of two components when compared a single component. This questions a bit the selection of that family for the posterior approximation.Significance:The experiments carried out in the paper show that the proposed method perform well in general and equivalently to other approaches based on Markov chain Monte Carlo or on deterministic variational inference. The advantage is that it needs not the computation in closed form of expectations with respect to the likelihood. These expectations are approximated by computing empirical averages with respect to a Gaussian distribution. The disadvantage is that this process is computationally more expensive, but not as expensive as Markov chain Monte Carlo methods. Again, the only point that questions a bit the significance of the proposed method is the fact that there seems to be no clear advantage of using a mixtures of Gaussians as the approximating distribution. An important point is that the authors claim advantages with respect to the method described in [16]. Specifically, they say they can very easily infer the kernel hyper-parameters, while with the method of [16] this is much more difficult. However, they do not compare with the particular approach of [16]. This questions the significance of the proposed method. A last criticism is that the authors only report gains in computation speed with respect to HMC in Section 4.4. I wonder what are the results when compared to VBO or EP. The method proposed is going to be more expensive, but by how much?Strengths:- Very well written manuscript.- Theorems to support the claims of the paper.- Simple but effective method.- Many methods compared on very different settings.Main Weaknesses:- The use of a Gaussian mixture is not supported by the experiments.- Only 5 folds are used in the experiments to compute averages.- Not comparing with respect to related methods such as [16].


A Wild Bootstrap for Degenerate Kernel Tests
A wild bootstrap method for nonparametric hypothesis tests based on kernel distribution embeddings is proposed. This bootstrap method is used to construct provably consistent tests that apply to random processes, for which the naive permutation-based bootstrap fails. It applies to a large group of kernel tests based on V-statistics, which are degenerate under the null hypothesis, and nondegenerate elsewhere. To illustrate this approach, we construct a two-sample test, an instantaneous independence test and a multiple lag independence test for time series. In experiments, the wild bootstrap gives strong performance on synthetic examples, on audio data, and in performance benchmarking for the Gibbs sampler. The code is available at https://github.com/kacperChwialkowski/ wildBootstrap.

In this paper, which is in the general context of nonparametric testing based on Hilbert space embeddings, a novel Type 1 error control calibration method for nonparametric hypothesis tests of two weakly dependent samples/processes is proposed. The paper strongly builds on recent work in mathematical statistics [18,25] regarding the wild bootstrap method. The main contribution consists in showing that V statistics (of any order) and in particular those based on RKHS embeddings (including MMD and HSIC) can be asymptotically consistent even for weakly dependent processes (tau mixing). The approach is evaluated in terms of the achieved Type 1 and 2 error rates on syntethic, audio, and Gibbs sampler data.First of all, the paper is very well written and concise, and despite the high complexity, relatively easy to follow. The paper addresses a the very relevant topic of nonparametric statistical testing based on RKHS embeddings of probability distributions, which has been a vivid topic in ML in recent years. The main contributions of the paper from my point of view are:- introducing the wild bootstrap method in ML (has solely been used in statistical literature up to now)- proving that the wild bootstrap yields asymptotic consistency under weak dependence assumptions (tau mixing), for both the null and alternative hypotheses- Indicating the empirical usefulnesssThe strongest parts of the paper are the methodological contribution and innovative idea, and of course the strong theoretical part, establishing deep theoretical foundations of the suggested testing procedure. The analysis, which is very sounds, is general enough to apply also to V statistics beyond the common ones in ML (i.e., beyond HSIC and MMD). The theory alone would make a good NIPS contribution, but in addition there is also a empirical analysis, which is -- although not entirely convincing when being picky (see detailed comments below) -- promising and shows superior performance of the method in comparison to appropriate baselines.In summary, this is a novel methodology for a relevant NIPS topic, which is addressed in deep theoretical detail, and included a promising empirical analysis. A clear accept.Minor, quite picky comments:- Introduce probability operator \Epsilon or use standard notation like \mathbb P- Use of the word "control": please correct me if I am wrong, but in my understanding the type 1 error is controlled if the truly measured type 1 error is *smaller or equal* to the desired level. In several experiments (e.g., MCMC Vb1) the actual rate is slighly higher than the desired one, which would strikly speaking mean the error is not entirely controlled  (if I get the terminology right).-  Introduction of the wild bootstrap at the end of Sect 2: this is very technical, but nobody in ML knows this quantity. Show an illustration, what does it do intuitively? Example of how to choose the wild bootstrap process: for instance, the statistical learning theory reader might wonder whether it makes sense to use a correlated Rademacher or Gaussian process here.- Beginning of Sec 4: "This quantity". Let's say I wouldn't know anything about MMD: it is unclear to which term "this quantity" refers.- Top of page 5: "[18] and 2" -> [18] and [25] ?- experiments: make clearer what the difference between MMD hat and Vb1 and Vb2 is. Also note that the Type 1 error is not controlled, it is just closer to the desired error- For a sound empirical analysis, you could use ROC curves. It is neccessary to compare the type 1 error and the power / type 2 error at various level simultaeoulsy for a fair comparision. So:1. at some places in the text (e.g., end of page 7) you mention the measured type 2 error. This is important and I would suggest mentioning this also in the captions of the figures.2. What does Figure 1 tell me? The methods could be better evaluated if you plotted a roc or pr curve or something like that, I would think.3. For the same reason: figure 2 and end of page 8. "was around 5%". This is a little vague and again this should be mentioned in the caption of figure 2.Same page: "The type... was equal or lower ..." This is again vague. A roc or pr curve would shed light on this.- The Bonferoni correction for HSIC-Lag will be too conservative under the alternative hypothesis. Is there a way to wrap a (wild) bootstrap around the whole lagged kind of procedure / come up with a joint statistics, which is calibrated via wild bootstrap?


Stochastic Variational Inference for Hidden Markov Models
Variational inference algorithms have proven successful for Bayesian analysis in large data settings, with recent advances using stochastic variational inference (SVI). However, such methods have largely been studied in independent or exchangeable data settings. We develop an SVI algorithm to learn the parameters of hidden Markov models (HMMs) in a time-dependent data setting. The challenge in applying stochastic optimization in this setting arises from dependencies in the chain, which must be broken to consider minibatches of observations. We propose an algorithm that harnesses the memory decay of the chain to adaptively bound errors arising from edge effects. We demonstrate the effectiveness of our algorithm on synthetic experiments and a large genomics dataset where a batch algorithm is computationally infeasible.

The paper presents stochastic variational inference (SVI)  for HMMs. Unlike LDA, in HMMs the chain-based dependence structure poses problems for SVI. Subsampling in each iteration cannot assume independence between series data, and skipping any data outside the chain can lead to errors.  (Network models present somewhat different difficulties, replacing chain-based with pair-wise dependence. Subsampling in this setting has been handled in prior work (Gopalan et al. (2012)).)In this work, the authors show how to scale the noisy subsample gradients for unbiased estimates and bound error induced by updating only variables corresponding to subchain samples. They augment subsampled subchains with extra observations to achieve this. They argue that this buffering and the forward-backward computations can be done incrementally, and therefore efficiently. They apply their algorithm to a large genomics data set, and demonstrate similar performance to VB.pros:the paper is written well; contributions through developing an SVI algorithm are clear and significant.cons:- my main criticism of this work is that a comparison to Johnson et al. (2014) is not provided. That paper develops an algorithm for large collections of independent series. surely, the SVI algorithm in your paper makes better (and correct) assumptions; but it's hard to know if these additional complexities are worth it, in practice, when running on real data. comparison to an EM algorithm is insufficient, as your gains could derive from using SVI. - You mention "significant gains" in computation for the genomic data compared to the EM algorithm, but no runtime evidence is provided. - computational complexity of the batch vs. SVI algorithm is not mentioned (although it's evaluated); i assume the SVI is still has quadratic per-iteration complexity in terms of states? - what is the worst-case cost when L is set poorly? the algorithm requires a good choice of subchain lengths (L) and number of chains per mini-batch. this is not unusual for SVI algorithms, where mini-batch size can plays a key role (Hoffman et al., (2010)). however, choosing a good L is critical, as it may cause the "buffering" algorithm to add too many observations for a good approximation. the authors state that this is not the case, but it's not convincing, as it's in the case of simulated data.


Tight convex relaxations for sparse matrix factorization
Based on a new atomic norm, we propose a new convex formulation for sparse matrix factorization problems in which the number of non-zero elements of the factors is assumed fixed and known. The formulation counts sparse PCA with multiple factors, subspace clustering and low-rank sparse bilinear regression as potential applications. We compute slow rates and an upper bound on the statistical dimension [1] of the suggested norm for rank 1 matrices, showing that its statistical dimension is an order of magnitude smaller than the usual 1 -norm, trace norm and their combinations. Even though our convex formulation is in theory hard and does not lead to provably polynomial time algorithmic schemes, we propose an active set algorithm leveraging the structure of the convex problem to solve it and show promising numerical results.

This paper proposes an atomic norm for estimating low-rank matrix whose factors are sparse. The authors show that the proposed atomic norm can be expressed as additive decomposition of a given (or parameter-) matrix into a sum of low-rank matrices, each of which is supported on at most k rows and q columns. The authors show a "slow rate" for the denoising performance of the proposed norm and compare an upper bound for it with a lower bound for another norm defined as a convex combination of the trace norm and the L1 norm. The method is applied to a drug-target-interaction-prediction problem and shows a AUC score better than the one predicted in the previous paper.The strength of the paper is that it tackles a problem that have wide applications (bilinear sparse regression, subspace clustering, sparse PCA) with a promising approach. The theory and experiments also seem to show the advantage of the proposed atomic norm over simply using the trace norm, L1 norm, or the convex combination of them.The weaknesses of the paper are as follows. - the theories (denoising bound and statistical dimension) are both limited to rank one. - optimization algorithm could be improved (it uses a generalization of sparse PCA algorithm [22] to compute the dual norm). Could the observation that the proposed norm reduces to the k-support norm for m_2=1 be used to develop a fast algorithm as in [2]? - Details of the drug-target-interaction experiment are missing. How were the training and test sets seprated? How sensitive was the result to the choice of k and q. - There were many typos. Sometimes it was hard to understand what the authors mean. For example, "By contrast to sequential approaches that estimate the principal components one by one [12], this formulation requires to find simultaneously a set of complementary factors." (line 164). What do you mean by complementary?Additional comments: - line 306: K cannot be PSD because K is in general asymmetric. - Does the theoretical guarantee in [22] extend to the proposed asymmetric generalization of TPI algorithm?


Beta-Negative Binomial Process and Exchangeable Random Partitions for Mixed-Membership Modeling
The beta-negative binomial process (BNBP), an integer-valued stochastic process, is employed to partition a count vector into a latent random count matrix. As the marginal probability distribution of the BNBP that governs the exchangeable random partitions of grouped data has not yet been developed, current inference for the BNBP has to truncate the number of atoms of the beta process. This paper introduces an exchangeable partition probability function to explicitly describe how the BNBP clusters the data points of each group into a random number of exchangeable partitions, which are shared across all the groups. A fully collapsed Gibbs sampler is developed for the BNBP, leading to a novel nonparametric Bayesian topic model that is distinct from existing ones, with simple implementation, fast convergence, good mixing, and state-of-the-art predictive performance.

This paper presents a new Bayesian nonparametric model for mixed-membership modeling of grouped data. The model is based on a Beta-Negative Binomial Process (BNBP). Whereas this stochastic process was exploited by previous work, as mentioned in the paper, there are several aspects that set this work apart. First, this work obtains an EPPF (exchangeable partition probability function), and derives the prediction rules thereon. The relations between Bayesian nonparametrics and EPPF has been studied for quite a while in statistics, following the seminal work by J. Pitman. Yet, this concept has not been extensively leveraged in the practice of BNP research. The connection between BNBP and EPPF made in this paper, to me, represents an important step along the direction of unifying the EPPF analysis and BNP research, which would potentially lead to more work along this line.Second, the idea of randomizing the group sizes (which are treated as given constants in most other work) is interesting. It turns out that this idea makes it possible to derive some marginal distributions.I think this paper has important theoretical contributions. However, the practical utility of this work remains in question. The inference algorithm is obviously more complex than HDP (in both theory and algorithms). The improvement demonstrated in the results is not sufficient to convince me that this increased complexity is worthy in practice. Also, the paper did not provide an intuitive explanation (or justification) as to why the proposed framework would actually work better than HDP in practice.


Efficient Partial Monitoring with Prior Information
Partial monitoring is a general model for online learning with limited feedback: a learner chooses actions in a sequential manner while an opponent chooses outcomes. In every round, the learner suffers some loss and receives some feedback based on the action and the outcome. The goal of the learner is to minimize her cumulative loss. Applications range from dynamic pricing to label-efficient prediction to dueling bandits. In this paper, we assume that we are given some prior information about the distribution based on which the opponent generates the outcomes. We propose BPM, a family of new efficient algorithms whose core is to track the outcome distribution with an ellipsoid centered around the estimated distribution. We show that our algorithm provably enjoys near-optimal regret rate for locally observable partial-monitoring problems against stochastic opponents. As demonstrated with experiments on synthetic as well as real-world data, the algorithm outperforms previous approaches, even for very uninformed priors, with an order of magnitude smaller regret and lower running time.

Summary=============The paper proposes a novel algorithm for the problem of online learning with partial monitoring against a stochastic adversary. The proposed algorithm (BPM) can incorporate a prior in the form of a Gaussian distribution over the parameter defining the outcome distribution. This Gaussian assumption is only used to define the algorithm, while its regret performance is analyzed for any possible outcome distribution. The regret analysis shows that BPM achieves similar performance as existing algorithms, while an empirical study on synthetic and real data shows a significant advantage both computationally and in terms of the regret.Quality==============The paper is of very good quality in general.1- The algorithm is very simple, since it boils down in computing a Gaussian posterior over a linear model and the selection rules are clear and well motivated as well. The only aspect which is not discussed in a rigorous way its the computational cost of the algorithm compared to existing algorithms. In fact, BPM still requires to invert a MxM matrix at each step and the selection rule BPM-LEAST requires a random sampling since its actual exact implementation is computationally very heavy.2- Although the theoretical analysis matches existing results, it is useful in assessing the soundness of BPM (at least in its BPM-LEAST variant). There are only a couple of aspects that I would ask the authors to comment in more detail:a- The regret is studied only in the case of locally observable games. What is the regret in case a non locally observable games? Is it possible to achieve the T^{2/3} regret for "hard" problems? This part is covered by the experiments but a theoretical comment would be useful.b- In the statement of Thm. 1, C is a problem-dependent quantity. It would useful to make explicit the characteristics of the problem contributing to C.c- The Thm. 2 used in the proof is defined for any *fixed* sequence of matrices. In the case of BPM the quantity X_s is random with dependence on the actions selected in the past. It seems like this may pose a problem since the samples are not iid anymore (which would require an Azuma version of the statement) and additional dependencies on the dimensionality of X_s may appear (in the same flavor as in the Lemma inYasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems (NIPS), 2011.It'd be great if the authors could explain whether this is the case or whether Thm.2 is indeed enough.3- In the experiments the authors consider non locally observable problems and show a significant improvement w.r.t. the state of the art with a much smaller regret in all the synthetic data considered in Section 5.2. The other interesting aspect is that a pseudo-real-data experiment is run. I call it "pseudo" since data are real but the experiment is still generated artificially from the data. Nonetheless, I think it is a significant result. The only thing which is really surprising is the difference in performance between the two variants of PBM. What is your explanation for it?Clarity==============The paper is well written and clear.Originality==============The idea used in the paper is not very original since it reuses tools available in the literature. Nonetheless, the result is novel enough.Significance==============Overall the result is significant since it provides an efficient and easy algorithm to solve the challenging problem of partial monitoring in online learning. Preliminary results including real data support the fact that the algorithm may find application in real problems.


Efficient Inference of Continuous Markov Random Fields with Polynomial Potentials
In this paper, we prove that every multivariate polynomial with even degree can be decomposed into a sum of convex and concave polynomials. Motivated by this property, we exploit the concave-convex procedure to perform inference on continuous Markov random fields with polynomial potentials. In particular, we show that the concave-convex decomposition of polynomials can be expressed as a sum-of-squares optimization, which can be efficiently solved via semidefinite programing. We demonstrate the effectiveness of our approach in the context of 3D reconstruction, shape from shading and image denoising, and show that our method significantly outperforms existing techniques in terms of efficiency as well as quality of the retrieved solution.

This is a very interesting and substantially novel paper that introduces an approach to solving continuous Markov random field energies with polynomial potentials.  An insightful and well-motivated approach towards this end (ADMM-Poly) was published at CVPR 2013 [20] and is the obvious baseline to compare against.  The present approach is convincingly shown to be preferable, as it is both elegant and computationally efficient.The main idea underlying the approach is to decompose the polynomials into a difference of convex functions.  Towards this end, a constructive approach is introduced for polynomials of even degree, one of the main contributions of the paper. As the authors show, the decomposition problem has a formulation as a semi-definite program and can be solved using standard solvers.  Given that decomposition, the overall inference task reduces to a DC programming problem, which the authors approach using the convex-concave procedure (CCCP).The main weakness of the approach seems to be that the semi-definite program for finding a DC decomposition of the polynomials may not have a solution, as it is only an approximation to the exact but NP-hard problem.  Even though the authors mention that the approach works well for polynomials of low degree in practice (confirmed by experiments), this point is somewhat worrisome. I would encourage the authors to elaborate on this point, as it seems like a potential barrier to adoption of the approach.  How often does this situation occur in practice (if at all)? For which polynomials? Are there any workarounds?  Clearly, an alternative strategy is needed in case this situation occurs.As the authors show, in some cases, for some continuous energies, a DC decomposition is apparent even without having to resort to the above construction algorithm.  The approach then basically becomes an application of CCCP to continuous energies.Quality:The paper is technically sound, though, as mentioned above, a more in-depth discussion of the potential failure to construct the decomposition of the polynomials would be appreciated.  The experiments are impressively diverse and the results are very convincing.  The proposed approach clearly works better than ADMM-Poly, the main baseline to compare against.  It is faster and often obtains solutions of better quality.Clarity:The paper is well-written, though not always easy to read, owing to the technical depth of the material.  One aspect that the authors may wish to clarify is the claimed global convergence of their approach (inherited from CCCP).  I assume that this is meant to be interpreted as "the algorithm converges, for *any* initialization", rather than "the algorithm converges to a global optimum".  As far as I am aware, the convergence properties of CCCP only guarantee convergence to either an optimum or a saddle point.  As it stands, readers might misinterpret the manuscript to assume that the proposed approach always finds a global optimum.Along the same line, the authors claim in lines 039ff that Gaussian belief propagation "retrieves the optimum for arbitrary graphs when the potentials are Gaussian".  This is not true.  There are two known convergence conditions relating to the diagonal dominance, and the spectral radius, of the system matrix, respectively.Originality:The proposed approach is novel to this reviewer, and substantial theory is developed along the way.Significance:The proposed approach has broad and important applications in computer vision and imaging.  The experimental results are impressive, and the approach hence has the potential to significantly benefit many members of these communities.


A Block-Coordinate Descent Approach for Large-scale Sparse Inverse Covariance Estimation
The sparse inverse covariance estimation problem arises in many statistical applications in machine learning and signal processing. In this problem, the inverse of a covariance matrix of a multivariate normal distribution is estimated, assuming that it is sparse. An 1 regularized log-determinant optimization problem is typically solved to approximate such matrices. Because of memory limitations, most existing algorithms are unable to handle large scale instances of this problem. In this paper we present a new block-coordinate descent approach for solving the problem for large-scale data sets. Our method treats the sought matrix block-by-block using quadratic approximations, and we show that this approach has advantages over existing methods in several aspects. Numerical experiments on both synthetic and real gene expression data demonstrate that our approach outperforms the existing state of the art methods, especially for large-scale problems.

This paper proposes a more efficient, block decomposition, algorithm for reconstruction of large sparse inverse covariance matrices. The paper is written clearly. The experimental results show considerable gains in speed and memory for comparable reconstruction accuracy. I am not an expert in this area so I cannot speak to the work's originality, but as far as I could understand the details, the method appears sound. The experimental results seem solid but not ground-breaking because the problems are still small enough to run on a single machine. It would have been nice to see an extension to the really large problems (>10^6 variables) that one sees in some applications by exploting distributed computation and sharding of the various relevant matrices. One concern for replicating the work is that there seems to be some "magic" in setting up the block decomposition and in the line search. It would have been nice to have a discussion of the setting of those choices.


From Stochastic Mixability to Fast Rates
Empirical risk minimization (ERM) is a fundamental learning rule for statistical learning problems where the data is generated according to some unknown distribution P and returns a hypothesis f chosen from a fixed class F with small loss . In the parametric setting, depending upon ( , F, P) ERM can have slow (1/ √ n) or fast (1/n) rates of convergence of the excess risk as a function of the sample size n. There exist several results that give sufficient conditions for fast rates in terms of joint properties of , F, and P, such as the margin condition and the Bernstein condition. In the non-statistical prediction with expert advice setting, there is an analogous slow and fast rate phenomenon, and it is entirely characterized in terms of the mixability of the loss (there being no role there for F or P). The notion of stochastic mixability builds a bridge between these two models of learning, reducing to classical mixability in a special case. The present paper presents a direct proof of fast rates for ERM in terms of stochastic mixability of ( , F, P), and in so doing provides new insight into the fast-rates phenomenon. The proof exploits an old result of Kemperman on the solution to the general moment problem. We also show a partial converse that suggests a characterization of fast rates for ERM in terms of stochastic mixability is possible.

This paper shows how to obtain fast rates for ERM from a very general assumption known as "stochastic mixability".The paper is clear, well-written, well-motivated, well-referenced, and presents a solid technical contribution.Specific comments:- in Sec. 2.1: the loss is assumed to be Lipschitz, so presumably this would not apply to classification problems with 0-1 loss.It should be made more clear that only "regression"-type learning problems are being considered.Also, F is stipulated as being compact, yet no metric/norm/topology is specified. An expert will infer that the norm in questionis L_1(P) --- which is vindicated by the covering numbers defined in Sec. 4. But the topology should be specified before compactnessis mentioned.- in Sec. 2.4, there is some inconsistency of notation between E Z_f and P Z_f. Either is fine, but please be consistent (or pointout the difference if the two are distinct). Finally, on the issue of bounded losses, which the authors bring up as an assumption to relax in the future, please take a look atA. Kontorovich: Concentration in unbounded metric spaces and algorithmic stability, ICML 2014and the numerous references therein.


Efficient Sampling for Learning Sparse Additive Models in High Dimensions
We consider the problem of learning sparse additive models, i.e., functions of the form:
Here S is an unknown subset of coordinate variables with |S| = k d. Assuming φ l 's to be smooth, we propose a set of points at which to sample f and an efficient randomized algorithm that recovers a uniform approximation to each unknown φ l . We provide a rigorous theoretical analysis of our scheme along with sample complexity bounds. Our algorithm utilizes recent results from compressive sensing theory along with a novel convex quadratic program for recovering robust uniform approximations to univariate functions, from point queries corrupted with arbitrary bounded noise. Lastly we theoretically analyze the impact of noise -either arbitrary but bounded, or stochastic -on the performance of our algorithm.

This paper considers the problem of estimating the components of a high-dimensional function f(x), where x is a vector of dimension d, which follows a sparse additive model, i.e. f(x) can be written as the sum of a small number of functions f_i(x_i). The authors propose sampling this function at regular points on a certain straight line and additionally on random vertices of a small hypercube surrounding these points. A convex program is then used to obtain an approximation of the gradient of f, exploiting the assumption that it is sparse. 4th order splines are employed to estimate the components of f from this approximation. The authors provide theoretical analysis showing that the method is guaranteed to detect the correct components of f in the absence of noise and also that the procedure is robust to noise.The main contribution of this paper is an innovative approach to perform robust estimation of gradients in high dimensions through randomized sampling. Although the theoretical analysis follows from standard compressed-sensing results, the idea is clever and the connection to regression with sparse additive models is intriguing. The second step of the algorithm, where the gradient estimates are used to estimate the individual components is less innovative. The authors also provide some useful analysis of the stability of the algorithm, which provides insight into the relationship between the grid size, the curvature of the components and robustness to noise (unfortunately, no numerical simulations are provided to support the theoretical conclusions).The following are the weakest aspects of the paper:-The authors do not exploit the group-sparse structure that is implicit in their model. They propose solving a large number of sparse regression problems that have the same sparsity pattern. This suggests two ways of improving the algorithm in terms of sampling complexity and robustness to noise:1. Estimate the sparsity pattern from a few samples. Then just estimate the derivative of the nonzero components by using a finite-difference approximation in their direction.2. Employ a regularizer that promotes group sparsity while estimating the gradients, e.g. an l1-l2 norm penalty.-The data, i.e. the samples of the function of f, are only used to approximate the gradients of the components f_i. They are not taken into account when the component functions f_i are constructed from the gradient estimate. It would probably make sense to ensure that the sum of the estimated components at each sample is consistent with the data up to the noise level. -The authors do not motivate sufficiently why the model would be of interest in practical applications. A concise example of an application would go a long way to clarify this.-The numerical simulations are extremely limited. In particular, no attempt is made to illustrate the limits of the method or how the method’s performance depends on the different parameters.Quality:The paper makes an interesting algorithmic contribution and is technically sound, although there are some caveats that I have mentioned above. The numerical simulations are insufficient.Clarity:The paper is written clearly and is easy to read. Originality:The idea of approximating a sparse gradient in high dimensions by sampling a function in random directions within a neighborhood is new to the best of my knowledge. However, recovering a function from its samples by penalizing the norm of its second derivative is not as innovative.Significance:This paper is significant from a theoretical point of view in that it provides a method for estimating gradients of a function in a neighbourhood by randomized sampling and convex programming. I am hesitant to state that this is very significant because the authors do not describe any relevant applications and because of the lack of numerical experiments.Other comments:The recovery of the components from their gradients must necessarily depend on the second-order behaviour of the components with respect to the chosen grid. A bound on the second derivative does appear in the error bounds, but the authors do not elaborate on its significance.After the author's rebuttal my views on the paper have not changed. Just to be clear, according to the authors' description of the model the sparsity pattern of the sparse recovery problems they solve is fixed and determined by the set S.


On a Theory of Nonparametric Pairwise Similarity for Clustering: Connecting Clustering to Classification
Pairwise clustering methods partition the data space into clusters by the pairwise similarity between data points. The success of pairwise clustering largely depends on the pairwise similarity function defined over the data points, where kernel similarity is broadly used. In this paper, we present a novel pairwise clustering framework by bridging the gap between clustering and multi-class classification. This pairwise clustering framework learns an unsupervised nonparametric classifier from each data partition, and search for the optimal partition of the data by minimizing the generalization error of the learned classifiers associated with the data partitions. We consider two nonparametric classifiers in this framework, i.e. the nearest neighbor classifier and the plug-in classifier. Modeling the underlying data distribution by nonparametric kernel density estimation, the generalization error bounds for both unsupervised nonparametric classifiers are the sum of nonparametric pairwise similarity terms between the data points for the purpose of clustering. Under uniform distribution, the nonparametric similarity terms induced by both unsupervised classifiers exhibit a well known form of kernel similarity. We also prove that the generalization error bound for the unsupervised plugin classifier is asymptotically equal to the weighted volume of cluster boundary [1] for Low Density Separation, a widely used criteria for semi-supervised learning and clustering. Based on the derived nonparametric pairwise similarity using the plug-in classifier, we propose a new nonparametric exemplar-based clustering method with enhanced discriminative capability, whose superiority is evidenced by the experimental results.

This paper presents clustering framework using pairwise similarity function model.Different data partitions are used to construct different nonparametric classifiers.The optimal clustering is obtained by minimizing the generalization error of the learned classifiers related to data partitions.The authors consider two nonparametric classifiers in their framework:the plug-in and the nearest-neighbor classifier. The authors prove also a result interesting in itself, namely that the generalizationerror bound for the unsupervised plug-in clustering is asymptotically  equal to theweighted volume of cluster boundary for Low Density Separation. This paper tights together two important machine learning subjects: multi-class classification problem and clustering. Furthermore, it has strong theoretical background.The paper is however very poorely written.The introduction is well-organized however in the next sections the authorsgo into several technical details and do not summarize their results in the compact understandable form. Two technical lemmas given in Section 2 only deepenthe confusion. Thus, it is extremely difficult to measure the real impact of this paper.Section 2 contains too many technical details that are given before giving the reader any sort of intuition. Besides the presented lemma statements are completely unclear. For instance:Lemma 1: - is it possible to say how n_{0} depends on \sigma_{0} and VC characteristics of K ? - the statements seems not to be put in the grammatical form, the authors should review itLemma 2: - again the authors introduce many parameters that mutually depend on each other,   it is almost impossible to guess from all the asusmptions required by the lemma what it    says in factLemma 3: - similar problems to those mentioned aboveLemma statements are unreadable and some of them are put at the very beginning of the paper.The authors should spend much more time to work on the presentation of their results.Maybe some summary of the obtained theoretical results should be conducted before giving exact statements.The authors should also give some intuition regarding all lemmas presented in the main body of the paper.The authors present how their techniques may be applied to the exemplar-based clustering but the comparisonwith existing state-of-the-art methods is missing. The experimental section is very tiny and does not shed any light of advantages of the technique developed by the authors over other clustering algorithms.The difficulty of the clustering heavily depends on the considered setting: whether data is truly high-dimensional,or maybe lies on the small-dimensional manifolds, whether data points are sparse or dense, whether the groundtruth clusters are convex or there are noassumptions about their shape, what is the objective function the authors aim to minimize or the groundtruthclustering they want to approximate well. Without detailed analysis of the performance of the algorithm in the specificsetting defined by these parameters it is almost impossible to say whether the presented method can be applied in practice and solves a nontrivial problem.What is the computational complexity of the algorithms that use presented technique ? This issue is not addressed at all.Is it possible to extend the analysis for other classifiers (only nearest-neighbor and plug-in classifiers were analyzed) ? Is it possible to release some source code files (with the implementation of the presented method) that were used by the authors to test their approach ? (The last remark is not crucial though.)


Graph Clustering With Missing Data : Convex Algorithms and Analysis
We consider the problem of finding clusters in an unweighted graph, when the graph is partially observed. We analyze two programs, one which works for dense graphs and one which works for both sparse and dense graphs, but requires some a priori knowledge of the total cluster size, that are based on the convex optimization approach for low-rank matrix recovery using nuclear norm minimization. For the commonly used Stochastic Block Model, we obtain explicit bounds on the parameters of the problem (size and sparsity of clusters, the amount of observed data) and the regularization parameter characterize the success and failure of the programs. We corroborate our theoretical findings through extensive simulations. We also run our algorithm on a real data set obtained from crowdsourcing an image classification task on the Amazon Mechanical Turk, and observe significant performance improvement over traditional methods such as k-means.

The paper analyzes two convex programs for graph clustering with missing data under the stochastic block model. Success conditions are provided for both programs, and failure conditions are also given for the first one. Understanding the exact performance boundary of these programs is an interesting and relevant problem. However, as mentioned by the authors, there are several existing work on the analysis of similar convex programs. It is not entirely clear how the results in this paper differentiate from existing ones, and the paper lacks a detailed comparison. For example:(1) The work of [16] also provides success conditions for the second program. In section 3.1 the authors claim that [16] has considerably weaker bounds, but it is not clear how this is the case. An explicit comparison is needed.(2) The work of [21] also gives success and failure conditions for the first program. Although it only considers the full observation case, the techniques seem similar.(3) The success and failure conditions for the first program (in THM 1) do not complement each other, even if one ignores the difference between Lambda_succ and Lambda_fail. For example, what happens when D_min < 1/Lambda_succ? Does the program fail for all choices of lambda?(4) The results seem even weaker than existing bounds in certain regimes. Suppose n_min = n/2 and r=1/2. The success condition for the second program requires, among other things, p > 1/sqrt(n); see eq.(2.4). However, if we treat missing entries as 0 and apply standard results for spectral clustering (e.g. McSherry), then we only need p > O(log(n)/n), which is order-wise better.In terms of writing, the statements of the main theorems are quite technical and complicated, and it is difficult to interpret them. It would be better if more discussion is given.


Accelerated Mini-batch Randomized Block Coordinate Descent Method
We consider regularized empirical risk minimization problems. In particular, we minimize the sum of a smooth empirical risk function and a nonsmooth regularization function. When the regularization function is block separable, we can solve the minimization problems in a randomized block coordinate descent (RBCD) manner. Existing RBCD methods usually decrease the objective value by exploiting the partial gradient of a randomly selected block of coordinates in each iteration. Thus they need all data to be accessible so that the partial gradient of the block gradient can be exactly obtained. However, such a "batch" setting may be computationally expensive in practice. In this paper, we propose a mini-batch randomized block coordinate descent (MRBCD) method, which estimates the partial gradient of the selected block based on a mini-batch of randomly sampled data in each iteration. We further accelerate the MRBCD method by exploiting the semi-stochastic optimization scheme, which effectively reduces the variance of the partial gradient estimators. Theoretically, we show that for strongly convex functions, the MRBCD method attains lower overall iteration complexity than existing RBCD methods. As an application, we further trim the MRBCD method to solve the regularized sparse learning problems. Our numerical experiments shows that the MRBCD method naturally exploits the sparsity structure and achieves better computational performance than existing methods.

This paper explores the integration of mini-batch, randomized block CCD, and variance reduction for regularized empirical risk minimization problem. The major two problems the author target is variance reduction and the incremental proximal point problems when dealing with empirical and expected risk function. This method aims to show advantage over a recently proposed method, SPVRG.Quality:The experiments seem not enough. There are a couple of recently proposed methods aiming the same problem the paper cited, such as semi-SG [6], SVRG [5] and (SAG) [14]. The reviewer would like to see the comparison results of these methods. The paper compared with BPG and BRBCD, which are batch methods and thus the comparison is not fair enough. Besides the lack of comparisons, the current experimental results are not so exciting, compared with the recently proposed SPVRG algorithm which also aiming at shooting the variance reduction problem, the proposed MRBCD-II shows visible yet minor improvement on two moderate-size domains. Experiments on larger domains, if applicable, is preferred.Clarity:Some details regarding the paper：The paper is not quite well written, there are some typos which can be easily found, e.g.,line 160 'shows how to reduce shows how to reduce'line 217 'full take' should be 'fully take'line 193-194, 'These adjustments can guarantee ... goes to zero'. Please give proof on this.In experiment 1, from  line 340-344, different stepsizes are chosen according to different approaches. The reviewer wants to know how they are chosen and if they are optimal stepsize w.r.t the method to make a fair comparison.The paper goes without a conclusion part, which is fine, yet it is better to be with a conclusion part. Originality:The contribution seems incremental.Significance:The paper could be improved given more detailed analysis along with comprehensive comparison experimental study and result analysis.


Optimal Neural Codes for Control and Estimation
Agents acting in the natural world aim at selecting appropriate actions based on noisy and partial sensory observations. Many behaviors leading to decision making and action selection in a closed loop setting are naturally phrased within a control theoretic framework. Within the framework of optimal Control Theory, one is usually given a cost function which is minimized by selecting a control law based on the observations. While in standard control settings the sensors are assumed fixed, biological systems often gain from the extra flexibility of optimizing the sensors themselves. However, this sensory adaptation is geared towards control rather than perception, as is often assumed. In this work we show that sensory adaptation for control differs from sensory adaptation for perception, even for simple control setups. This implies, consistently with recent experimental results, that when studying sensory adaptation, it is essential to account for the task being performed.
to the control task), followed by constructing a controller based on the estimated state. This idea, referred to as the separation principle in Control Theory, while optimal in certain restricted settings (e.g., Linear Quadratic Gaussian (LQG) control) is, in general, sub-optimal [6]. Unfortunately, it is in general very difficult to provide optimal solutions in cases where separation fails. A special case of the separation principle, referred to as Certainty Equivalence (CE), occurs when the controller treats the estimated state as the true state, and forms a controller assuming full state information. It is generally overlooked, however, that although the optimal control policy does not depend directly on the observation model at hand, the expected future costs do depend on the specifics of that model [7]. In this sense, even when CE holds, costs still arise from uncertain estimates of the state and one can optimise the sensory observation model to minimise these costs, leading to sensory adaptation. At first glance, it might seem that the observation model that will minimise the expected future cost will be the observation model that minimises the estimation error. We will show, however, that this is not generally the case.
A great deal of the work in computational neuroscience has dealt independently with the problem of sensory adaptation and control, while, as stated above, these two issues are part and parcel of the same problem. In fact, it is becoming increasingly clear that biological sensory adaptation is task-dependent [8,9]. For example, [9] demonstrates that task-dependent sensory adaptation takes place in purely motor tasks, explaining after-effect phenomena seen in experiments. In [10], the authors show that specific changes occur in sensory regions, implying sensory plasticity in motor learning. In this work we consider a simple setting for control based on spike time sensory coding, and study the optimal coding of sensory information required in order to perform a well-defined motor task. We show that even if CE holds, the optimal encoder strategy, minimising the control cost, differs from the optimal encoder required for state estimation. This result demonstrates, consistently with experiments, that neural encoding must be tailored to the task at hand. In other words, when analyzing sensory neural data, one must pay careful care to the task being performed. Interestingly, work within the distributed control community dealing with optimal assignment and selection of sensors, leads to similar conclusions and to specific schemes for sensory adaptation.
The interplay between information theory and optimal control is a central pillar of modern control theory, and we believe it must be accounted for in the computational neuroscience community. Though statistical estimation theory has become central in neural coding issues, often through the Cramér-Rao bound, there have been few studies bridging the gap between partially observed control and neural coding. We hope to narrow this gap by presenting a simple example where control and estimation yield different conclusions. The remainder of the paper is organised as follows: In section 1.1 we introduce the notation and concepts; In section 2 we derive expressions for the cost-to-go of a linear-quadratic control system observed through spikes from a dense populations of neurons; in section 3 we present the results and compare optimal codes for control and estimation with point-process filtering, Kalman filtering and LQG control; in section 4 we discuss the results and their implications.

Summary-------The authors study stochastic optimal control problems with incomplete state information. In particular, they consider problems where the sensors are adaptive. They compare sensor configurations which are optimal under a standard signal detection paradigm with sensor configurations that are optimal for a given control problem.Comments--------Optimal sensor adaptation has a long history in computational neuroscience. Usually, optimality is defined wrt estimation (infomax etc). The main point of the paper, that optimal sensor adaptation for control problems will yield different results in general, is very important and could potentially have a big impact on the field. Standard optimal coding approaches were often criticized but concrete alternatives were lacking.Here the authors propose optimal control as a guiding principle and illustrate this with a few well-chosen examples. Not being an expert in control theory, the paper seems technically sound (I checked most equations, not all).It is well written, if a little too dense at times. Some results / examples require further elaboration (details below), it might be worth dropping one (eg the harmonic oscillator in 3.1) in favor of more details for the other examples.Specific comments:- l22: ``However, this sensory adaptation is geared towards control rather than perception, as is often assumed.'' In principle I agree with this statement. However, there's an interesting discussion to be had based on the following argument: Assume that the sensors are used in many different control problems, and therefore adapting them for a single one might result in poor performance on the others. In this situation it might be beneficial to adapt the sensory based on optimal estimation after all. I'd be curious to hear the opinion of the authors on this point.- l103: ``MMSE'' is probably the minimum MSE, but it's never defined. Furthermore, the authors seem to use MSE / MMSE inconsistently, eg in the legend of figs 3&4, shouldn't it be just MSE instead of MMSE?- l110: Shouldn't the minimization be over $\mathbf U_0$ instead of $\mathbf U_t$? And similarly the expectation over $\mathbf X_0$, etc...- l117: The notation here is a bit sloppy I think. On the RHS of the eqn, it prob should be $\gamma$ applied to the time-series of filtered estimates of $X_t$ given $Z_{[0,t]}$. By definition in l108 $\mathbf Z_t$ is the future of $Z$, and not the past...- l146: typo: $E[X_t\vert Y_t]$ should prob be $E[X_t\vert Y_{[0,t]}$- l158, eqn 3: the authors state that only the third term depends on the parameters $F,G$ of the observation process $Y$. For $t > 0$, why don't the first two terms depend on $F,G$ via $\nu_t$ and $K_t$? For $t=0$, I agree that only the 3rd term depends on $F,G$.- l182, eqn 5: Concerning the dense coding hypothesis: I was wondering weather it can be achieved with a finite number of neurons each having an RBF / squared-exponential receptive field (my intuition tells might not, but I might be wrong). If one indeed needs infinitely many neurons, than one could possibly replace the spike process $dN_t$ with a diffusion / Wiener process and hence simplify eqns 6a and 6b. Is this possible? The authors might be interested in the use of the dense packing hypothesis in [Implicit embedding of prior probabilities in optimally efficient neural populations. D Ganguli and E P Simoncelli. ArXiv]- l258: typo, this should probably be the mutual information between $X_t$ and $\{N^m_{[0,t]}\}_{m=1,\ldots,M}$- l270: ``This trade-off can be tilted to either side in the case of control depending on the information available at the start of the problem.'' I don't fully understand this point. Furthermore, I cant find the promised detailed discussion of this point in section 3.1.- fig 2: The left and right panels use different labels in the legends for the same quantities. Left panel: the x-axis label should probably be also $\zeta$- l413: ``Note the extreme example of the stochastic oscillator, where the optimal encoder is concentrating all the resources in one dimension, essentially ignoring the second dimension.'' It's important to spell out in detail what this means for the tuning curve width in direction and perpendicular to the ``important control dimension''


"How hard is my MDP?" The distribution-norm to the rescue
In Reinforcement Learning (RL), state-of-the-art algorithms require a large number of samples per state-action pair to estimate the transition kernel p. In many problems, a good approximation of p is not needed. For instance, if from one state-action pair (s, a), one can only transit to states with the same value, learning p(·|s, a) accurately is irrelevant (only its support matters). This paper aims at capturing such behavior by defining a novel hardness measure for Markov Decision Processes (MDPs) based on what we call the distribution-norm. The distributionnorm w.r.t. a measure ν is defined on zero ν-mean functions f by the standard variation of f with respect to ν. We first provide a concentration inequality for the dual of the distribution-norm. This allows us to replace the problem-free, loose || · || 1 concentration inequalities used in most previous analysis of RL algorithms, with a tighter problem-dependent hardness measure. We then show that several common RL benchmarks have low hardness when measured using the new norm. The distribution-norm captures finer properties than the number of states or the diameter and can be used to assess the difficulty of MDPs.

Note to authors: As I stated in my original review, though I think it couldbe improved upon (and what paper couldn't?), I believe this paper should be accepted. My apologies for not making that more clear. I have edited my review below in an attempt to clarify some of my comments that were perhaps not stated carefully enough originally. ---Summary of Paper---The paper presents a novel measure of "hardness" for MDPs (based on anovel concentration inequality) which essentially measures thestochasticity of the *value* of the next state. Implications forperformance bounds for RL algorithms are sketched out. Empirically thenew measure is demonstrated to be tighter than previously knownindicators of MDP hardness.---Quality---The paper appears to be technically sound. The main results aretheoretical in nature and are well-supported. Due to space constraints, Section 3.1 comes across as rushed andincomplete. I understand the authors' excitement about the potentially broad implications of their result, but unfortunately there is simplynot enough room to fully roll those implications out. As a result,the paper currently presents some key implications (the newsimulation lemma, the modified UCRL bound), but leaves a lot offollow through as exercise for the reader (see lines 288-292, 311-314, 315-317, 338-341). I certainly couldn't expect the authorsto fill in all those details in the available space, and I agree that someof what they hint at is straightforward given sufficiently intimatefamiliarity with the relevant literature. Nevertheless, in my opinion,the presentation of this section leaves the reader feeling unresolved,and, depending on how much their intuition overlaps with the authors,perhaps unsure of whether the various paths the authors point to willreally be as smooth as they suggest. I wonder if the paper might havebeen better served by thoroughly and concretely rolling out oneimpactful consequence with hints in other directions, rather than thesample platter of incomplete offshoots that is presented here.The empirical results are helpful and evocative, and do support thecase that the new measure is tighter than other known measures. On theother hand, they stop short of demonstrating a more concreteconnection between the bounds given and learningperformance. A more thorough evaluation might have measured if the distinctions made by this quantity (some examples have notably higher C than others) are reflected in practice, or included some examples thought to be more difficult to show that the measure responds accordingly. Also a variation on UCRL is presented with an improved regret bound due tothe result in this paper, but whether this also results in betterperformance in practice is not investigated. The authors rightly pointout in their response that they make no claim to improve practicalperformance, and I suppose therefore they are not beholden to demonstratepractical impact. Nevertheless, this is the question we ultimately careabout, is it not? So it seems like it would have been worthwhile toinvestigate/report, regardless of the outcome.---Clarity---I found the paper to be by and large clear, readable, andwell-written. I appreciated the authors' efforts to continuallyconnect technical statements/results with high level ideas and goals.Continuing on my comments above, I will say that I feel the clarity ofSection 3 is harmed by the sheer number of points to be made. Spaceconstraints being what they are, this section moves at a rapid clipand, as mentioned, leaves a lot as "exercise for the reader." I fear thatreaders who are not as intimately familiar with the surrounding theoreticalcontext as the authors may not so readily leap to the conclusions thatthe authors intend to communicate.A couple of minor suggestions:- On lines 19-21 the abstract contains a very nice intuitive example  of when the new bound can be much tighter. That example does appear  in the main text, but not until page 5, and even then it is  expressed in more technical terms. I'd recommend adding this  intuition to the introduction as well. I think it is useful for the  reader to have it in their head as they work through the results,  and the abstract should not be relied upon for the flow of the  paper.- I'd humbly recommend that the authors give their measure a  name. Just writing this review it became clear how cumbersome it is  to discuss a nameless quantity. It may also have a positive impact  on readers' retention/understanding to have something more compact  and more evocative way to refer to this thing than C^\pi_M.---Originality---As far as I know the main results are novel and I agree with theauthors that their bound likely represents a new level of granularityin formally characterizing the "hardness" of learning MDPs. The workfeels sufficiently contextualized in the relevant literature.---Significance---I think there is clear theoretical/scientific significance to theresult. The authors are correct to point out the often wide gapbetween theoretical guarantees and practical experience. If we areever to gain a formal understanding of the phenomena we actuallyobserve (and if theoretical guarantees are ever going to be meaningfulto practitioners) results like this one, that examine problems at afar finer granularity, will be required. It's not clear that thisresult closes this gap, but I think it is compellingly demonstratedthat it takes a step in the right direction.At the moment the practical significance of the result is unclear. Thepaper does show that modifications to algorithms may be suggested bythe result, but it was not investigated whether the modification hadan impact in practice. I do agree with the authors that there is very real *potential* forthis result to have wide-reaching consequences both for the analysisof existing algorithms and for the development of new ones. Though insome ways I feel the paper could benefit from being retooled, in the end, I think that the potential for significant follow-up results overrides my concerns about presentation.


Structure Regularization for Structured Prediction
While there are many studies on weight regularization, the study on structure regularization is rare. Many existing systems on structured prediction focus on increasing the level of structural dependencies within the model. However, this trend could have been misdirected, because our study suggests that complex structures are actually harmful to generalization ability in structured prediction. To control structure-based overfitting, we propose a structure regularization framework via structure decomposition, which decomposes training samples into mini-samples with simpler structures, deriving a model with better generalization power. We show both theoretically and empirically that structure regularization can effectively control overfitting risk and lead to better accuracy. As a by-product, the proposed method can also substantially accelerate the training speed. The method and the theoretical results can apply to general graphical models with arbitrary structures. Experiments on well-known tasks demonstrate that our method can easily beat the benchmark systems on those highly-competitive tasks, achieving record-breaking accuracies yet with substantially faster training speed.

Summary: The authors propose “tag structure regularization”, a novel regularization scheme whereby at training time, a structured prediction model is penalized based on losses on sub-structures of the training example. The key is that the model is forced to make predictions for each sub-structure independently, so it cannot rely on long-reaching information from other parts of the graph. They provide analysis based on stability arguments that the regularization strength decreases bounds on generalization error and increases the convergence rate of SGD. On several sequential prediction tasks they show that varying the regularization strength can produce state-of-the-art accuracies on several tasks and at the same time speed up training.Major comments: (+ Pros, - Cons)+ On the one hand, the central idea in this paper is very interesting. The authors argue that by breaking up training samples into sub-samples during learning, we can increase the generalization of structured models. Intuitively, this makes a certain kind of sense: the structured model is regularized to rely more on local information than incoming messages from the rest of the graph. This might prevent errors from propagating, and results in a “simpler” model. Another way to think about it is that if a model can make accurate predictions without relying on passing messages, the “tag structure regularization” will choose that simpler model, while more standard approaches do not have such preference.- On the other hand, the actual theoretical analysis in this paper seems like it isn’t making the correct assumptions or taking the right approach to analysis. Yes, increasing alpha does reduce the bound -- but at the tightest setting alpha = n, the bound is still far looser than other structured prediction generalization bounds. E.g. the original MMMN paper (Taskar et al.) had a logarithmic complexity in both the multi class label size and the number of variables (l in that paper, n here). More recently London et al. used PAC Bayes analysis to show that for *templated* models, increasing the size of the example actually **decreases** generalization error: in the limit, one could learn an entire templated model from a single example. So, from that perspective, using n as the measure of structure complexity makes no sense, since in most applications (including those in this paper) feature templates are used (bi-grams, etc.). So while I believe that their analysis is technically correct, the authors must reconcile their analysis with previous work in order for this paper to make sense.- n^4 seems like an awfully large term for a generalization bound: following the supplemental, it seems like it stems from the fact that the bounds rely on decomposing loss linearly in Lemma 6, and then due to having to multiple the norms of 2 examples (which could be O(n)) in addition to that, as well as decomposing the regularization term linearly (O(n/alpha)). To me, that suggests that this is really not the right approach to take here.- The aforementioned previous work uses a more subtle measure of graph complexity in their bounds, based on concentration inequalities, that measures the maximum dependence of one variable on the others in the graph. It seems like a better approach to analysis would be to relate the novel regularization to the resulting complexity of the learned model in terms of a complexity measure like that, where one can assume some sort of templating. Instead of just assuming complexity = n.+ I hate to say this (because this idea is so trendy), but the tag structure regularization reminds me a lot of dropout: essentially, for each example, you generate new examples by removing edges in the graph. In the case of sequences, this creates disconnected components, but one could imagine more generally just removing edges. So in that sense I think this paper does help shed light on other ideas in the field.+ All that being said about the theory, the experimental results are very strong, and the idea is simple enough to be easy to experiment with and verify.


A Framework for Testing Identifiability of Bayesian Models of Perception
Bayesian observer models are very effective in describing human performance in perceptual tasks, so much so that they are trusted to faithfully recover hidden mental representations of priors, likelihoods, or loss functions from the data. However, the intrinsic degeneracy of the Bayesian framework, as multiple combinations of elements can yield empirically indistinguishable results, prompts the question of model identifiability. We propose a novel framework for a systematic testing of the identifiability of a significant class of Bayesian observer models, with practical applications for improving experimental design. We examine the theoretical identifiability of the inferred internal representations in two case studies. First, we show which experimental designs work better to remove the underlying degeneracy in a time interval estimation task. Second, we find that the reconstructed representations in a speed perception task under a slow-speed prior are fairly robust.

In this article, the authors propose a framework for performing model comparison of Bayesian models on behavioral data. To do so, they summarize the Bayesian Decision Theory framework, pinpoint areas of non-identifiability, and outline the types of constraints that can be used to make each term in the Bayesian framework identifiable. They then make assumptions to constrain each term in the Bayesian framework, explore how differentiable parameter values are in their model, and apply the technique to two studies that use Bayesian decision theory to explain behavioral responses: time interval estimation and motion perception. Issues of identifiability of internal representations and processes have been prominent issues within cognitive science and psychology for decades. For example, Anderson (1978) analyzed when representational formats can mimic each other, proving that they are indistinguishable without some constraints on processes (whenever the representations make equivalent distinctions of their inputs).In recent years, it is one of the major drawbacks of the Bayesian framework that has hindered its widespread adoption (e.g., Jones & Love, 2011; Bowers & Davis, 2012 – the latter directly addresses speed perception as an example). Although the article touches on an issue of great theoretical importance, is well written, and generally does so in a reasonable and effective manner, I have two major reservations with it. First, it ignores the large previous literature on model comparison within cognitive science and psychology, which makes it difficult to evaluate its contribution appropriately. The general framework for model comparison proposed by the authors is not so different from some of this previous work (e.g., Pitt, Myung, & Zhang, 2002; Navarro, Pitt, & Myung, 2004). Second, the authors make assumptions that seem arbitrary at times and it is unclear how much of their framework depends on them. For example, why should $p_{meas}$ be defined by equation 3? Is defining the values of the parameters in Equation 12 really enough to define and capture all the characteristics of a cognitive or perceptual process? I understand that it is robust and can express a large class of sensory models. However, it is not clear that is a sufficient justification for the assumption and at the very least, the article should discuss and justify the assumptions used to constrain the Bayesian models so that they are identifiable. Minor comments:Line 53: “It is a trivial mathematical fact…” It might be more helpful to explain why it is true rather than declare it trivial. It might not be trivial for all readers.Line 73: I believe the second “nor” should be an “or”.Figure 2: What do the star and dashed lines denote? From the caption, it sounds like they both denote the true value. ReferencesAnderson, J. R. (1978). Arguments concerning representations for mental imagery. Psychological Review, 85, 249-277.Bowers, J. S., & Davis, C. J. (2012). Bayesian Just-So Stories in Psychology and Neuroscience. Psychological Bulletin, 138(3), 389-414.Jones, M., & Love, B. C. (2011). Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34, 169-188.Navarro, D. J., Pitt, M. A., & Myung, I. J. (2004). Assessing the distinguishability of models and the informativeness of data. Cognitive Psychology, 49, 47-84.Pitt, M. A., Myung, I. J., & Zhang, S. (2002). Toward a Method of Selecting Among Computational Models of Cognition. Psychological Review, 109(3), 472-491. (also see Myung, Pitt, Zhang, & Balasubramanian, 2000 NIPS 13).


Learning a Concept Hierarchy from Multi-labeled Documents
While topic models can discover patterns of word usage in large corpora, it is difficult to meld this unsupervised structure with noisy, human-provided labels, especially when the label space is large. In this paper, we present a model-Label to Hierarchy (L2H)-that can induce a hierarchy of user-generated labels and the topics associated with those labels from a set of multi-labeled documents. The model is robust enough to account for missing labels from untrained, disparate annotators and provide an interpretable summary of an otherwise unwieldy label set. We show empirically the effectiveness of L2H in predicting held-out words and labels for unseen documents.

Given a corpus of multi-labeled documents, the author(s) propose a topic model which associates each label with a topic, and recovers a tree-structured hierarchical taxonomy of the topics/labels.  The model is evaluated on a political science data set.  As well as finding such interpretable structure, as seen in qualitative results in the paper, the model is reported to improve perplexity over LDA and labeled LDA, and also provides features which outperform TF-IDF and labeled LDA at multi-label document classification.Using label information to create a hierarchy of topics and labels is a nice idea, bringing together separate strands of work on hierarchies of topics and hierarchies of labels.  The proposed model is elegant and is a principled approach to the problem.  An approximation must be made in the Gibbs sampling procedure (Equation 7), though it is unclear how accurate the approximation is.  Although the procedure of label selection using TF-IDF when performing document classification seems somewhat ad-hoc, to be fair it is a reasonable approach, and the strategy has been used before by Ramage et al.However, unfortunately, the procedure used to estimate the perplexity scores for Equation 9 is incorrect.  In the following, for simplicity of exposition, I will describe the issue for standard LDA instead of for the proposed model, ignoring the variables which are specific to this model such as the labels. The notation here also explicitly conditions on \alpha.In the manuscript, for each MCMC sample of the topics, the procedure seems to be to draw Z from its posterior given each test document.  This procedure fits Z to the *test* data.  It would therefore over-estimate the likelihood of data which are truly unobserved, and would reward over-fitting of Z.In more detail, the procedure in the manuscript estimates \sum_z P(W|Z, \phi, \alpha) P(Z|W, \phi, \alpha).  However, what should be estimated for Equation 9 is, for each sample of \phi, the likelihood for the test document, P(W|\phi, \alpha) = \sum_z P(W,Z|\phi, \alpha) = \sum_z P(W|Z, \phi) P(Z|\alpha).  In other words, the present procedure computes an average over the *posterior*.  However, the likelihood is equal to an average over the *prior*.To correct this, an unbiased estimate for the likelihood, to plug into Equation 9, can be computed using these samples from the posterior, by instead taking the harmonic mean instead of the arithmetic mean (Equation 15 in Wallach et al. 2009).  However, Wallach et al. found that this method has extremely high variance, and hence is not recommended.  They suggest alternative algorithms such as the left-to-right method and annealed importance sampling.Another sensible alternative strategy is to repeat the exact procedure used in the manuscript, except first splitting each test document into two portions, "test_train" and "test_test."  The procedure in the paper (sample Z from the posterior given the data and topics) is performed on "test_train," and the Rao-Blackwellized estimates of theta from the resulting Z's are used to evaluate perplexity on "test_test," thus computing P(test_test|test_train, \phi).  This is the "estimated theta document completion" strategy, described in Section 5.1 of Wallach et al. (2009).  For more information on evaluating topic models, see:Wallach, H. M., Murray, I., Salakhutdinov, R., & Mimno, D. (2009). "Evaluation methods for topic models." Proceedings of the 26th Annual International Conference on Machine Learning.Going forward, it would be good if the author(s) could comment on the semantics of the extracted hierarchy.  Clearly the model does not always place strictly more specific topics/labels below more general topics/labels, although the generative model encourages this somewhat.  So if the tree structure does not always encode generality relationships, how should we interpret it?   It would also be good to present results on more than one data set to give more evidence that the results hold generally.*** Thanks to the authors for addressing my concern regarding the evaluation procedure.  I have adjusted my score accordingly.


A Drifting-Games Analysis for Online Learning and Applications to Boosting
We provide a general mechanism to design online learning algorithms based on a minimax analysis within a drifting-games framework. Different online learning settings (Hedge, multi-armed bandit problems and online convex optimization) are studied by converting into various kinds of drifting games. The original minimax analysis for drifting games is then used and generalized by applying a series of relaxations, starting from choosing a convex surrogate of the 0-1 loss function. With different choices of surrogates, we not only recover existing algorithms, but also propose new algorithms that are totally parameter-free and enjoy other useful properties. Moreover, our drifting-games framework naturally allows us to study high probability bounds without resorting to any concentration results, and also a generalized notion of regret that measures how good the algorithm is compared to all but the top small fraction of candidates. Finally, we translate our new Hedge algorithm into a new adaptive boosting algorithm that is computationally faster as shown in experiments, since it ignores a large number of examples on each round.

This paper proposes a new scheme for constructing and analyzing algorithms for online learning based on the equivalence of a broad class of online optimization games and the class of so-called drifting games. This newly found equivalence not only allows the authors to provide an alternative derivation for the well-known exponential-weights algorithm for prediction with expert advice and online convex optimization, but also leads to a new parameter-free learning algorithm called AbnormalHedge (a variant of NormalHedge originally proposed by Chaudhuri, Freund and Hsu, 2009). The authors also use their technique to prove strong guarantees on the so-called \epsilon-regret of the resulting algorithms, defined as the regret against the best \epsilon-quantile of the pool of experts/decision space. Finally, the authors apply AbnormalHedge in a boosting setting and highlight several useful properties of the resulting boosting algorithm.Originality & Significance--------------------------The results of the paper clearly constitute a substantial step forward in understanding potential-based algorithms for online learning. While this is not the first work to draw connections between drifting games and online learning, the general recipe of constructing near-optimal online learning algorithms by relaxing the minimax optimal strategy is novel. These results tie in nicely with the recent work of Rakhlin, Shamir and Sridharan (2012), who proposed a recipe for constructing online learning algorithms that work by relaxing the minimax strategy for the online learning game itself. The power of the proposed approach is clearly demonstrated by the simple and straightforward analysis provided for AbnormalHedge, which is especially elegant when compared to the complicated original analysis of NormalHedge. To the best of my knowledge, this is also the first time that the continuous exponential-weights algorithm is derived via a (relaxed) minimax analysis. The extension of the results to the multi-armed bandit setting -- while providing no spectacular new results -- can also serve as basis for interesting future work.Clarity & Quality-----------------The writing is superb, there is really not much to fix except a handful of typos (see below). The technical quality is also very good for the most part, the derivations are clean and seem correct. The only exception is Section 3.4, where the authors claim that their approach can yield high-probability regret guarantees without relying on concentration-of-measure results. Although I can see how this result follows from a modification of Theorem 2, it would be probably better to provide this proof at least in the appendix for completeness.Detailed comments-----------------212: \Phi -> \Phi_t in the expression of the optimal value of w. Same typo on line 229.212: "0-1 loss function" -> "the 0-1 loss function"234-235: This discussion is a bit difficult to follow. How does one "solve \Phi_0(0)<\epsilon" and how does such a procedure "give R > \underline{R}"?289: The logical expression in the indicator is a bit difficult to understand: how is a constant compared against an asymptotic expression? This should be rephrased.309: This result probably follows from an optimized setting of "a", which should be given in the text.381: The EWOO algorithm of Hazan, Agarwal and Kale (2007) precedes the work of Narayanan and Rakhlin on continuous EXP, and this is also probably a better reference for the trick used for proving Thm 6 than [7].443: The correct date for [5] is 2014.Reference---------E. Hazan, A. Agarwal and S. Kale: Logarithmic regret algorithms for online convex optimization. Machine Learning Journal 69(2-3), 2007, pp. 169 - 192.


Variational Gaussian Process State-Space Models
State-space models have been successfully used for more than fifty years in different areas of science and engineering. We present a procedure for efficient variational Bayesian learning of nonlinear state-space models based on sparse Gaussian processes. The result of learning is a tractable posterior over nonlinear dynamical systems. In comparison to conventional parametric models, we offer the possibility to straightforwardly trade off model capacity and computational cost whilst avoiding overfitting. Our main algorithm uses a hybrid inference approach combining variational Bayes and sequential Monte Carlo. We also present stochastic variational inference and online learning approaches for fast learning with long time series.

The paper presents a VB method for learning nonlinear state-space models using sparse GPs to model the nonlinear state transition and observation mappings. The proposed method looks very good and efficient, but the empirical evaluation is relatively weak.Quality: The paper appears technically sound, save one minor problem listed below. The method is based on existing solid principles with VB-based sparse GPs, stochastic variational inference and sequential Monte Carlo. The experimental evaluation is limited to two relatively small systems and includes no comparisons to other methods. Explicit experimental comparisons with existing nonlinear SSM learning methods would strengthen the paper.Clarity: The paper is in principle well-written and easy to read. A lot of details are missing, especially about the SMC but also about other details such as number and selection of the inducing inputs as well as more details of the covariance functions. Including all possible details in the paper is especially important as there is no code available for the method.Originality: The proposed method is novel. The differences of the method and previous work are explained clearly and relevant earlier work is cited properly.Significance: Learning models of nonlinear dynamical systems is an important problem with many potential applications. The paper promises significant improvement in efficiency over previous approaches in this problem. If there were a nice implementation available, I believe it could have quite significant impact. As is, without an implementation the direct practical impact is more uncertain because of the complexity of the method and its implementation, but it will still likely be useful as a basis of future research.Specific comments:Eq. (4): Shouldn't you have (f_{1:t-1}-mu_f_{1:t-1}) in the mean instead of just f_{1:t-1}?Line 262-264: I believe a sparse Gaussian MRF with a sparse precision matrix (zeros between non-neighbouring states) would make more sense than a sparse covariance matrix.Line 268: The comment on biasedness of "vanilla" SMC estimators suggests that the question of sampling q^*(x) should probably be discussed in more detail. The current text is quite confusing as on one hand it says the method uses SMC without specifying any details, and next it says SMC is biased. For someone aiming to implement the method, it would be valuable to discuss this in more detail to know what is valid and what is not.Sec. 6.1-6.2: Which mean functions were used in the experiments?Sec. 6.2: which covariance function was used? How big was the data set?Update after author feedback: the new experimental results presented in feedback clearly strengthen the argument of the paper. The authors seem to have avoided the comments about clarity and lack of details - I really hope they will take those into account in the final submission if the paper is accepted.


Fast Sampling-Based Inference in Balanced Neuronal Networks
Multiple lines of evidence support the notion that the brain performs probabilistic inference in multiple cognitive domains, including perception and decision making. There is also evidence that probabilistic inference may be implemented in the brain through the (quasi-)stochastic activity of neural circuits, producing samples from the appropriate posterior distributions, effectively implementing a Markov chain Monte Carlo algorithm. However, time becomes a fundamental bottleneck in such sampling-based probabilistic representations: the quality of inferences depends on how fast the neural circuit generates new, uncorrelated samples from its stationary distribution (the posterior). We explore this bottleneck in a simple, linear-Gaussian latent variable model, in which posterior sampling can be achieved by stochastic neural networks with linear dynamics. The well-known Langevin sampling (LS) recipe, so far the only sampling algorithm for continuous variables of which a neural implementation has been suggested, naturally fits into this dynamical framework. However, we first show analytically and through simulations that the symmetry of the synaptic weight matrix implied by LS yields critically slow mixing when the posterior is high-dimensional. Next, using methods from control theory, we construct and inspect networks that are optimally fast, and hence orders of magnitude faster than LS, while being far more biologically plausible. In these networks, strong -but transient -selective amplification of external noise generates the spatially correlated activity fluctuations prescribed by the posterior. Intriguingly, although a detailed balance of excitation and inhibition is dynamically maintained, detailed balance of Markov chain steps in the resulting sampler is violated, consistent with recent findings on how statistical irreversibility can overcome the speed limitation of random walks in other domains.

Summary: This paper deals with sampling methods based on linear rate-based neural-networks. First, it shows that symmetric weights (a common constraint in many models) significantly hurt the mixing rate. Then it shows that a (more physiological) non-normal network can have a much faster mixing rate, if the connectivity is optimized for this purpose. This works even if more biological constraints (Dale's law) are imposed.Clarity: Very clear and well organized paper.Originality: To the best of my knowledge, these results are novel.Significance: This is not completely clear too me. The importance of this paper mainly depends on how does this generalizes to non-linear networks - which are more physiological, and can be used to generate more sophisticated samplers. Perhaps this could be shown in a simulation. If this does not generalize, then the significance critically depends on how the issues below are resolved.Quality: The derivations appear to be correct and elegant. However, I'm quite confused as to what is the advantage of the suggested optimized networks. Given a linear network as defined in the paper, it is straightforward to construct a network that mixes as fast as possible (with timescale \tau_m) - if we are allowed to use more neurons in the network than components in the sampled vector (r). Simply have two layers (r & h) of neurons with no connections within each layer, and a feedforward connectivity \sqrt(\Sigma-I) from the 'hidden' layer h to the visible layer r. Then it straightforward to show that we can sample from the r layer with the timescale of a single neuron. Clearly, this is similar to the pre-conditioned solution, except the noise sources are not external. Note that you can make it more physiological by adding recurrent connections to the hidden layer (and modifying accordingly the feedforward connectivity to \sqrt(\Sigma-I)*A, where A is the appropriate 'whitening' matrix) and outgoing connections from the r layer (to some other, external neurons). Also we can maintain Dale's Law by summing together several hidden layers. I'm not sure what is the problem with such a simple solution. My final assessment of this paper will mainly depend on the answer to this question, as well the following (closely related) questions:1) What is the asymptotic scaling of slowing factor (\tau_max / \tau_m) for the optimized networks? Can you always maintain a single neuron timescale for the mixing, as N increases? Or do you have to decrease the simulation timestep in order to maintain numerical stability? If so, by how much?2) Why do you need the L2-norm penalty on Eq. 13? Is it to maintain numerical stability? Does the network has to be close the edge of stability in order to mix fast?*** After Author's feedback ***The author feed back have mostly satisfied my objections. I'm still not convinced that my suggestion is not biophysically plausible. If the hidden neurons have recurrent connections, which are then "compensated for" by the feedforward connectivity to the visible neurons, it is not necessarily wasteful, since the hidden neurons can simultaneously do something else (e.g., not sampling). In other words, using feedforward connectivity, you can use any other part of the brain as a stochastic "source" which generates fast sampling. It is not clear to me how this would be non-consistent with literature. Anyway, if you can show that your method works also for non-linear networks, this makes this issue less important.