Online and Stochastic Gradient Methods for Non-decomposable Loss Functions
Modern applications in sensitive domains such as biometrics and medicine frequently require the use of non-decomposable loss functions such as precision@k, F-measure etc. Compared to point loss functions such as hinge-loss, these offer much more fine grained control over prediction, but at the same time present novel challenges in terms of algorithm design and analysis. In this work we initiate a study of online learning techniques for such non-decomposable loss functions with an aim to enable incremental learning as well as design scalable solvers for batch problems. To this end, we propose an online learning framework for such loss functions. Our model enjoys several nice properties, chief amongst them being the existence of efficient online learning algorithms with sublinear regret and online to batch conversion bounds. Our model is a provable extension of existing online learning models for point loss functions. We instantiate two popular losses, Prec @k and pAUC, in our model and prove sublinear regret bounds for both of them. Our proofs require a novel structural lemma over ranked lists which may be of independent interest. We then develop scalable stochastic gradient descent solvers for non-decomposable loss functions. We show that for a large family of loss functions satisfying a certain uniform convergence property (that includes Prec @k , pAUC, and F-measure), our methods provably converge to the empirical risk minimizer. Such uniform convergence results were not known for these losses and we establish these using novel proof techniques. We then use extensive experimentation on real life and benchmark datasets to establish that our method can be orders of magnitude faster than a recently proposed cutting plane method.

Summary:This paper studies online learning techniques for optimizing non-decomposable loss functions, such as precision@k and partial AUC, aiming to enable incremental learning as well as design scalable solvers for batch learning tasks. They prove sublinear regret and online to batch conversion bounds for the proposed algorithms. This paper develops scalable stochastic gradient descent solvers for non-decomposable loss functions and validates its efficiency via empirical studies.Quality:This paper proposes a practical online learning framework for non-decomposable loss functions. It presents two algorithms with mini batches, and gives theoretical guarantee with sublinear regret and online to batch conversion bounds. Empirically, the two proposed algorithms are compared with two existing batch learning algorithms, and found better results on several real-world data sets. In general, the work appears to be very similar to the series of studies in [13,14]. However, the authors did not clearly clarify the differences and contributions as compared to the similar work in [13,14], making it difficult to justify the novel contributions.  Clarity:The manuscript is generally well written and easy to follow, except for some minor typos. For example, In line 170, it should be “not clear and remains” instead of “not clear an remains”. Originality:This paper extends the ideas in [13, 14] to tackle the general online learning tasks with non-decomposable loss functions. Instead of optimizing AUC, the authors consider two types of other losses, precision@k and partial AUC, and propose two specific online learning algorithms (single-pass and two-pass online learning) using the idea of mini-batches (which are essentially very similar to the buffering techniques as used in [13,14]). The extensions are OK, but not particularly novel. Significance:This paper addresses an important problem, online learning with non-decomposable loss functions, which could be more difficult than regular online learning with pointwise loss functions. The paper has presented two new online/stochastic learning algorithms for optimizing partial AUC and precision@k, and found better results in comparison to two existing batch learning algorithms. The limitations of this work is that the authors did not justify the significance of the proposed algorithms over the existing baselines in literature (e.g., [13,14]). Comments on specific issues:1.	The empirical study has not been done in a thorough way. Since the paper aims to address online learning tasks, it is not sufficient to compare only with two batch learning algorithms. In particular, one would expect a direct comparison of the proposed algorithms with some closely related online learning algorithms in literature (e.g., [13,14]). 2.	The two proposed algorithms (1PMB and 2PMB) have their obvious drawbacks although they seem to outperform the batch learning algorithms. In particular, for the first algorithm could fail to exploit relationships between data points, as admitted by the authors; for the second algorithm, simply collect a (single) random sample of positive example B+ may be neither effective (if sampling size s is too small) nor memory efficient (if sampling size s is too large). I think the buffer sampling treatments proposed in [13, 14] seem to be more practical to address such drawbacks of the proposed algorithms in this work. 3.	Tuning parameters is not well explained. The authors only stated “all tunable parameters were chosen using a small validation set”. As the proposed algorithm depends on several important parameters such as the buffer size s of B+, B-, the step scale length \eta, which could affect the performance. However, there are no discussions or experiments for examining how these parameters.  4.	The empirical results fail to validate efficacy of the proposed two-pass algorithm (2PMB). It seems that 2PBM is similar or even worse than the Single-pass algorithm (1PMB) for most cases. Since this algorithm needs to take multiple passes, the value of the 2PMB algorithm seems in doubt.


Graphical Models for Recovering Probabilistic and Causal Queries from Missing Data
We address the problem of deciding whether a causal or probabilistic query is estimable from data corrupted by missing entries, given a model of missingness process. We extend the results of  by presenting more general conditions for recovering probabilistic queries of the form P (y|x) and P (y, x) as well as causal queries of the form P (y|do(x)). We show that causal queries may be recoverable even when the factors in their identifying estimands are not recoverable. Specifically, we derive graphical conditions for recovering causal effects of the form P (y|do(x)) when Y and its missingness mechanism are not d-separable. Finally, we apply our results to problems of attrition and characterize the recovery of causal effects from data corrupted by attrition.

The paper deals with the problem of identifiability in the presence of various generated mechanisms for missing data, using a framework introduced by Mohan and others.  Some existing results are improved upon, and some nice examples are given (such as identifiability of causal distributions even when conditional distributions are unavailable).QualityTechnical quality seems high, the results are very involved but appear to be correct.  Results like Theorem 3 are clear and interpretable.  Some of the results are incremental improvements upon existing ones, and don't contribute (so far as I can see) to a bigger picture of what is really going on here.Section 3 seems orthogonal to the rest of the paper, and its main result (Theorem 1) is quite hard to comprehend.  It seems to be closely related to the Markov blanket results in Richardson (2009).  The quality improves later on, with section 5 giving some nice clear answers, and Section 6 some interesting results about causal queries.---ClarityThis is the paper's main weakness.  Firstly, there is probably too much material for a conference paper here (though the importance of some of it is unclear), as evidenced by the density of the text and the 13 page technical appendix.  The article does not flow clearly, and is hard to follow as a result - probably this would work better as a longer, more considered, journal paper.Whilst finding good notation for so many quantities is not easy, the notation could certainly be improved.  The use of the same capital 'V' to denote so many different objects makes it hard work for the reader, and in places the subscripts become illegible.  In addition, the fact that V_i is a vertex but V_o a set of vertices is particularly confusing.  The set equation in Corollary 1 is perhaps the most extreme example of poor presentation.---Originality and SignificanceSections 5 and 6 are clear and interesting, and potentially quite significant.  This paper deals with an important problem, and contains results which seem genuinely interesting and useful.  It would perhaps be better to step back and work out what is going on in the general case, and (try to) come up with a complete criterion for identifiability in this framework (as was achieved in the case of finding causal effects without missingness).  It is difficult to imagine a statistician trawling through a series of papers like this to find the precise result which shows (non)-identifiability of the causal effect she is interested in, especially if that result is as complicated as Theorem 1.Theorem 2 and Corollary 1 seem to be a form of inverse probability weighting.  The reason that these results work is presumably something to do with the conditioning sets being Markov blankets for R(i) in the mixed graphs the author is using.  Richardson (2003) and (2009) might be relevant.  ---Other Commentsp4, l214: R^part _is_ a partition of R, not "a set containing partitions".  The condition on line 214 should then be that Rx and Ry "belong to distinct elements of the partition".Language is sometimes a bit casual: e.g. 'oftentimes'p4 l213.  Use \leftrightarrow rather than <--> Is Garcia (2013) actually wrong, or just a different framework?A notation for graphs constructed by severing edges is used in Definition 4 and Theorem 4, but not defined anywhere.V_o and V_m are used in the sentence before they are defined (apparently unnecessarily).---ReferencesRichardson, T.S., Markov properties for acyclic directed mixed graphs, SJS 30(1) 145-157, 2003Richardson, T.S., A factorization criterion for acyclic directed mixed graphs, UAI 25, 2009.


A ⇤ Sampling
The problem of drawing samples from a discrete distribution can be converted into a discrete optimization problem [1,2,3,4]. In this work, we show how sampling from a continuous distribution can be converted into an optimization problem over continuous space. Central to the method is a stochastic process recently described in mathematical statistics that we call the Gumbel process. We present a new construction of the Gumbel process and A ⇤ Sampling, a practical generic sampling algorithm that searches for the maximum of a Gumbel process using A ⇤ search. We analyze the correctness and convergence time of A ⇤ Sampling and demonstrate empirically that it makes more efficient use of bound and likelihood evaluations than the most closely related adaptive rejection sampling-based algorithms.

This paper introduces a new approach to sampling from continuous probability distributions. The method extends prior work on using a combination of Gumbel perturbations and optimization to the continuous case. This is technically challenging, and they devise several interesting ideas to deal with continuous spaces, e.g. to produce an exponentially large or even infinite number of random variables (one per point of the continuous/discrete space) with the right distribution in an implicit way. Finally, they highlight an interesting connection with adaptive rejection sampling. Some experimental results are provided and show the promise of the approach. The paper is well written (I have a few suggestions below) and generally easy to read (Figure 1 is nice and very helpful to understand the algorithm). This is very novel work where the authors explore an interesting combination of ideas from computer science and traditional AI (like A* search) with advanced statistics. The paper deals with a very important problem which is very relevant to NIPS.The experimental evaluation is a bit limited (and only on synethetic data) but I think it might be good enough as an initial step. One issue is that the algorithm doesn't seem to be able to handle high-dimensional spaces well, but that is of course very hard. One concern I have is that the performance metric used in the experimental section is "likelihood evaluations per sample". What about runtime? Are there overheads in using A* sampling?Other comments:It would be good to specify clearly what is the input and what is the output of the algorithms reported as pseudocode (Algorithm 1 and 2).Line 177. I would expand the first paragraph of section 4. The decomposition in tractable and intractable parts is key. The results in the previous sections obviously don't make any sense if we can already sample and integrate. I would try to say more about what the decomposition means, maybe mentioning some of the examples used later.  Explanation in section 6.2 is a bit confusing (especially the last paragraph is rather obscure)In 6.3, why is it compared to rejection sampler and not with OS*? Is it so slow that can only do 3 points and 5 runs?How does one deal with ties? If the argmax contains more than one element, do we have to sample uniformly from the set or is it sufficient to output one? I am guessing in principle it only happens with probability zero so it might not be an issue, but since you are generating an exponentially large number of random variable using finite precision arithmetic can it still happen?It's unclear what the probability is conditioned on in Eq (3)Line 85-86: at that point it's unclear what it means that max and argmax are independentline 4 of pseudocode: are there technical assumptions for the max to be defined (as opposed to a supremum)


Fast Kernel Learning for Multidimensional Pattern Extrapolation
The ability to automatically discover patterns and perform extrapolation is an essential quality of intelligent systems. Kernel methods, such as Gaussian processes, have great potential for pattern extrapolation, since the kernel flexibly and interpretably controls the generalisation properties of these methods. However, automatically extrapolating large scale multidimensional patterns is in general difficult, and developing Gaussian process models for this purpose involves several challenges. A vast majority of kernels, and kernel learning methods, currently only succeed in smoothing and interpolation. This difficulty is compounded by the fact that Gaussian processes are typically only tractable for small datasets, and scaling an expressive kernel learning approach poses different challenges than scaling a standard Gaussian process model. One faces additional computational constraints, and the need to retain significant model structure for expressing the rich information available in a large dataset. In this paper, we propose a Gaussian process approach for large scale multidimensional pattern extrapolation. We recover sophisticated out of class kernels, perform texture extrapolation, inpainting, and video extrapolation, and long range forecasting of land surface temperatures, all on large multidimensional datasets, including a problem with 383,400 training points. The proposed method significantly outperforms alternative scalable and flexible Gaussian process methods, in speed and accuracy. Moreover, we show that a distinct combination of expressive kernels, a fully non-parametric representation, and scalable inference which exploits existing model structure, are critical for large scale multidimensional pattern extrapolation.

The authors combine two recent advances in Gaussian processes, spectral mixture kernels [5], and scalable Gaussian processes for data in grids [14, 22 see below], in order to tackle applications with high amount of data points, like texture extrapolation, inpainting, and video extrapolation.  The paper includes a thorough evaluation of the framework proposed, and comparisons against sparse GP methods, with general purpose covariance functions, and spectral mixture kernels.QualityThe paper is technically sound. The framework proposed by the authors achieves outstanding results in the different applications studied in the paper. The authors claim that they relax the grid assumption, by completing with imaginary observations. It is important to discuss this issue when each of the experiments is being introduced. The imaginary observations are mentioned in section 2.1 and hardly ever mention again. How would the method work in higher dimensional settings (greater than three)? How prone to overfitting is the method? What are the weaknesses of the method?ClarityThe paper is clearly written and well organized. OriginalityAs the authors clearly stated it, the framework proposed is a combination of the spectral mixture kernels, and fast Gaussian process regression on grids. From this point of view, the novelty of the method is incremental. SignificanceI must say that the quality of the results obtained by the proposed method is impressive. The method beats any other of the GP alternatives. I wonder how methods in the image inpainting related literature would compare against the ones presented by the authors.[22] Yuancheng Luo and Ramani Duraiswami (2013): Fast Near-GRID Gaussian process regression, at AISTAST 2013.


Active Learning and Best-Response Dynamics
We examine an important setting for engineered systems in which low-power distributed sensors are each making highly noisy measurements of some unknown target function. A center wants to accurately learn this function by querying a small number of sensors, which ordinarily would be impossible due to the high noise rate. The question we address is whether local communication among sensors, together with natural best-response dynamics in an appropriately-defined game, can denoise the system without destroying the true signal and allow the center to succeed from only a small number of active queries. By using techniques from game theory and empirical processes, we prove positive (and negative) results on the denoising power of several natural dynamics. We then show experimentally that when combined with recent agnostic active learning algorithms, this process can achieve low error from very few queries, performing substantially better than active or passive learning without these denoising dynamics as well as passive learning with denoising.

SUMMARY:This paper combines dynamics from game theory and active learning techniques to solve a problem where a center tries to determine a decision boundary from noisy sensors.  Under the assumption that the noise is independent, the authors prove that simple simultaneous or asynchronous game dynamics successfully denoise the system with high probability, given enough sensors.  To illustrate the effectiveness of the denoising together with active learning (which is known to perform poorly in the presence of large amounts of noise), the authors give experiments in the plane which show superior results compared with learning without denoising and passive learning with denoising.CRITIQUE:The results are interesting, the approach is novel, and the exposition is clear.  My only concern is the significance.  I believe the authors make a good case in general that denoising a collection of sensors, using simple distributed local computations, can be effective enough in some situations to allow active learning to be applied, thereby enabling an efficient sensor query load.  However, the bounds given in the analysis leave a lot of room to question the applicability of this approach in practice.  Is it really reasonable to have N = 10000 for r = 0.1, as in the experiments?  Some rough sense of reasonable settings of these parameters is badly needed to evaluate the utility of this approach.  For example, if r = 0.8, then surely one would want a more sophisticated denoising algorithm, since the theoretical guarantees break down, and moreover nodes have unreasonable communication loads.  Similarly, if r=0.001, then the communication graph is likely to be disconnected, unless N is even larger.  What settings of r, N, eta, etc are reasonable?  Likewise, is a uniform distribution reasonable?As another high level point, I was surprised not to see more discussion of the 2r band around the separator within which all bets are off -- are we assuming r is so small that this is negligible?  For large r, it seems that a slightly more sophisticated distance-weighted approach would be much better here.  This (or some similar procedure) is stated in a footnote to perform similarly to the simple majority; intuitively, the two should behave roughly the same away from the boundary, but the distance weighting should perform significantly better near the boundary, even for smaller r.  A more fine-grained comparison near the boundary would be nice here.SPECIFIC COMMENTS:pg 1. This paper would be much more accessible if terms such as active learning and agnostic active learning were defined in simple terms after their first use (e.g. "active learning, a branch of machine learning where...")pg 2. "sensors within distance r are connected by an edge" -- as mentioned above, the relationship between r and d (dim) and N is not discussed much in the text; aside from what values one should expect in practice, it might be good to mention some results from theoretical computer science / graph theory about the connectedness of the communication graph.pg 3. it would help to give an overview of section 3 at the top, since it was not stated priorpg 7. "Pockets of noise appear to be more difficult to denoise." -- the algorithm was not designed for this case, so this is not surprising.  One might expect pockets to be less severe in higher dimensions.pg 8. "A synchronous round" -- perhaps change to "One synchronous round" to avoid confusion with "asynchronous"


Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Predicting depth is an essential component in understanding the 3D geometry of a scene. While for stereo images local correspondence suffices for estimation, finding depth relations from a single image is less straightforward, requiring integration of both global and local information from various cues. Moreover, the task is inherently ambiguous, with a large source of uncertainty coming from the overall scale. In this paper, we present a new method that addresses this task by employing two deep network stacks: one that makes a coarse global prediction based on the entire image, and another that refines this prediction locally. We also apply a scale-invariant error to help measure depth relations rather than scale. By leveraging the raw datasets as large sources of training data, our method achieves state-of-the-art results on both NYU Depth and KITTI, and matches detailed depth boundaries without the need for superpixelation.

The authors present a new method for estimating the depth map of a scene using a single image. They use two CNNs: the first outputs a coarse depth prediction based on the entire image. This coarse depth prediction is used as the input, together with the original image, to the second CNN which refines the depth prediction locally. They also present and apply a scale-invariant error measure for the depth prediction. They demonstrate state of the art results on NYU Depth and KITTI datasets.The paper is well written and clear, I believe researchers who would like to reproduce the results could do that using the details found in the paper. I also find that the problem they solve - estimating the depth of a scene using a single image, is relevant and interesting to the NIPS community.However, much of the significance of the paper and the interest it raises is related to the two CNNs it uses - global and local. I would like to see better analysis  of the importance of this two networks. It seems that quantitatively there isn't much of a difference to when using only the coarse CNN and both CNNs. How would the results look like if we would feed the second CNN with some other depth prediction which is not "coarse" (e.g. the output of Make3D), or if the first CNN will also be more local?In the end of the first paragraph of 3.1.1 the authors explain the importance of global cues for depth estimation, such as vanishing points and room alignment. Is there a way to understand the patterns learned as such cues?The fact that the choice of the lambda parameter of the loss doesn't give numeric gain but changes the quality of the prediction is a bit troubling (as the use of the second network)  and suggests that the error measure used is not the most suited for this task.I must say that when examining qualitatively the results presented both in the paper and in the supplementary material it seems that: (a) the depth estimate is still quite far from the ground truth. (b) on the KITTI dataset, it doesn't seem to me that the suggested architecture is better than Move3D.I would like to see the running time and the computational resources needed for training and testing.Minor comment:Can it be that you forgot a factor 2 when moving from eq. 2 to eq. 3?


Proximal Quasi-Newton for Computationally Intensive 1 -regularized M -estimators
We consider the class of optimization problems arising from computationally intensive 1 -regularized M -estimators, where the function or gradient values are very expensive to compute. A particular instance of interest is the 1 -regularized MLE for learning Conditional Random Fields (CRFs), which are a popular class of statistical models for varied structured prediction problems such as sequence labeling, alignment, and classification with label taxonomy. 1 -regularized MLEs for CRFs are particularly expensive to optimize since computing the gradient values requires an expensive inference step. In this work, we propose the use of a carefully constructed proximal quasi-Newton algorithm for such computationally intensive M -estimation problems, where we employ an aggressive active set selection technique. In a key contribution of the paper, we show that the proximal quasi-Newton method is provably super-linearly convergent, even in the absence of strong convexity, by leveraging a restricted variant of strong convexity. In our experiments, the proposed algorithm converges considerably faster than current state-of-the-art on the problems of sequence labeling and hierarchical classification.

## SummaryThe paper introduces a proximal quasi Newton algorithm for the solution ofcomputationally intensive l1-regularized m-estimators. CRFs are an example ofsuch a setting (it is relatively slow to compute the function value and/orgradient of an instance). The main contribution is a superlinear convergenceproof of the algorithm under the assumption of strong convexity in a subspace ofthe original optimization.## Specific Points- it's Lipschitz, not Lipchitz!- 134: one-dimensional problem- 183: of the inner problem- 201: include a pointer to the appendix in the caption?- 241: q-superlinear without q appearing in the formula...- 249: state assumptions onn \ell(w) again?- 259: CRF problems- 321: one-dimensional- 330: only in very few blocks- 354: I'd be instered to see how performance is for different \lambda and  possibly also different feature functions (say non-quadratic unary features)  to get an understanding of whether the algorithm improves performance across  the board.- 374: why no cross-validation? Or at least show results for several \lambda- 408: how do you choose \lambda=2 and \eta_0=0.1? Does performance change if  you alter \lambda?- 427: I feel it is rather unfair to include the time for the whole function  evaluation in SGD, can't you just *not* include the time required to compute the  objective for plotting purposes? Or show plots with epochs, rather than time  on the x-axis.- 429: rephrase second part of sentence- [430: short discussion/summary is always appreciated]## QualityThe experiments should be improved (fix SGD and include some other settings of\lambda). Also, it is not obvious to me whether the convergence rate still holdfor the actual implementation of Prox-QN with the inexact inner problem solution /shrinking strategy.## ClarityThe paper is well written and clear in most parts. I would appreciate some moredetails in the main paper about the shrinking strategy. It is also not clear tome how the shrinking strategy comes into play with the convergence proof?## OriginalityAs far as I can say the work is novel. Overlap with [1] might be something toworry about, because there might the CNSC part is probably present in bothworks.## SignificanceThe experiments seem to support the superlinear convergence, which is quiteimpressive (a plot with test error might though still be interesting). Efficienttraining of l1-regularized CRFs is an important problem and such substantialspeedups make the work significant.


On the Relationship Between LFP & Spiking Data
One of the goals of neuroscience is to identify neural networks that correlate with important behaviors, environments, or genotypes. This work proposes a strategy for identifying neural networks characterized by time-and frequency-dependent connectivity patterns, using convolutional dictionary learning that links spike-train data to local field potentials (LFPs) across multiple areas of the brain. Analytical contributions are: (i) modeling dynamic relationships between LFPs and spikes; (ii) describing the relationships between spikes and LFPs, by analyzing the ability to predict LFP data from one region based on spiking information from across the brain; and (iii) development of a clustering methodology that allows inference of similarities in neurons from multiple regions. Results are based on data sets in which spike and LFP data are recorded simultaneously from up to 16 brain regions in a mouse.

Summary:The authors propose a convolutional dictionary model that captures relationships between spikes and LFP signals across multiple brain regions. Significant contributions are 1) the use of a clustering prior on the dictionary elements, 2) the ability to model dynamics of the dictionary elements, and 3) the corresponding mean-field variational inference procedure for the model.The authors then present a multitude of results on two large datasets. They find that time-varying dictionary elements yield superior performance over static elements, and that the use of clustering allows for improved interpretability of the data.In the novel environment experiment, they find that some dictionary elements change at the introduction to the new environment while others do not – and that the clustering reveals, for example, that a particular stable element is mostly associated with HP cells.Quality:The authors show convincingly that clustering and dictionary dynamics offer significant improvements both in quantitative fit and qualitative interpretability over simpler spiking-to-LFP models. A strong point of the paper is the variety of results presented on both datasets, showcasing the utility of the method.The model choices are reasonable. For example, the observation that similar results were found for a wide range of Bs suggest that the choice of modeling the dynamics over large discrete bins is sensible.With that said, what motivation is there for a first-order AR process over higher orders? This seems a stringent limitation on the dynamics of the dictionary elements.The paper might benefit from a discussion on how both spiking and LFP signals change over the course of both experiments. That is, are the dynamics of the dictionary elements due to changes in spiking statistics, LFP signals, or both? The emphasis was on the changes in dictionary elements themselves, illustrating that it is the relationship between spikes and LFP that changes over the course of the experiment, but it might help to understand this point in relation to the spiking and LFP signals themselves.Clarity:This paper is exceptionally clear.Small details:- There is a mismatch in the text on page 6 and the corresponding figures 2 and 3 (text says thalamus LFP, figure says HP LFP, etc)- best values not bolded in table 2.- how many clusters are typically identified. E.g. figure 2 shows 3 of how many?Originality:The method is original, and the authors present several original types of analyses one could perform with the method (e.g. see results in figure 3).Significance:This work appears to offer significant improvements over previous methods, and has potential to see widespread use as multi-region datasets become more common.


Semi-Separable Hamiltonian Monte Carlo for Inference in Bayesian Hierarchical Models
Sampling from hierarchical Bayesian models is often difficult for MCMC methods, because of the strong correlations between the model parameters and the hyperparameters. Recent Riemannian manifold Hamiltonian Monte Carlo (RMHMC) methods have significant potential advantages in this setting, but are computationally expensive. We introduce a new RMHMC method, which we call semi-separable Hamiltonian Monte Carlo, which uses a specially designed mass matrix that allows the joint Hamiltonian over model parameters and hyperparameters to decompose into two simpler Hamiltonians. This structure is exploited by a new integrator which we call the alternating blockwise leapfrog algorithm. The resulting method can mix faster than simpler Gibbs sampling while being simpler and more efficient than previous instances of RMHMC.

The authors introduce a new Hamiltonian and claim that it is better than existing Hamiltonians for heirarchical models. The technical material is very well-written, and the Hamiltonian they present is (as far as I know) novel. It is also very plausible that the new Hamiltonian leads to substantial improvements. However, I found it to be most confusing (and least convincing) in the few places that it argues the new method is better than the old one.Major Issues:1. The authors say that they are measuring "effective sample size" but don't say how they are computing this; they reference the reader to [6]. I haven't read [6] carefully, but certainly it doesn't define effective sample size (indeed the article doesn't contain the word "effective" and uses "size" only to refer to actual counts). Normally I wouldn't be so picky, but I think this is critical to their claims. In particular, for 2 of the 3 tables, their proposed method is MUCH slower than the competitors as measured by clock time; they rely on normalized effective sample size to argue that their method is better despite this. I actually think that, almost regardless of the normalization method they use for computing ESS, this is going to be a pretty unconvincing argument (the same table could be used to show that almost any adaptive algorithm is better than almost any other, since adaptive algorithms care so much about initial burn-in periods). But it is especially unconvincing if the normalization is not defined.This leads me into a bigger reason that their numerics aren't as convincing as I might hope. It is clear from the last column of table 1 (and figure 1) that some of the competitor algorithms are still quite far from stationarity after the number of steps that they run. If each step had a similar cost, that would be a fair comparison: they would be showing that their algorithm converges more quickly, and that's that. The whole variance thing would be a bit of a red herring in this situation (most of the error for the competitors is coming from bias, not variance), but it would be a useful thing to discuss. However, from their own tables, this doesn't seem to be the case. Their algorithm seems to take much more time per step than the competitor algorithms, and they have chosen a NUMBER of STEPS such that their algorithm has converged but the competitors haven't. This really doesn't seem to be a fair comparison, since effective sample size, bias, etc. can all get better as the algorithm runs.To describe the unfairness a bit more graphically, I'll describe an artifical example that might get something like their table 1. The original algorithm the paper discusses is MCMC driven by kernel $K$ with mixing time $T$. The paper proposes a new algorithm: run $K^{7}$ (that is, take every 7'th point from a Markov chain driven by $K$). We then look at a sample obtained from running both algorithms for $\frac{T}{3}$ steps. Just like Table 3, the new algorithm would take about 7 times as long to run as measured by wall clock. Also just like table 3, we would expect the old algorithm to have enormous bias (it hasn't mixed yet) while the new algorithm would have very tiny bias (it has mixed well). Its clear in my artificial example that the comparison is unreasonable, and I think that for the same reason it is a little unreasonable in this paper.Minor Issues:1. Bottom of pp.2, the authors claim that some dynamics are not `valid' because they aren't reversible. They probably meant to say something about how they don't converge to the target.2. There seems to be a bit of a bait-and-switch at the bottom of page 3. Up until now, the new method has been motivated by pointing out that competitor algorithms scale poorly in high dimensions. When the scaling of this paper's algorithm is revealed, it looks pretty similar for the example under consideration, but this is brushed away as being unimportant. This isn't such a big deal for the applicability of the paper; it just makes the note feel a little dishonest. Alternatively, maybe this is really a big difference, in which case they should explain why.3. I wish section 4.3 were longer! This seems like the most important section, but it wasn't enough for me to figure out what was going on. This is the sort of section that would be *vastly* improved by *any* toy example where this heuristic could be made precise. In particular, it would be nice to see that there is some optimal value of total energy in the Gibbs version of HMC, and that their method of passing energy back and forth between components (rather than keeping it constant) actually does better than that optimal value. Even just stating the algorithm and values would be enough, which could be done in a few lines. If the claim really is obvious, I hope that they have such an example.4. In figure 1, I don't think the author's explain the coloring on pictures 2 and 4.


Localized Data Fusion for Kernel k-Means Clustering with Application to Cancer Biology
In many modern applications from, for example, bioinformatics and computer vision, samples have multiple feature representations coming from different data sources. Multiview learning algorithms try to exploit all these available information to obtain a better learner in such scenarios. In this paper, we propose a novel multiple kernel learning algorithm that extends kernel k-means clustering to the multiview setting, which combines kernels calculated on the views in a localized way to better capture sample-specific characteristics of the data. We demonstrate the better performance of our localized data fusion approach on a human colon and rectal cancer data set by clustering patients. Our method finds more relevant prognostic patient groups than global data fusion methods when we evaluate the results with respect to three commonly used clinical biomarkers.

This paper generalizes multi-kernel k-means clustering (solved via relaxation) to the case where each clustered item (here, a person) gets an item-specific set of weights over the multiple kernels, rather than the traditional, shared, global weighting of the kernels. Using TCGA (cancer) data, with 3 modalities, they demonstrate that this generalization yields better clusterings than the traditional (global approach), when measured against 3 "bronze standard" clusterings arising from known clinical clusters.The writing is clear, making for an easy read.Although this is a somewhat incremental-seeming tweak, I think it was clever, with the potential to actually be used (rather than lost in the NIPS archives), and therefore of some significance. Other comments:In the introduction you mention that k-means is susceptible to local minima, and then use this to motivate the relaxation approach. However, as described on p3, even the relaxation approach ends up using k-means clustering, so this is a little puzzling to the reader.Following on my previous comment, it would seem that your approach is sensitive to initialization, and susceptible to local minima. Please make clear if this is true or not, and if it is, please explain how you deal with this issue (in your, and the other algorithms) in the experimental section.Also, why not use a naturally continuous-valued clustering like mixture of Gaussians, rather than the hard version of it of k-means? On p. 5 you discuss "computational effort" and doing things "more efficiently". Please give the time complexity for the various algorithms being discussed so that the reader can appreciate these points. Similarly, you mention the notion of time complexity in the Discussion, but not any actual time complexities.Following on the last point, what is the largest data set you expect to be able to handle with the current algorithm. 204 patients is rather small for current genomics data sets (and other data sets).  As a point of reference, please give a timing for your experiments.Concluding the success of the algorithm on the basis of one experiment, while not unusual for NIPS, is a little weak. Would be great if you could add in further experiments, not to mention some with a ground truth for objective comparison, if possible.You use always only 2 clusters, but it would be interesting to know if your model is just as amenable to model selection (i.e. # of clusters) as other methods. Presumably so, but would be good to know.p. 6 "aligned to each other"--slightly odd phrasing, but I eventually understood by looking at the figures. Similarly so the "nine biomarker-metric pairs".p.7 would be good if you annotated the tie with non-bold (maybe with a *), as I was confused for a while about where the 8/9 things were, until I realized there was a tie.


Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces
This paper introduces a novel mathematical and computational framework, namely Log-Hilbert-Schmidt metric between positive definite operators on a Hilbert space. This is a generalization of the Log-Euclidean metric on the Riemannian manifold of positive definite matrices to the infinite-dimensional setting. The general framework is applied in particular to compute distances between covariance operators on a Reproducing Kernel Hilbert Space (RKHS), for which we obtain explicit formulas via the corresponding Gram matrices. Empirically, we apply our formulation to the task of multi-category image classification, where each image is represented by an infinite-dimensional RKHS covariance operator. On several challenging datasets, our method significantly outperforms approaches based on covariance matrices computed directly on the original input features, including those using the Log-Euclidean metric, Stein and Jeffreys divergences, achieving new state of the art results.

Summary: This paper extends the idea of Log-Euclidean metric (LEM) on the Riemannian manifold of positive definite matrices to an infinite-dimensional setting. In this work, the proposed Log-Hilbert-Schmidt metric (LHSM) provides ways of computing distance between positive definite operators on a Hilbert space. From a theoretical perspective, extending the LEM to an infinite dimensional space is not a trivial task because one cannot obtain the infinite-dimensional counterpart of LEM by simply letting the dimension approach infinity. There are two fundamental problems which have been addressed in this paper. First, if A is a self-adjoint operator in an infinite-dimensional Hilbert space, log(A), which is an important ingredience in Log-Euclidean metric, is not well-defined and may not be bounded. Second, the identity operator in this space is not Hilbert-Schmidt and would have infinite distance from any Hilbert-Schmidt operator. This paper gives a rigorous solutions to the aforementioned problems. Moreover, when the covariance operator is defined on an RKHS induced by a positive definite kernel, the paper gives an explicit formulas for the proposed metric that can be evaluated entirely via the corresponding Gram matrix.From application point of view, the proposed metric is applied in the task of multi-category image classification in which each image is represented by an infinite-dimensional RKHS covariance operator constructed from "original features" in the input space. The proposed metric is found to outperform the state-of-the-art algorithm which only considers the finite covariance matrix in the input space. However, it will be more interesting to compare this method to other image classification approaches, e.g., those based on deep learning.The idea presented in this paper makes sense intuitively, and will be very useful in practice because one can view this framework as a non-linear extension of learning problems when the samples are represented by the covariance matrices. In addition to images, I believe one can broadly apply this metric to other kinds of data which have high-order correlation. Moreover, defining the metric in the RKHS allows even more flexibility as one can freely choose different kernel functions.Quality:The paper is of high quality. The detail of the claims and proofs are given in great detail.Clarity:The paper is clearly written and I am able to follow the logic.Originality:The novelty of this work is the rigorous formulation of infinite-version of Log-Euclidean metric. The problem is not trivial and I think the contribution of this work in term of technical results is quite substantial.Significance:The proposed metric is shown to outperform state-of-the-art algorithms, so the contribution of this work should be significant in practice. However, the authors didn't compare different approaches in term of computational complexity, so it is difficult to conclude from this aspect.Some minor comments:- If I understand correctly, the covariance operator is defined for each sample, and therefore is fundamentally different from the standard way that the covariance operator has been defined such as in kernel PCA. That is, the covariance operator is constructed from a sample and as a sample size goes to infinity, we obtain the population counterpart of the underlying distribution. On the other hand, in this work each image is represented by a covariance operator. Do you have an intuitive explanation of what the population counterpart of this covariance operator is? Are they fundamentally the same or different?- In section 6, what are the "original features" extracted from the image and how exactly these features are generated?


Streaming, Memory Limited Algorithms for Community Detection
In this paper, we consider sparse networks consisting of a finite number of nonoverlapping communities, i.e. disjoint clusters, so that there is higher density within clusters than across clusters. Both the intra-and inter-cluster edge densities vanish when the size of the graph grows large, making the cluster reconstruction problem nosier and hence difficult to solve. We are interested in scenarios where the network size is very large, so that the adjacency matrix of the graph is hard to manipulate and store. The data stream model in which columns of the adjacency matrix are revealed sequentially constitutes a natural framework in this setting. For this model, we develop two novel clustering algorithms that extract the clusters asymptotically accurately. The first algorithm is offline, as it needs to store and keep the assignments of nodes to clusters, and requires a memory that scales linearly with the network size. The second algorithm is online, as it may classify a node when the corresponding column is revealed and then discard this information. This algorithm requires a memory growing sub-linearly with the network size. To construct these efficient streaming memory-limited clustering algorithms, we first address the problem of clustering with partial information, where only a small proportion of the columns of the adjacency matrix is observed and develop, for this setting, a new spectral algorithm which is of independent interest.

The paper describes two related algorithms for finding clusters under the stochastic block model assumption when there is partial information (in the sense that only a fraction of the columns of the adjacency matrix are revealed) or in a data streaming setting (where fewer than the size of the input bits are retained to perform the computation).Pros: Another contribution to an area of recent interest, both with respect to data-constrained computation, clustering matrices, etc.The main algorithm splits the problem into two steps, which is probably the correct way to do streaming clustering under model assumptions like this stochastic block model.  Making that explicit highlights both important points.Cons:The algorithm is of theoretical interest, as it makes unrealistically strong assumptions for realistic networks of even moderate size and since it doesn't work at important "boundaries" of edge density, e.g., extremely sparse graphs.Claims with respect to how this work fits into the streaming and community literature are very much overstated, as mentioned below.The description of the algorithm seems to not make clear what I think is going on: that under the stochastic block model with large blocks, then input has very nice structure, and so the main goal of the algorithm is to make sure that the sampling doesn't overly concentrate certain quantities in the "empirical" estimates.  For example, I think that is what the trimming step is doing, and I think that is similar to what Feige and others, Montaneri and others, and probably others have done.  A few other comments.  (1) The problem as stated is unrealistic for what most people would consider community detection in realistic networks.  For example, the clusters don't overlap, and the results don't go through when the graphs are extremely sparse (which is typical) or extremely dense (which may arise occasionally).  Thus, they are best viewed as contributing to the recent work on spectral-like algorithms for stochastic block (which also suffer from similar drawbacks) and related models under various computational constraints (which is of interest).(2) No empirical evaluation.  This isn't necessarily a bad thing, since if there were it would likely be trivial (in the sense that they would simply show a phase transition which they establish) or overly-idealized (i.e., they would have to "simulate" memory limitations, since dealing with realistic memory management is very nontrivial).(3) The columns are revealed one by one.  This is a rather strong assumption in terms of motivations, and it means that it is much easier to obtain much stronger results than if elements or blocks are revealed.  For example, much stronger results are obtained in low-rank matrix approximation when columns are sampled than when elements are sampled.  More formal data streaming models, e.g., in the theoretical computer science literature consider other data presentation formats.(4) Finally, if the authors don't know of other related work on clustering or partitioning or community finding, it is for lack of effort.  For example, a few quick web searches find many papers, including the following papers:FENNEL: Streaming Graph Partitioning for Massive Scale Graphs, by Tsourakaki, et al.Online Analysis of Community Evolution in Data Streams, by Aggarwal and YuStreaming Graph Partitioning for Large Distributed Graphs, by Stanton and KliotSparse Cut Projections in Graph Streams, by Das Sarma, Gollapudi and PanigrahyThese papers are all sufficiently different than the present paper that there is novelty in the present paper and no overlap.  But they illustrate that it much more accurate to describe the present paper as an interesting but relatively minor improvement on recent work on stochastic blockmodeling under various well-motivated memory constraints than as the first community detection algorithms in the data stream model or as a paper that introduces the problem of community detection with partial information, both of which are claimed but neither of which is correct.


Capturing Semantically Meaningful Word Dependencies with an Admixture of Poisson MRFs
We develop a fast algorithm for the Admixture of Poisson MRFs (APM) topic model [1] and propose a novel metric to directly evaluate this model. The APM topic model recently introduced by Inouye et al. [1] is the first topic model that allows for word dependencies within each topic unlike in previous topic models like LDA that assume independence between words within a topic. Research in both the semantic coherence of a topic models [2,3,4,5] and measures of model fitness [6] provide strong support that explicitly modeling word dependencies-as in APM-could be both semantically meaningful and essential for appropriately modeling real text data. Though APM shows significant promise for providing a better topic model, APM has a high computational complexity because O(p 2 ) parameters must be estimated where p is the number of words ([1] could only provide results for datasets with p = 200). In light of this, we develop a parallel alternating Newton-like algorithm for training the APM model that can handle p = 10 4 as an important step towards scaling to large datasets. In addition, Inouye et al.
[1] only provided tentative and inconclusive results on the utility of APM. Thus, motivated by simple intuitions and previous evaluations of topic models, we propose a novel evaluation metric based on human evocation scores between word pairs (i.e. how much one word "brings to mind" another word [7]). We provide compelling quantitative and qualitative results on the BNC corpus that demonstrate the superiority of APM over previous topic models for identifying semantically meaningful word dependencies. (MATLAB code available at:

The paper addresses the computational challenge for a recently proposed admixture models of Poisson MRFs to model word dependencies in topic models. The authors proposed a parallelizable Newton-like algorithm which would speed up the computation. Although the paper is technically sound, I do have couple of concerns regarding to the proposed method and experiment results.- The paper would be much stronger if comparing admixture model with Poisson MRF with some other models. Many other topic models also model the correlation on either document and/or words. For example, hierarchical topic models with nested CRP by Wang et al. models the hierarchies on the topics, hence the correlations among topics and words. Boyd-Graber et al. also proposed topic models for word sense disambiguation using WordNet. Comparison against these models would be a much stronger evidence for the model.- This paper uses the evocation evaluation metric. It would be great to include some other standard evaluation of correlations, such as PMI (Newman, et al.) of coherence score (Mimno, et al.).- One other big concern I have regarding to the model is the scalability of the algorithm. The experiments are running on only one dataset, and it is relatively small, e.g., the BNC dataset contains less than 5k documents, with about 1500 word types only. Even though, from the experiment results, it seems like the model took around half an hour to train, after parallelization. It would be great to demonstrate that the model is able to scale to much larger datasets and hence, be potentially more useful to the research community. In addition, the authors conduct experiments only on the BNC dataset, it would be great to include experiments evaluation on other larger datasets to demonstrate the model is efficient and effective.The paper is considerably well written, and well organized, except couple of comments.- The paper heavily rely on the knowledge of the recent proposed admixture model of Poisson MRFs, which is a very strong assumption about the readers. Authors should include more explanation about the background knowledge. - Authors could draw connections with other topic models extensions for correlation modeling.In terms of significance, the paper address the scalability issue of a recent proposed admixture model of Poisson MRFs, but the experimental results seem less convincing. Author should also focus on the scalability of the model, and examine the algorithm on a much larger dataset to demonstrate the effectiveness and efficiency of the model. Without these, it is hard to see whether the paper would make a significant impact to the research community.%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%After check Authors' feedback:One of my biggest concern is the real scalability of this algorithm. The authors scaled out the inference, but the experiments with only one small dataset are not convincing to show the scalability, which make me doubt how well the proposed algorithm can scale up.In the author's feedback, they provide a little bit more results with 10k vocab. This is a reasonable size. If they integrate the new results to final version and release code, this is more convincing to me now. But on the other hand, the evocation evaluation should be better explained and supported, and it would be great if standard evaluation scores like PMI are included as well.


Algorithms for CVaR Optimization in MDPs
In many sequential decision-making problems we may want to manage risk by minimizing some measure of variability in costs in addition to minimizing a standard criterion. Conditional value-at-risk (CVaR) is a relatively new risk measure that addresses some of the shortcomings of the well-known variance-related risk measures, and because of its computational efficiencies has gained popularity in finance and operations research. In this paper, we consider the mean-CVaR optimization problem in MDPs. We first derive a formula for computing the gradient of this risk-sensitive objective function. We then devise policy gradient and actor-critic algorithms that each uses a specific method to estimate this gradient and updates the policy parameters in the descent direction. We establish the convergence of our algorithms to locally risk-sensitive optimal policies. Finally, we demonstrate the usefulness of our algorithms in an optimal stopping problem.

This paper derives policy gradient algorithms for risk-sensitive MDPs for the particular criterion CVaR - a recent and popular criterion. First, the author derive gradients for the objective based on a Lagrangian relaxation of the constrained optimization. This naturally turns into a policy gradient algorithm where the expected return that appears in the gradient is estimated from full trajectories (reinforce-like). They then propose a scheme to obtain incremental actor-critic versions, where the critic computes the value (and other quantities) of an augmented MDP convenient for gradient estimation. An experimental section illustrates these algorithms on a small problem.The paper is well-written, and the introduction clearly motivates the paper. The paper is quite technical, but I didn't have trouble understanding the notation and following the equations/algorithms. I am only mildly familiar with the literature on risk-sensitive MDPs so I can't provide insightful comments on the impact/significance of the contributions in the paper. My understanding is that, being incremental and compatible w/ function approximation, the algorithms in the paper have the potential to scale to larger domains than what is currently the norm with risk-sensitive planning.A couple of questions:- About the dependence of the policy on the accumulated cost and time, I understood that by making the feasibility assumption then you can restrict yourself to state-only (but stochastic) policies. Is that correct? Does it mean there might be better history-based policies that satisfy the CVaR constraint but that you are not considering? It might be worth clarifying this in the paper.[EDIT: Thanks for clarifying]- About the optimized policy, does converging to a saddle point in the Lagrangian necessarily imply that we reach any kind of local optimum in the original optimization problem? If not, then is it really fair to say that the algorithms converge to "locallly risk-sensitive optimal policies" (abstract)?   - In the intro, it is claimed that [27] (the closest competitor it seems) does not have a convergence proof for their policy-gradient algorithm, but a quick look at [27] reveals the following claim: "We [...] prove the convergence of the policy gradient algorithm to a local optimum". Maybe this is worth clarifying?[EDIT: Fair enough, I didn't pay attention to the arxiv version/date of submission.]While I appreciate the theoretical contribution, I was a bit disappointed by the experimental section, which does not really convey the impression of scalability and robustness (in contrast, [27] had experiments with the tetris domain using CVaR). Future work #3 seem particularly critical for sample-based algorithms in this risk setting, so it is a pity that this wasn't addressed in this work.Overall, a nice paper with clear contributions. Experiments are lacking but there is a lot of theoretical work.Some minor typos:- Equation 4: max lambda should have a non-negativity constraint(?)- line 1313:  xi_3 --> xi_4- line 269 (footnote): SPSA not yet defined. Note: I haven't verified all the proofs in the long supplementary material provided with the paper.


Grouping-Based Low-Rank Trajectory Completion and 3D Reconstruction
Extracting 3D shape of deforming objects in monocular videos, a task known as non-rigid structure-from-motion (NRSfM), has so far been studied only on synthetic datasets and controlled environments. Typically, the objects to reconstruct are pre-segmented, they exhibit limited rotations and occlusions, or full-length trajectories are assumed. In order to integrate NRSfM into current video analysis pipelines, one needs to consider as input realistic -thus incomplete-tracking, and perform spatio-temporal grouping to segment the objects from their surroundings. Furthermore, NRSfM needs to be robust to noise in both segmentation and tracking, e.g., drifting, segmentation "leaking", optical flow "bleeding" etc. In this paper, we make a first attempt towards this goal, and propose a method that combines dense optical flow tracking, motion trajectory clustering and NRSfM for 3D reconstruction of objects in videos. For each trajectory cluster, we compute multiple reconstructions by minimizing the reprojection error and the rank of the 3D shape under different rank bounds of the trajectory matrix. We show that dense 3D shape is extracted and trajectories are completed across occlusions and low textured regions, even under mild relative motion between the object and the camera. We achieve competitive results on a public NRSfM benchmark while using fixed parameters across all sequences and handling incomplete trajectories, in contrast to existing approaches. We further test our approach on popular video segmentation datasets. To the best of our knowledge, our method is the first to extract dense object models from realistic videos, such as those found in Youtube or Hollywood movies, without object-specific priors.

The paper introduces a new method for non-rigid structure from motion (NRSfM). Existing work on NRSfM typically make assumptions about their inputs, and usually require complete point trajectories, perfect segmentations, or strong object-specific shape priors. The proposed method overcomes these limitations. The method achieves competitive results on an existing synthetic NRSfM metric, and is applied to real-world videos.Quality: Overall the quality is acceptable, but there are some concerns about the experiments that should be addressed: - The only quantitative comparison of the proposed method with existing methods is on the synthetic dataset introduced in [11], where the proposed method is provided with full-length trajectories. In this experiment the method from [11] outperforms the proposed method on all video sequences; this is somewhat concerning.- Even so, the proposed method is not directly comparable to the methods of [6], [28], and [11] since the proposed method can be used even in the presence of incomplete trajectories and imperfect segmentations. The authors claim that these benefits make their method useful for real-world video sequences, but there is no quantitative evaluation of these claims. The authors provide reconstruction results of their method on example videos from the VSB100 and Moseg datasets, and the reader is expected to qualitatively assess these results. A quantitative metric for evaluating these results would be much more convincing.Clarity: Overall the paper does a good job explaining the method and comparing it to existing work. The description of the method is mostly clear up to line 206; after that the explanation becomes more dense and harder to follow. Some specific comments:- Why are Equations 5 and 6 solved via successive subproblems? Is this standard in the literature? Would it be possible to optimize jointly over all variables using some convex relaxation of Equations 5 and 6?- Typo on line 232: W = UDV^T = UDV^T = …- The Euclidean upgrade step is confusing. This could be a result of my unfamiliarity with the literature, but this section is difficult for me to follow. Why is the rank of W truncated? Why is orthogonality of \hat{R}_{2t-1}G and \hat{R}_{2t}G desirable?- You claim that your work shows that a rank 3 decomposition of W is better than the standard rank 3K decomposition used in prior work. Do you have any explanation for why this might be the case? Do higher values of K_r lead to overfitting? If so, could this effect be mitigated somehow?- What is the intuition behind minimizing \|S^V\|_\ast? My (possibly incorrect) interpretation is that when S^V is low rank, the object can be described by a small number of planar regions which do not deform too much between frames. Is really a type of shape or deformation prior? If so how does it differ from the shape priors used in other NRSfM methods?Originality: I am not very familiar with existing SfM methods or literature, so it is hard for me to accurately assess the originality of the proposed method.Significance: Nonrigid structure from motion from real-world monocular video sequences is an an important problem. The method proposed by this paper claims to make sizeable advances toward solving this problem. If these claims are valid then this paper is quite significant.


Low-Rank Time-Frequency Synthesis
Many single-channel signal decomposition techniques rely on a low-rank factorization of a time-frequency transform. In particular, nonnegative matrix factorization (NMF) of the spectrogram -the (power) magnitude of the short-time Fourier transform (STFT) -has been considered in many audio applications. In this setting, NMF with the Itakura-Saito divergence was shown to underly a generative Gaussian composite model (GCM) of the STFT, a step forward from more empirical approaches based on ad-hoc transform and divergence specifications. Still, the GCM is not yet a generative model of the raw signal itself, but only of its STFT. The work presented in this paper fills in this ultimate gap by proposing a novel signal synthesis model with low-rank time-frequency structure. In particular, our new approach opens doors to multi-resolution representations, that were not possible in the traditional NMF setting. We describe two expectation-maximization algorithms for estimation in the new model and report audio signal processing results with music decomposition and speech enhancement.

This paper presents a extension in the family of low-rank spectrogram-factorization models of sound decomposition that generalizes the latent factors over an arbitrary (potentially overcomplete) set of bases, meaning that decomposition can be performed on "multiresolution" (e.g. tonal + transient) bases.  This is argued to help with source separation and denoising.  It starts from Gaussian Composite Model of Fevotte et al, which models the variance of each cell of a spectrogram with a low-rank matrix factorization, but then expands the implicit convolution with the multiple gabor impulse responses corresponding to each row of the spectrogram to give a purely time-domain expression for what is being estimated, that can then be modified to work for different sets of impulse responses, including nonorthogonal ones.  An EM (local optimum) analysis procedure is presented.Quality: I think this is an interesting insight, although I'm still trying to figure out the exact implications.  It seems like, at the outset, it's just another way of understanding Fevotte's GCM, but the ability to include overcomplete dictionaries is nice.  On the other hand, it seems like this ought to cause some complications, and I didn't immediately understand why these didn't cause more trouble.  I assume the estimation procedure is correct (and I'm impressed by this kind of analysis) although I couldn't follow it in detail.Clarity: The work is well explained and motivated.  The evaluation, however, was very superficial - we're given one purely illustrative result, and one SNR improvement measurement: I hardly think we can draw any conclusions from a single example.  I was grateful for the included sound examples, but I wasn't all that impressed.  For one thing, the OMLSA enhanced example seems to be much longer (7 sec c/w 2 sec) than the new result, which makes me wonder if the comparison has been done fairly.  Secondly, although OMLSA includes more background noise (in part because it is longer), the speech sounds much higher quality to me.  The denoising of the music example comes at the expense of very significant artifacts - I would expect conventional wavelet shrinkage or weiner filtering to give much higher perceived quality in this case.Originality: The work seems to rely very heavily on the Fevotte et al foundation, although the time-domain interpretation certainly made me think about it in a new way.Significance: On the face of it, the ability to apply matrix-factorization techniques to overcomplete bases seems like it could have significant impact.  However, the examples and evaluation presented in this work were too thin and superficial to really show any of this, so as it stands it is an act of faith only to rate this work as having substantial significance.


Optimization Methods for Sparse Pseudo-Likelihood Graphical Model Selection
Sparse high dimensional graphical model selection is a popular topic in contemporary machine learning. To this end, various useful approaches have been proposed in the context of 1 -penalized estimation in the Gaussian framework. Though many of these inverse covariance estimation approaches are demonstrably scalable and have leveraged recent advances in convex optimization, they still depend on the Gaussian functional form. To address this gap, a convex pseudo-likelihood based partial correlation graph estimation method (CONCORD) has been recently proposed. This method uses coordinate-wise minimization of a regression based pseudo-likelihood, and has been shown to have robust model selection properties in comparison with the Gaussian approach. In direct contrast to the parallel work in the Gaussian setting however, this new convex pseudo-likelihood framework has not leveraged the extensive array of methods that have been proposed in the machine learning literature for convex optimization. In this paper, we address this crucial gap by proposing two proximal gradient methods (CONCORD-ISTA and CONCORD-FISTA) for performing 1 -regularized inverse covariance matrix estimation in the pseudo-likelihood framework. We present timing comparisons with coordinate-wise minimization and demonstrate that our approach yields tremendous payoffs for 1 -penalized partial correlation graph estimation outside the Gaussian setting, thus yielding the fastest and most scalable approach for such problems. We undertake a theoretical analysis of our approach and rigorously demonstrate convergence, and also derive rates thereof.

This paper addresses the use of pseudo-likelihood for solving (sparse) Graphical Model selection problems. The use of log likelihood for Gaussian Graphical Models has been the subject of considerable research, but this is limited to the Gaussian assumption. This has not been the case for pseudo-likelihood –based methods, and non-Gaussian graphical models. Since the Gaussian assumption is limiting, there are potentially large benefits in this line of work. At a high-level the formulation is based on a recent approach called CONCORD, which optimizes Qcon, a convex pseudo-likelihood objective. CONCORD uses coordinate-wise descent. The paper proposes proximal gradient techniques to minimize Qcon  and provides a theoretical analysis. Proximal gradient has become a common useful tool in machine learning. The methods are tested in learning non-Gaussian graphical models.The paper derives convergence rates for CONCORD-ISTA/-FISTA which is of significance, it also compares CONCORD-ISTA/FISTA and coordinate-wise minimization. Results suggest that CONCORD-ISTA is much faster than coordinate-wise, specially in high-dimensions. Since their computational complexity is similar O(p^2), a primary reason for this is the possibility of parallelizing the computation in the proposed method, while this is not the case for the coordinate-wise method. The derived converge rates are not fully consistent across pretty much all of the results. It would be great to have further discussion on this.


Scalable Kernel Methods via Doubly Stochastic Gradients
The general perception is that kernel methods are not scalable, so neural nets become the choice for large-scale nonlinear learning problems. Have we tried hard enough for kernel methods? In this paper, we propose an approach that scales up kernel methods using a novel concept called "doubly stochastic functional gradients". Based on the fact that many kernel methods can be expressed as convex optimization problems, our approach solves the optimization problems by making two unbiased stochastic approximations to the functional gradient-one using random training points and another using random features associated with the kernel-and performing descent steps with this noisy functional gradient. Our algorithm is simple, need no commit to a preset number of random features, and allows the flexibility of the function class to grow as we see more incoming data in the streaming setting. We demonstrate that a function learned by this procedure after t iterations converges to the optimal function in the reproducing kernel Hilbert space in rate O(1/t), and achieves a generalization bound of O(1/ √ t). Our approach can readily scale kernel methods up to the regimes which are dominated by neural nets. We show competitive performances of our approach as compared to neural nets in datasets such as 2.3 million energy materials from MolecularSpace, 8 million handwritten digits from MNIST, and 1 million photos from ImageNet using convolution features.

The paper proposes a large-scale kernel machine variant based on a combination of stochastic functional gradient descent and kernel random features. It is shown that the resulting estimate of the gradient is unbiased, thereby opening the door to convergence proofs. Generalization bounds are given, and the paper concludes with collection of experiments mainly involving datasets enlarged with virtual examples.The clarity, organization, and flow of the paper is good (although there are some grammatical errors that should be corrected).  It is clear the authors gave some thought to their presentation.  A noteworthy aspect of this submission is that it is thorough, particularly in light of the space constraints. The authors explain where the problem is, how their proposed solution addresses those problems, provide some theory to ensure the suggested algorithm is reasonable, and test their ideas on multiple, relatively realistic datasets against some existing alternatives. The theoretical component may, perhaps, be seen as among the more interesting contributions of the paper (more on why below). The observation that one need only store a random seed and a pseudo-RNG routine to reproduce a sequence of random projections is a nifty one. The experiments constitute a decent illustration of the method, and the authors should be commended for providing the additional experimental detail given in the supplementary material. A concern with the paper is a possible lack of novelty and significance. Fundamentally, the main idea is to combine two pre-existing ideas: functional gradient descent and random features approximating a (shift-invariant) kernel. Large n is the target problem addressed by both, but the key motivation for using kernel random features, and for increasing complexity, is to allow to reduce test-time storage by exchanging memory for additional computation. However, the paper should address the following  important questions around this reasoning:- It is true that testing requires minimal storage, under the assumption that you only really need to store the RNG seed from which one can then re-generate the random projections assumed during training. But this is also true for the vanilla random features approach anyways. - The proposed method exchanges low test-time memory footprint for a potentially huge, redundant computational burden associated with repeated random number generation and projection steps. The regime where this becomes acceptable could be for very large n indeed, particularly since random projections are computed anew during training iterations. It could be argued, even without random projections of any kind, that for all but the largest training sets, the test time storage burden is less of a concern. O(n) -- a single pass over the data -- may not be all that bad for many large applications. If random features are used, then a similar argument might be made for storing the projections. - The functional gradient descent approach comes with its own drawbacks. It may only be useful for more exotic loss functions, since if we use random features alone in the setting where (n is much larger than p), the square loss or hinge loss will result in a very small linear problem with random features alone. SGD may not be necessary.In light of these questions, the method may not have the strongest practical implications. The theory, if correct, could however serve as an interesting illustration of martingale methods, since the (mathematical) problem is clear, and the goals are relatively self-contained.


Sparse space-time deconvolution for Calcium image analysis
We describe a unified formulation and algorithm to find an extremely sparse representation for Calcium image sequences in terms of cell locations, cell shapes, spike timings and impulse responses. Solution of a single optimization problem yields cell segmentations and activity estimates that are on par with the state of the art, without the need for heuristic pre-or postprocessing. Experiments on real and synthetic data demonstrate the viability of the proposed method.

Comments:This paper presents a complete method for calcium imaging analysis with few user-adjustable parameters.  This high-quality work is clearly written and significant to a large number of researchers working with calcium imaging data, and the chosen approach of simultaneously optimising all parameters of the analysis is original.  There are a few issues that the authors should address to finalise this work.Major issues:1) Why is the limit on the number of types of impulse responses also K, the number of cells?  More importantly, why do the authors choose only 2 impulses for the analysis?  It seems quite clear from Figure 5 that the two filters are the calcium indicator responses to one or multiple action potentials.  Either reducing the number of time points for impulses (i.e. reduce F) or increasing the number of possible impulses (increase L) would be wise in order to capture the range of actual impulses that are observed in the fluorescence response to the many action potentials that may occur in the 3 second long time window that is currently used.2) One of the key advantages of the proposed method is the apparent reduction in the amount of “fiddling” necessary to go from raw calcium imaging data to observed events in cells in known positions.  The authors admit “the price to pay for this elegance .. is a much higher computational cost” without going into detail.  If this method is to be useful to researchers besides the authors, a detailed quantification of this “price” is needed.  Can this method be run in a reasonable amount of time on a decent PC, or is a computational cluster required?  How much longer does this optimisation take compared to the other methods the authors test?3) Figure 5 is difficult to interpret.  First of all, the learned “truly sparse” representation cyan and magenta symbols are difficult to see even when zooming in digitally, and they appear on both left and right panels of traces even though the legend claims they are on the right. Additionally, how are there are what appear to be signals in the right panel in locations without symbols?  Lastly, this data appears to come from a paper in which cell-attached recordings were performed.  If at all possible, the best possible test would be to see whether the “truly sparse” impulse times match actual recorded action potentials. Of course, the cell-attached data in the cited paper is zoomed in tightly on the recorded neuron, so the analysis problem is much reduced, but if any other data were available along these lines, that could lead to a highly calibrated method that would be used by many researchers. Minor issues:Please do not capitalise Calcium imaging.Page 1 line 040 “only a fraction of neuron fires” should be “only a fraction of neurons fire”Page 3 line 137 It was unclear until nearly the end of the paper exactly what an “impulse” or “pattern” of activity would like, perhaps the authors could plot one of the filters from the final figure earlier on or give a more precise description at the first usage of the term.  The first mention of the length of these ‘patterns’ says “the number of taps or coefficients F that the learned impulse responses may have”.  What is meant by “taps” ?Page 3 line 139 “six .. parameters”  I count 5 (K,H,J,L,F) unless the authors are also counting the background smoothness parameter described in the next section.Page 5 line 258 typo “we disregards”Page 5 line 263 please italicise priori if also italicising “a”Page 7 line 356 typo “SSTD does as well the”  should be “as well as the”


Self-Adaptable Templates for Feature Coding
Hierarchical feed-forward networks have been successfully applied in object recognition. At each level of the hierarchy, features are extracted and encoded, followed by a pooling step. Within this processing pipeline, the common trend is to learn the feature coding templates, often referred as codebook entries, filters, or over-complete basis. Recently, an approach that apparently does not use templates has been shown to obtain very promising results. This is the second-order pooling (O2P) [1,2,3,4,5]. In this paper, we analyze O2P as a coding-pooling scheme. We find that at testing phase, O2P automatically adapts the feature coding templates to the input features, rather than using templates learned during the training phase. From this finding, we are able to bring common concepts of coding-pooling schemes to O2P, such as feature quantization. This allows for significant accuracy improvements of O2P in standard benchmarks of image classification, namely Caltech101 and VOC07.

The paper aims to provide a more general interpretation of a recent class of `tensor representations’ based on second order information. In particular the authors provide derivations in support of the claim that such second order pooling methodologies can be viewed as coding-pooling schemes that adapt the feature coding pattern to the input features. Moreover, the authors show that by incorporating coding into the new interpretation some improvements can be achieved in benchmarks (N.B. the results on VOC07 are significant; however, the results on Caltech101 by no means represent a significant improvement over the state of the art).In general this is an interesting, clearly written paper (modulo some issues, having to do with the presentation of the contributions, described below) which provides valuable insight into a class of successful, emerging techniques. Overall I find this paper to be relevant and worth publishing at NIPS. I would appreciate if the authors considered the following comments.- CommentsI would say that some of the main contributions associated with the (second-order) tensor representation are probably more attributable to [1],[16], than to [1],[2] (line 133). Equally so, line 142 should read `…celebrated Log-Euclidean metric [4,5]’ instead of just [4] as these have been introduced in the vision context simultaneously at ECCV12. Overall I would make it clear also that [3] uses a different metric than [4,5], for example. That’s why the approach authors take in baptizing slightly different methods as TR could be misleading in places like Table 1 or 2. Moreover, as far as I can understand the various approaches denoted Pyr TR w/o SQ = O2P [5]. Particularly as some of the code in [5] appears to be used (do the authors use the enriched SIFT features of [5] in their experiments with Caltech 101, or in the coded features they propose?), it would be appropriate to not `remove all traces’ and present existing methods as if they were proposed by the authors, under a new name. Best would be to conserve the original names and use new ones only when new elements are proposed by the authors, e.g. Pyr. Tr + SQ.The list of references is good. The authors can consider adding: The Devil Is in the Details: an Evaluation of Recent Feature Encoding Methods: Chatffiled et al., BMVC 2011.Efficient Match Kernels between Sets of Features for Visual recognition, Bo et al., NIPS 2009.which also discuss different encoding strategies (former) and provide potentially useful kernel interpretations for bag of words models (latter).


Extended and Unscented Gaussian Processes
We present two new methods for inference in Gaussian process (GP) models with general nonlinear likelihoods. Inference is based on a variational framework where a Gaussian posterior is assumed and the likelihood is linearized about the variational posterior mean using either a Taylor series expansion or statistical linearization. We show that the parameter updates obtained by these algorithms are equivalent to the state update equations in the iterative extended and unscented Kalman filters respectively, hence we refer to our algorithms as extended and unscented GPs. The unscented GP treats the likelihood as a 'black-box' by not requiring its derivative for inference, so it also applies to non-differentiable likelihood models. We evaluate the performance of our algorithms on a number of synthetic inversion problems and a binary classification dataset.

The authors present a simple idea for approximating the variational lower bound on the marginal likelihood for GPs in the case of an intractable likelihood p(y | f) with a particular form ---  the observations y equal an arbitrary nonlinear function g(f) with additive Gaussian noise. The lower bound requires solving an expectation involving the nonlinear function g(f), which can't be done analytically. The authors suggest linearising the function about the current estimate of the posterior mean on f. The paper contains two methods for performing the linearisation: by differentiation of g (as in the extended Kalman filter), and by use of `sigma points' (as in the unscented transform). The authors show how variational inference can then be carried out with their approximations along with learning of the GP hyperparameters, and making predictions at new test points. Empirical evaluation is provided by using their method for regression with 4 different nonlinear functions g(f), and for classification on the USPS dataset using a logistic likelihood.Although the idea is fairly simple it is still a worthy addition to the family of approximate GP inference methods for intractable likelihoods. By approximating the lower bound the method does lose many of the nice statistical properties of variational inference and the paper could benefit from some discussion of this.The paper is mostly well written and provides a clear exposition. I would like the authors to provide some disambiguation around the terms `input' and 'inverse model' however: in section 2 the latent variable f is referred to as the input, whereas in section 3 we now have x which is the input. This is compounded by the discussion of inverse models, by which the authors mean the mapping from the observations y to the latent variable f but which could easily be understood in the GP setting to mean the mapping from the `output' f (or y) to the `input' x. Other than this I find the paper to be eminently understandable.The experiments seem a little suspect to me. There is very little discussion of the results presented in table 1, which the paper would benefit from as very few of the results appear to be significant, given the quoted standard deviations --- for NLL `exp(f)' and `sin(f)' might be significant but the polynomial and tanh functions don't appear to be. Is this also the case for the results in table 2? The error bars are not reported here. The authors should also compare, both theoretically and experimentally, to the variational classification method implemented in the GPML toolbox for the logistic likelihood.


Inference by Learning: Speeding-up Graphical Model Optimization via a Coarse-to-Fine Cascade of Pruning Classifiers
We propose a general and versatile framework that significantly speeds-up graphical model optimization while maintaining an excellent solution accuracy. The proposed approach, refereed as Inference by Learning or in short as IbyL, relies on a multi-scale pruning scheme that progressively reduces the solution space by use of a coarse-to-fine cascade of learnt classifiers. We thoroughly experiment with classic computer vision related MRF problems, where our novel framework constantly yields a significant time speed-up (with respect to the most efficient inference methods) and obtains a more accurate solution than directly optimizing the MRF. We make our code available on-line [4].

This paper combines a coarse-to-fine cascade with pruning classifiers to speed up the optimization of the MRF. Although the greedy method that is used to prune the solution space seems interesting, the paper does not have sufficient theoretical and empirical evidence to support the argument that the heuristic works. Also, the type of MRFs which may benefit from the method is limited to the ones whose solutions are piecewise smooth. Some part of the paper is not clear enough. For example, in the description of the pruning matrix, it is not clear enough what "active" means. The pruning matrix updating of the algorithm is also not very clear. I would suggest the author provide a toy example to illustrate the algorithm better. On the other hand, the proposed approach part of sec 1 is too long and some contents may be redundant. Figure 2 is also not very informative. The experiment is interesting but also not clear enough. First of all, there are three algorithms which are compared, but there is only one curve shown in fig 1 (Since there are three algorithms in comparison, only showing the ratio is not sufficient). Secondly, since the energy ratio is between the current energy and the lowest computed energy by any strategy, why is it less than 1 in many cases? ** After author feedback: The authors clarify some of the confusions in the original paper. I expect the authors to correct all the typos and address all the concerns by the reviewers. I move my rating from 4 to 5, because there is still no theoretical justification for the pruning method. A few more comments about the experiments: - In column(d) of fig 1, both two curves in the 3rd row have less than 96% agreement, while in the paper it says "the agreement is never worse than 96%". - Consider moving fig 2 to appendix. Add more experiments or analysis on hyperparameter choosing.


From MAP to Marginals: Variational Inference in Bayesian Submodular Models
Submodular optimization has found many applications in machine learning and beyond. We carry out the first systematic investigation of inference in probabilistic models defined through submodular functions, generalizing regular pairwise MRFs and Determinantal Point Processes. In particular, we present L-FIELD, a variational approach to general log-submodular and log-supermodular distributions based on sub-and supergradients. We obtain both lower and upper bounds on the log-partition function, which enables us to compute probability intervals for marginals, conditionals and marginal likelihoods. We also obtain fully factorized approximate posteriors, at the same computational cost as ordinary submodular optimization. Our framework results in convex problems for optimizing over differentials of submodular functions, which we show how to optimally solve. We provide theoretical guarantees of the approximation quality with respect to the curvature of the function. We further establish natural relations between our variational approach and the classical mean-field method. Lastly, we empirically demonstrate the accuracy of our inference scheme on several submodular models.

This paper investigates an interesting idea of approximating the normalization constant (thereby computing various inference quantities like marginals, conditional distributions etc.) for the general class of log-submodular and log-supermodular distributions.On the positive side, I thought the idea of this paper was neat -- to use the semigradients to provide upper/lower bounds of the functions, and compute the constants for these bounds. In this way, they leveraged a lot of the existing results concerning approximation bounds, to measure the tightness of the approximation.A main drawback of this paper is that the results seem theoretically pretty weak, given that the main contribution of this paper is theoretical. Theorem 2 is, in my opinion, the only result which talks about the tightness of the approximation, and provides a factor, which is in the worst case O(n). I wonder if there is a notion of hardness factor in computing the normalization factor (using, for instance, the construction of [Balcan and Harvey, 2009]). In my opinion, the paper is somewhat incomplete without a discussion on this, or even possible directions of computing the normalization factor. Without a notion of hardness, the worst case factor can look very discouraging, and there is no sense of how hard this problem can be (apart from the #P-hardness result, which is stated in lines 150 -- it would be nice to have a reference or a proof of this). Can one say something like, no polynomial time algorithm can compute Z up to a factor of p(n), where p(n) is a polynomial in n?I would also have loved to see some more discussion on the utility of computing these quantities. For instance, does this give method to approximately sample from a log-submodular distribution? Does this give guarantees in terms of learning mixtures of submodular functions? It seems to me that it would not be hard to show guarantees for Maximum-Likelihood learning of mixtures of submodular functions under this model. Consider the following setup. We have a mixture of submodular functions, and define p(X) \propto \exp(\sum_i w_i f_i(X)). One could then do approximate max-Likelihood learning to learn the weights w. This might be an interesting application of the framework. It would also be nice to see some stronger empirical utility of these models on real world problems, like for e.g. summarization. Does this framework admit sampling algorithms from submodular functions? On a whole, the paper misses some important aspects of this story. I would really love to see more discussion on this.