General Table Completion using a Bayesian Nonparametric Model
Even though heterogeneous databases can be found in a broad variety of applications, there exists a lack of tools for estimating missing data in such databases. In this paper, we provide an efficient and robust table completion tool, based on a Bayesian nonparametric latent feature model. In particular, we propose a general observation model for the Indian buffet process (IBP) adapted to mixed continuous (real-valued and positive real-valued) and discrete (categorical, ordinal and count) observations. Then, we propose an inference algorithm that scales linearly with the number of observations. Finally, our experiments over five real databases show that the proposed approach provides more robust and accurate estimates than the standard IBP and the Bayesian probabilistic matrix factorization with Gaussian observations.

In the manuscript "General Table Completion using a Bayesian Nonparametric Model" the authors propose a method for database completion using the Indian buffet process (IBP) which is a Bayesian nonparametric prior on binary matrices with a fixed number of rows and an unbounded number of columns (which correspond to features of the data associated with the rows). The authors suppose that the columns of the database fall into the following 5 classes: reals, nonnegative reals, discrete, ordinal or integer. For each of these classes, the authors provide a latent IBP model linking the entries of the database with latent features for each row. They find the latent features and likelihood parameters using standard MCMC methods for the IBP and apply their model to predicting missing entries in 5 diverse datasets ranging in size from 20k to 2m entries. This model is important for the NIPS community because it constitutes a very general application of Bayesian nonparametrics. Recently in the machine learning community there has been lots of interest in automatic statistics (see Lloyd et al. "Automatic construction and natural-language description of nonparametric regression models" 2014).In each of the 5  classes, the latent features interact with the likelihood through a Gaussian random variable which is passed through a link function. The covariance matrix of the Gaussian random variable is learned, as are the parameters of the link functions. As the authors point out, this approach is not incredibly innovative (the IBP has been used to model each of these classes separately before, as can be seen by the citations in the manuscript). The novel aspect of the authors' work is a uniform framework in which all of these classes are combined in order to do inference on an arbitrary database. Further, in the experiments I think that the authors give quite good results defending the idea that adding the link functions (i.e., treating the classes differently instead of just versions of Gaussian random variables that just happen to be observed at positive or at integral values) really does add more predictive accuracy.  Another innovative aspect of the manuscript was in the Gibbs updates, which build upon the IBP sampler from Chu and Ghahramani "Gaussian processes for ordinal regression" (2005) which marginalizes over the latent feature assignments of the data items. The authors extend this method to also marginalize over the weighting factors of the features, improving the efficiency of the algorithm.Overall, I thought that this manuscript was good and that it should be accepted. The datasets they applied their model to are new and their results do show that learning link functions for IBP based methods improves upon standard Gaussian models. I did, however think that they should have compared with more methods. For example, they could have compared with methods based on Dirichlet process mixtures such as CrossCat (the likelihoods and link functions used in this paper could also be used to augment the CrossCat model (Shafto et al. 2011) in a way such that it becomes appropriate for the same sort of mixed-class matrices).This manuscript still left some questions about inference based on the IBP model open: the IBP has symmetries arising from simultaneous transformations of the latent feature weights and also the feature assignments. For example, if the sign of all the entries in the feature weight matrix change, and if the feature matrix is simultaneously inverted (so 0s go to 1s and 1s go to 0s) then the likelihood of the data remains unchanged. Because modes in MCMC are attractive, this leads to slower mixing when the MCMC state is `the same distance' between two equivalent modes, it is not pulled strongly to either of them. Unfortunately, the inference method used by the authors did not address this.In the introduction of the manuscript, the authors mention that Bayesian methods can be used to detect errors in databases (i.e., entries where there has been some corruption or mistake in data entry). But I found that they did not actually explore this in the paper. I do think that this is an excellent direction for Bayesian nonparametrics so I was disappointed that it wasn't picked up upon again in the manuscript after the introduction. They could also look at detection of the class of a column, which might not always be obvious by looking at the entries of the column alone. For example, in a database it might turn out that the discrete classes of a column actually have an ordinal relation. Also, a bunch of -9.0000's in a column of real numbers might suggest that those entries are actually missing. Or, a column with both marked missing entries and entries with 0.0000, might actually have the 0.0000 entries through a default, missing value also.I felt like the experiments conducted by the authors demonstracted clearly that using link functions over improves the accuracy. However, I did wish that the authors explored the inference on the experiments more. For example, why does the SIBP (which is an IBP with purely Gaussian assumptions) method outperform the author's method for the `internet usage' dataset? Also, by examining the columns in Figure 3(c) for example, it is clear that there is some structure in the order in which the columns are registered in the database. Why does the BPMF method perform poorly on the first and last columns of the Nesarc dataset?


Optimal Regret Minimization in Posted-Price Auctions with Strategic Buyers
We study revenue optimization learning algorithms for posted-price auctions with strategic buyers. We analyze a very broad family of monotone regret minimization algorithms for this problem, which includes the previously best known algorithm, and show that no algorithm in that family admits a strategic regret more favorable than Ω( √ T ). We then introduce a new algorithm that achieves a strategic regret differing from the lower bound only by a factor in O(log T ), an exponential improvement upon the previous best algorithm. Our new algorithm admits a natural analysis and simpler proofs, and the ideas behind its design are general. We also report the results of empirical evaluations comparing our algorithm with the previous state of the art and show a consistent exponential improvement in several different scenarios.

This paper studies an established model of repeated posted-price auctions with a single, strategic agent with an unknown value for a good. The algorithm posts a price at each time step, and the agent makes a purchasing decision based on his surplus from the current stage and the effect of his purchasing decision on future prices. In particular, the agent may strategically choose to not buy the good in the hopes of lowering the prices offered in future rounds. The authors extend an existing price-exploration algorithm for the non-strategic buyer case to this strategic case and show that their algorithm achieves near optimal strategic regret. They provide limited empirical evaluation to demonstrate superior performance over the previous work in this model. This is a good paper and is a valuable contribution to the literature at the intersection of machine learning and algorithmic game theory. The techniques, and the associated technical exposition, are very clean and intuitive. Throughout the paper, the authors do a good job of providing the high level ideas and also do a good job of explaining the relationship between their methods and previous work. While the model studied might not be the most realistic one for the context of repeated ad auctions, the techniques and analysis in this paper are likely to helpful in the study of more realistic and complex models. One particularly nice aspect of the approach in this paper is that the technique feels like it can be used again in similar settings. As is noted, the discount rate of the buyer is really the only aspect of this model that a designer can use to enforce good incentive properties and the algorithm in this paper seems like a general technique for leveraging the discount rate in repeated games, e.g. in a model where buyers don’t have an explicit discount rate but have some sort of ad impression budget that they must satisfy before the end of the game.  The only minor complaint I have is regards the knowledge of the discount rate. While knowledge of such parameters is standard in theoretical models, robustness to such an assumption is a desirable property (e.g. the literature on prior-free mechanism design). The authors do show a few experimental examples where their method performs well even without knowledge of the discount rate but I didn’t find that section to be particularly convincing. Perhaps more extensive empirical work could have made this point better (although that would have been difficult due to space constraints). Still this is only a minor point. Side Notes: - In figure 2, it is not immediately obvious that the y-axis is on a different scale. So it initially appeared that the regret with a known discount parameter was actually higher than the unknown.


Inferring sparse representations of continuous signals with continuous orthogonal matching pursuit
Many signals, such as spike trains recorded in multi-channel electrophysiological recordings, may be represented as the sparse sum of translated and scaled copies of waveforms whose timing and amplitudes are of interest. From the aggregate signal, one may seek to estimate the identities, amplitudes, and translations of the waveforms that compose the signal. Here we present a fast method for recovering these identities, amplitudes, and translations. The method involves greedily selecting component waveforms and then refining estimates of their amplitudes and translations, moving iteratively between these steps in a process analogous to the well-known Orthogonal Matching Pursuit (OMP) algorithm [11]. Our approach for modeling translations borrows from Continuous Basis Pursuit (CBP) [4], which we extend in several ways: by selecting a subspace that optimally captures translated copies of the waveforms, replacing the convex optimization problem with a greedy approach, and moving to the Fourier domain to more precisely estimate time shifts. We test the resulting method, which we call Continuous Orthogonal Matching Pursuit (COMP), on simulated and neural data, where it shows gains over CBP in both speed and accuracy.

"Inferring sparse representations of continuous signals with continuous orthogonal matching pursuit" is a nice, compact, well-written paper on a method that should be of relevance for anyone dealing with spike sorting. The authors build up on work of Ekanadham and all (2011a,b) by considering an alternative basis for decompositions, and a different optimisation method. They provide comparison with previous results and report significant improvement in computation time, and evidence for increased accuracy of detection and estimation of waveform parameters.One question about the comparison - in order to be fair to the older method, the authors pruned waveforms with amplitude <.3. Was that value optimized for (was it equal to the lower bound on a in COMP case)? Could CBP score better, if that parameter was further increased? How was lambda of the L1-penalty chosen?Also - is it possible to plot hits and misses in separate plots? Or are these not telling more about the nature of typical errors of various methods?Are all "hit errors" of COMP calculated from the "Fourier" method?Figure 1 Might be a bit confusing to the reader - having observed the original waveform, I expected to see the reconstructed waveform. It took me a while to realise the black shapes in panels 2-4 were not the reconstructions. I would suggest to at least expand the legend.The average errors provided in the legend seem to be much lower than values plotted in the figure.    Since the paper is so well written, I include suggestion to double-check lines:41 (spurious "the")85 (spurious "here")153 (theta missing from the argument of sinus)299/300 please, correct all indices (j' and n are swapped, (i,j) came out of nowhere)311 and 315 - indices swapped411 eps=1 - do you mean 1 Delta?


SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.

This important paper introduces an incremental gradient method called SAGA for minimizing f+h when f is an average of n functions f_i (like ERM) and h is a possibly non-smooth regularizer (for which we know a prox operator), that has provable linear convergence rates when f (or each f_i according to the case) is strongly convex and the same algorithm without a change has a 1/k rate when strong convexity is absent, hence allowing it to adapt to the level of strong convexity in the problem, automatically interpolating between the rates.One of the important contributions of the paper is to show the similarities between a vast array of existing algorithms for this problem like SAG (which did not provably handle prox operators and was non-adaptive to strong convexity), and SDCA (developed over the last few years), and more recent algorithms from the past year like MISO and SVRG. While the relationship to SAG is rather straightforward (as hinted by the name SAGA), it was very interesting to see the connections to MISO and SVRG and even more so SDCA. One of the most interesting features of the paper was its ability to verbally explain why this method works well and how it relates to other methods. I found it rather interesting that it uses a biased gradient unlike SAG which results in a reduced variance. It is also interesting how SVRG is very similar to SAGA but trades off computation for space by not storing and updating the \phi vectors frequently. The relation to MISO/Finito is also interesting and not direct. Lastly, viewing the SDCA algorithm in the primal might also be of independent interest and even then it is unclear that there is any relation between the algorithms, so writing it in the same form as MISO is very interesting.The main contribution of the paper is its simple and clear proofs of convergence (as compared to SAG!) for both the strongly convex and the non-strongly convex case in the appendix, which will be of use in understanding these algorithms as well as designing new ones. Since it is adaptive to the presence or absence of strong convexity, it seems practically extremely useful when it is not known what the strong convexity constant is.Because of the importance of these kinds of problems in ML (ERM with regularizers with or without strong convexity), and the comparison to existing methods, I think this is a very important paper to many people in the community (while optimization people might be interested in the proofs, it also seems very practically applicable for everyone else). Possible typos - It seems to me like the "s" in many places (description of MISO and in proof of Lemma 1) should actually be "mu". In the appendix "manor" should be "manner".


Discrete Graph Hashing
Hashing has emerged as a popular technique for fast nearest neighbor search in gigantic databases. In particular, learning based hashing has received considerable attention due to its appealing storage and search efficiency. However, the performance of most unsupervised learning based hashing methods deteriorates rapidly as the hash code length increases. We argue that the degraded performance is due to inferior optimization procedures used to achieve discrete binary codes. This paper presents a graph-based unsupervised hashing model to preserve the neighborhood structure of massive data in a discrete code space. We cast the graph hashing problem into a discrete optimization framework which directly learns the binary codes. A tractable alternating maximization algorithm is then proposed to explicitly deal with the discrete constraints, yielding high-quality codes to well capture the local neighborhoods. Extensive experiments performed on four large datasets with up to one million samples show that our discrete optimization based graph hashing method obtains superior search accuracy over state-of-the-art unsupervised hashing methods, especially for longer codes.

Summary:  This paper proposes an approach to learn r-bit binary hash codes for a given dataset, that are smooth with respect to an affinity graph. Unlike spectral hashing, the method does not relax the binary-value constraints on the hash codes upfront, and uses an approximate affinity graph based on a small subset of anchor points. After some transformations, the resulting optimization problem is solved using alternating updates, for which monotonic descent and convergence is established.Pros:- Detailed empirical comparisons on multiple datasets against a large set of alternative hashing techniques show that the proposed method consistently performs best. - Despite being a nasty-looking discrete optimization problem, the authors are able to derive relatively simple alternating update rules for which some notion of convergence can apparently be established.Cons:- The algorithm has a large number of hyper-parameters (excluding bit length): s,m in Anchor graph construction, rho, and number of iterations in the alternating updates. It is not really made clear how the performance varies with respect to choices for these hyper parameters for a given new problem.- The notion of convergence in Theorem 1 is not made clear. A proof should be sketched.- Ultimately, the method is based on non-convex optimization and the quality of the solution cannot be guaranteed.


Scalable Methods for Nonnegative Matrix Factorizations of Near-separable Tall-and-skinny Matrices
Numerous algorithms are used for nonnegative matrix factorization under the assumption that the matrix is nearly separable. In this paper, we show how to make these algorithms scalable for data matrices that have many more rows than columns, so-called "tall-and-skinny matrices." One key component to these improved methods is an orthogonal matrix transformation that preserves the separability of the NMF problem. Our final methods need to read the data matrix only once and are suitable for streaming, multi-core, and MapReduce architectures. We demonstrate the efficacy of these algorithms on terabyte-sized matrices from scientific computing and bioinformatics.

This paper describes a non-negative matrix factorization for tall and skinny matrices.  This algorithm works in the bigdata scenario because it need only pass over the tall skinny matrix one time.  This linear read of the tall skinny matrix does not fully utilize the distributed mapreduce framework.  I wonder, is it possible to parallelize the reading of the matrix and combine the results from subsets of the data into one final result?  Perhaps this is already being done, but is buried in the details of TQSR and not mentioned.  In figure 1, r=10 is chosen based on the GP curve, and then used to compare all 3 methods.  A fairer approach would be to choose the "elbows" for each method independently.  I'm also unclear on what the right hand subfigure of Figure 1 refers to.  The text says the lines correspond to the rows, but the legend says they correspond to columns.  From a readability perspective, this paper could use some effort towards cohesion.  Each section reads well, but it is harder to put the sections together to make a coherent story.  For example, name your new extension and mention it early.  Perhaps describe the content of the paper (i.e. a roadmap) in the introduction and remind the reader of where you are in each section.If I understand correctly, you are applying your new technique to three existing algorithms.  But there is no comparison (in either reconstruction error or time to compute) of your extension vs without your extension.  For this reason it is unclear to me what the contribution of this paper really amounts to.  You state that these results constitute the largest NMF to date, which is impressive, and so perhaps none of the 3 algorithms could be run on these data sets.  If true, I suggest and exposition of run time on  smaller simulated data sets to really illustrate the advantage your algorithm affords.Minor comments:line 253 simultaneous -> simultaneously


A Complete Variational Tracker
We introduce a novel probabilistic tracking algorithm that incorporates combinatorial data association constraints and model-based track management using variational Bayes. We use a Bethe entropy approximation to incorporate data association constraints that are often ignored in previous probabilistic tracking algorithms. Noteworthy aspects of our method include a model-based mechanism to replace heuristic logic typically used to initiate and destroy tracks, and an assignment posterior with linear computation cost in window length as opposed to the exponential scaling of previous MAP-based approaches. We demonstrate the applicability of our method on radar tracking and computer vision problems.
The field of tracking is broad and possesses many applications, particularly in radar/sonar [1], robotics [14], and computer vision [3]. Consider the following problem: A radar is tracking a flying object, referred to as a target, using measurements of range, bearing, and elevation; it may also have Doppler measurements of radial velocity. We would like to construct a track which estimates the trajectory of the object over time. The Kalman filter [16], or a more general state space model, is used to filter out measurement errors. The key difference between tracking and filtering is the presence of clutter (noise measurements) and missed detections of true objects. We must determine which measurement to "plug in" to the filter before applying it; this is known as data association. Additionally complicating the situation is that we may be in a multi-target tracking scenario in which there are multiple objects to track and we do not know which measurement originated from which object.
There is a large body of work on tracking algorithms given its standing as a long-posed and important problem. Algorithms vary primarily on their approach to data association. The dominant approach uses a sliding window MAP estimate of the measurement-to-track assignment, in particular the multiple hypothesis tracker (MHT) [1]. In the standard MHT, at every frame the algorithm finds the most likely matching of measurements to tracks, in the form of an assignment matrix, under a one-to-one constraint (see Figure 1). One track can only result in one measurement, and vice versa, which we refer to as framing constraints. As is typical in MAP estimation, once an assignment is determined, the filters are updated and the tracker proceeds as if these assignments were known to be correct. The one-to-one constraint makes MAP estimation a bipartite matching task where algorithms exist to solve it exactly in polynomial time in the number of tracks N T [15]. However, the multi-frame MHT finds the joint MAP assignment over multiple frames, in which case the assignment problem is known to be NP-hard, although good approximate solvers exist [20].
The field of tracking is broad and possesses many applications, particularly in radar/sonar [1], robotics [14], and computer vision [3]. Consider the following problem: A radar is tracking a flying object, referred to as a target, using measurements of range, bearing, and elevation; it may also have Doppler measurements of radial velocity. We would like to construct a track which estimates the trajectory of the object over time. The Kalman filter [16], or a more general state space model, is used to filter out measurement errors. The key difference between tracking and filtering is the presence of clutter (noise measurements) and missed detections of true objects. We must determine which measurement to "plug in" to the filter before applying it; this is known as data association. Additionally complicating the situation is that we may be in a multi-target tracking scenario in which there are multiple objects to track and we do not know which measurement originated from which object.
There is a large body of work on tracking algorithms given its standing as a long-posed and important problem. Algorithms vary primarily on their approach to data association. The dominant approach uses a sliding window MAP estimate of the measurement-to-track assignment, in particular the multiple hypothesis tracker (MHT) [1]. In the standard MHT, at every frame the algorithm finds the most likely matching of measurements to tracks, in the form of an assignment matrix, under a one-to-one constraint (see Figure 1). One track can only result in one measurement, and vice versa, which we refer to as framing constraints. As is typical in MAP estimation, once an assignment is determined, the filters are updated and the tracker proceeds as if these assignments were known to be correct. The one-to-one constraint makes MAP estimation a bipartite matching task where algorithms exist to solve it exactly in polynomial time in the number of tracks N T [15]. However, the multi-frame MHT finds the joint MAP assignment over multiple frames, in which case the assignment problem is known to be NP-hard, although good approximate solvers exist [20]. Despite the complexity of the MHT, it only finds a sliding window MAP estimate of measurementto-track assignments. If a clutter measurement is by chance associated with a track for the duration of a window then the tracker will assume with certainty that the measurement originated from that track, and never reconsider despite all future evidence to the contrary. If multiple clutter (or otherwise incorrect) measurements are associated with a track, then it may veer "off into space" and result in spurious tracks. Likewise, an endemic problem in tracking is the issue of track swaps, where two trajectories can cross and get mixed up as shown in Figure 1. Alternatives to the MAP approach include the probabilistic MHT (PMHT) [9,Ch. 4] and probabilistic data association (PDA). However, the PMHT drops the one-to-one constraint in data association and the PDA only allows for a single target. This led to the development of the joint PDA (JPDA) algorithm for multiple targets, which utilizes heuristic calculations of the assignment weights and does not scale to multiple frame assignment. Particle filter implementations of the JPDA have tried to alleviate these issues, but they have not been adopted into real-time systems due to their inefficiency and lack of robustness. The probability hypothesis density (PHD) filter [19] addresses many of these issues, but only estimates the intensity of objects and does not model full trajectories; this is undesirable since the identity of an object is required for many applications including the examples in this paper.
Lázaro-Gredilla et al. [18] made the first attempt at a variational Bayes (VB) tracker. In their approach every trajectory follows a Gaussian process (GP); measurements are thus modeled by a mixture of GPs. We develop additional VB machinery to retain the framing constraints, which are dropped in Lázaro-Gredilla et al. [18] despite being viewed as important in many systems. Secondly, our algorithm utilizes a state space approach (e.g. Kalman filters) to model tracks, providing linear rather than cubic time complexity in track length. Hartikainen and Särkkä [11] showed by an equivalence that there is little loss of modeling flexibility by taking a state space approach over GPs.
Most novel tracking algorithms neglect the critical issue of track management. Many tracking algorithms unrealistically assume that the number of tracks N T is known a priori and fixed. Additional "wrapper logic" is placed around the trackers to initiate and destroy tracks. This logic involves many heuristics such as M -of-N logic [1,Ch. 3]. Our method replaces these heuristics in a model-based manner to make significant performance gains. We call our method a complete variational tracker as it simultaneously does inference for track management, data association, and state estimation.
The outline of the paper is as follows: We first describe the full joint probability distribution of the tracking problem in Section 1. This includes how to solve the track management problem by augmenting tracks with an active/dormant state to address the issue of an unknown number of tracks. By studying the full joint we develop a new conjugate prior on assignment matrices in Section 2. Using this new formulation we develop a variational algorithm for estimating the measurement-totrack assignments and track states in Section 3. To retain the framing constraints and efficiently scale in tracks and measurements, we modify the variational lower bound in Section 4 using a Bethe entropy approximation. This results in a loopy belief propagation (BP) algorithm being used as a subroutine in our method. In Sections 5-6 we show the improvements our method makes on a difficult radar tracking example and a real data computer vision problem in sports.
Our paper presents the following novel contributions: First, we develop the first efficient deterministic approximate inference algorithm for solving the full tracking problem, which includes the framing constraints and track management. The most important observation is that the VB assignment posterior has an induced factorization over time with regard to assignment matrices. Therefore, the computational cost of our variational approach is linear in window length as opposed to the exponential cost of the MAP approach. The most astounding aspect is that by introducing a weaker approximation (VB factorization vs MAP) we lower the computational cost from exponential to linear; this is a truly rare and noteworthy example. Second, in the process, we develop new approximate inference methods on assignment matrices and a new conjugate assignment prior (CAP). We believe these methods have much larger applicability beyond our current tracking algorithm. Third, we develop a process to handle the track management problem in a model-based way.

This paper describes a probabilistic model-based approach to tracking.  The probabilistic model includes important aspects of the tracking problem (namely track management) that are often left to external heuristics and thus not included in the joint probabilistic inference (hence the term "complete" in the paper's title).  In addition, by performing a richer kind of Bayesian inference (namely a mean field method with conjugate exponential family pairs as opposed to e.g. a MAP approximation) the proposed method offers a more complete representation of posterior uncertainty, improving performance and avoiding some (apparently) common tracker failure modes.The model's construction includes Markov structure (namely in the (conditionally) linear dynamical systems (X_t,Y_t) and the Markov structure on the meta-states (S_t), shown in Figure 1) that allows the corresponding mean field updates to be carried out efficiently, requiring time only linear in the sequence (or sliding window) length as opposed to the cubic complexity in a recent Gaussian Process-based method (though at the cost of a somewhat less rich class of dynamical models for the tracks themselves).  Finally, to make efficient another update required in the mean field algorithm, this paper introduces (or at least clarifies, c.f. line 117) a conjugate assignment prior (CAP) on the assignment matrices A and shows that, while some mean field computations with such variational factors are likely intractable to perform exactly (lines 193-201), a loopy belief propagation update (Section 4) can be used to estimate the expectation required to perform the mean field updates (and, possibly, to compute a lower bound on the intended mean field variational lower bound on the model evidence, c.f. 288-290 and 323).  The CAP's definition is naturally related to the prior used in corresponding MAP methods (Section 2) and the choice of method for approximating the expectation is naturally related to other variational approximations (namely the Bethe entropy approximation) and to the use of similar belief propagation algorithms for the matrix permanent problem.This is a high-quality paper. The writing is extremely clear and the explanations are very thorough.  The problem, model, and approximate inference algorithms are very well motivated, and the choices made all sound reasonable.I'm not familiar with the literature on trackers, and so my estimates of the originality and significance of this work may be both noisy and biased upwards (affecting my confidence score below).  That also means I'm not in a good position to be critical of the experiments, which certainly look reasonable from my naive perspective.  However, given the summary of prior work provided in Section 1, this work sounds like a clear step forward, with originality both in its more thorough Bayesian approach as well as its model and inference algorithm choices, which involve nice (if not "major") technical contributions.  It may not be a "major impact" paper (I wish the impact score weren't binary...) in that its direct impact may be limited to work on trackers.  But it may have a significant impact for those interested in Bayesian tracking methods (hard for me to judge) and, perhaps more importantly, I think many others in the broader NIPS community would enjoy reading this paper as much as I did.Here are some miscellaneous comments and questions:- I don't like using "vec" around line 175; instead, one can just define an inner product on matrices using trace.- As a personal pet peeve, "variatonal Bayes" is somewhat less descriptive than "a mean field variational approximation to the posterior". It might be nice to include the phrase "mean field" at least once in the paper.- Line 42, is "a complex discrete optimization task" just bipartite matching?  If so, it might be nice to name.- Around 210-212, you might consider saying "local and global" or "extensive and intensive" since those phrases are sometimes used in the literature.- Around 164-165, the phrase "we must to compute" probably wasn't meant to include the word "to".- On lines 96-99 and again on 426-428 you say it's surprising that by using mean field instead of MAP the (update) complexity is reduced from exponential to linear. However, since MAP can in many cases be written as mean field where some factors are constrained to be delta functions, it's not clear to me where the gain is coming from. On lines 44-45 you imply the exponential complexity comes from considering multiple frames; is the reduction in complexity really from the imposed Markov structure (namely on the s variables) which makes the assignment matrices conditionally iid? If so, it's not very surprising. If not, anything you can say to clarify the issue would be appreciated!


Near-optimal Reinforcement Learning in Factored MDPs
Any reinforcement learning algorithm that applies to all Markov decision processes (MDPs) will su er ( Ô SAT ) regret on some MDP, where T is the elapsed time and S and A are the cardinalities of the state and action spaces. This implies T = (SA) time to guarantee a near-optimal policy. In many settings of practical interest, due to the curse of dimensionality, S and A can be so enormous that this learning time is unacceptable. We establish that, if the system is known to be a factored MDP, it is possible to achieve regret that scales polynomially in the number of parameters encoding the factored MDP, which may be exponentially smaller than S or A. We provide two algorithms that satisfy near-optimal regret bounds in this context: posterior sampling reinforcement learning (PSRL) and an upper confidence bound algorithm (UCRL-Factored).

New regret bounds for the problem of online RL in factored MDPs have been presented. The bounds improve on the-state-of-the-art in generic  RL in terms of  dependency on the size of MDP assuming that there exists some known graphical structure for the problem.  For highly structured MDPs with local transitions this can lead to an exponential improvement in terms of dependency on the size of problem.Quality and clarityThe paper is clearly written. I also quickly checked the technical parts ( proof sketches) which seems  correct  to me. As for the clarity of results  I think the main trick to generalize the results of generic RL to the case of  factored MDP  have been clearly explained.   In regard to the quality of results,  I think  the idea of exploiting the graphical structure to improve the regret bounds of RL seems  a very promising idea to deal with large scale MDPs.   However some of the assumptions such as the one that requires the prior knowledge of  graphical  structure seems to be a bit restrictive, as it is recognized by the authors, and may limit the applicability of this result. The paper would benefit from a more in-depth discussion of this issue. In particular  a discussion on the possibility of combining the standard structure discovery algorithms with  UCRL or PSRL  would greatly add to the quality of the paper (also whether it is possible to prove similar regret bounds  in that case). Finally the paper claims that the result is "near-optimal" however no corresponding lower bound to support this claim is provided.Significance and noveltyThe problem of exploration-exploitation trade off in factored MDPs has been  extensively studied before in the RL literature . In that sense the idea of using the graphical structure to deal with large scale problems is not new. The main novelty  is that here  the bounds are expressed in terms of  accumulated regret, whereas the focus of those prior works are mostly on PAC exploration bounds. This seems a rather  incremental contribution since it is known that one can transform PAC complexity bounds to regret bounds and vice versa (See Jasch et al 10 and NIPS  11 workshop talk by Peter Auer). As for the novelty of the proof techniques,   the paper mainly relies on the  analysis of UCRL and PSRL and the main lemma to extend those results to the case of factored MDP is a standard algebraic trick.   Minor comments:line 140: The connection between X and S and A should be defined earlier ( before definition 3). line 298: where appropriate-> where it is appropriateline 308-309: M^* and M_k are equal in law : this is only true when we use the  true prior  (prior from which M^* is sampled) in Bayesian inference. If we use a wrong prior for M_k then they are not any more equal.


Concavity of reweighted Kikuchi approximation
We analyze a reweighted version of the Kikuchi approximation for estimating the log partition function of a product distribution defined over a region graph. We establish sufficient conditions for the concavity of our reweighted objective function in terms of weight assignments in the Kikuchi expansion, and show that a reweighted version of the sum product algorithm applied to the Kikuchi region graph will produce global optima of the Kikuchi approximation whenever the algorithm converges. When the region graph has two layers, corresponding to a Bethe approximation, we show that our sufficient conditions for concavity are also necessary. Finally, we provide an explicit characterization of the polytope of concavity in terms of the cycle structure of the region graph. We conclude with simulations that demonstrate the advantages of the reweighted Kikuchi approach.

This paper considers the so-called reweighted version of the Kikuchi variational approximation of the partition function, and provides sufficient conditions under which the variational objective is concave and hence can be optimized more effectively. The conditions are shown to be necessary under additional assumptions. Some experimental results on synthetic problems are shown to confirm the theoretical predictions.The paper is interesting and well written. It provides the necessary background and it does a good job explaining the results at an intuitive level and placing them into the context of what was previously known. As far as I know the results are novel, and they are a nice contribution to our understanding of variational approximations to the partition function. The results should be of interest for the sub-community of NIPS working on message passing and variational techniques. It's another step forward towards better understanding these techniques and their empirical success. One limitation is that it's a bit unclear if the work has practical implications. For example, it does not introduce any new algorithmic approach to the problem (in the experimental results section, they use the reweighted sum product algorithm of Wainwright et al). It's great that we get a better understanding and we can guarantee convexity and hence convergence, but does the theory provide any a-priori indication on how to set $rho$ for a practitioner? It seems that in practice rho_cycle and rho_tree are fairly close to each other and the corresponding approximations do not differ much.In the plot in figure 1(d), it seems that the error increases by going from rho_tree to rho_cycle, and it starts underestimating the value of the partition function. Am I reading it correctly? Is that a counterexample that the approximation is not always better and there is no hope it provides an upper bound up to rho_cycle? Minor:Line 126: I am curious why it "is not in general convex". Isn't a convex combination of two realizable marginals (points in Delta_R, say realized by tau_1 and tau_2) realizable by taking a convex combination of tau_1 and tau_2 with the same coefficients?


A framework for studying synaptic plasticity with neural spike train data
Learning and memory in the brain are implemented by complex, time-varying changes in neural circuitry. The computational rules according to which synaptic weights change over time are the subject of much research, and are not precisely understood. Until recently, limitations in experimental methods have made it challenging to test hypotheses about synaptic plasticity on a large scale. However, as such data become available and these barriers are lifted, it becomes necessary to develop analysis techniques to validate plasticity models. Here, we present a highly extensible framework for modeling arbitrary synaptic plasticity rules on spike train data in populations of interconnected neurons. We treat synaptic weights as a (potentially nonlinear) dynamical system embedded in a fullyBayesian generalized linear model (GLM). In addition, we provide an algorithm for inferring synaptic weight trajectories alongside the parameters of the GLM and of the learning rules. Using this method, we perform model comparison of two proposed variants of the well-known spike-timing-dependent plasticity (STDP) rule, where nonlinear effects play a substantial role. On synthetic data generated from the biophysical simulator NEURON, we show that we can recover the weight trajectories, the pattern of connectivity, and the underlying learning rules.

In this paper, the authors present a beefed up GLM model where the matrix which defines interactions between the neurons is time-varying based on some learning rule such as STDP. Also, sparsity is promoted with a spike-and-slab like factorization of the weight matrix. Inference in the model is performed using particle MCMC which is a powerful hybridization of SMC and MCMC (in this case Gibbs sampling).I think this paper presents an important variation in models of neural dynamics in that it attempts to model time-varying dynamics - a problem that computational neuroscientists and the NIPS community are likely interested in. While it doesn't assess its performance on real data, it does assess its performance on data generated from a biophysical model with unobserved neurons - the latter being an important component to consider. Though the results are on toy networks, the method's potential is convincing. Also, it is the first application of pMCMC to computational neuroscience that I'm aware of. That being said, I have one main area of concern in this draft.I think the relationship between this model and the current methods in experimental neuroscience is not always presented correctly. In line 41 the word "optogenetics" may be inappropriate. Optogenetics is not related to recording activity, only stimulating. Maybe use the word optical here instead of optogenetic. I'd also reconsider the usage of reference 2. That is a voltage imaging paper where the analysis requires trial averaging. Also, voltage imaging is not really ready to be used for population imaging. However, a more recent paper from Adam Cohen's group does have a demonstration of this (All-optical electrophysiology in mammalian neurons using engineered microbial rhodopsins). Generally people use Ca++ indicators for population imaging. For recent advances in Ca++ imaging, see "Ultrasensitive fluorescent proteins for imaging neuronal activity" from a collaboration of Janelia Farm labs. There are plenty of other Ca++ imaging papers that would be appropriate as well. The one I listed doesn't demonstrate poulation imaging, but the indicator from that paper is regularly used to image populations of neurons in vitro and in vivo.The authors often cite optical methods as motivation for their model. The idea is that optical methods allow the recording of cells that are near each other and presumably some of them are synaptically connected. There is a major issue that goes unmentioned in this paper: extracting spikes from imaging data is a non-trivial problem (see papers by Josh Vogelstein and Liam Paninski for commonly used algorithms), and the temporal resolution of imaging does not often permit the precise measurements needed to identify STDP type rules. The papers referenced which do infer synaptic connections [5,6] use electrophysiological data which does provide highly accurate and precise spike times. That being said, fast and reliable optical indicators of action potentials are likely on the horizon, and the model is still interesting even if it does not admit a causal, synaptic interpretation. Besides that, there are a few smaller specific points that should be clarified. Because the GLM model does not necessarily admit a true PSP or PSC interpretation, using units of nA here maybe disingenuous. Furthermore, for PSC amplitudes, even 1 nA would be very big. One might expect a max current in the range of 5-100 of pA. In addition, you reference PSPs earlier (post-synaptic potentials) in the paper. These would be measured in millivolts not pico- or nanoamps. A PSC (post-synaptic current) would be measured in amperes. Because the membrane resistance changes over time, the two are not interchangeable.Given that the probability of spiking is voltage-dependent, not current-dependent, you could stick with a PSP interpretation and change the units to millivolts.  That being said, many STDP models vary the max current parameter - they don't explicitly model some max potential change. Perhaps it's worth considering dropping the units here and saying that the unitless GLM weights are modulated in a way that is based on STDP. I think this is reasonable and avoids the slippery-slope of claiming your model captures electrophysiological properties.It's not clear how the noise variable is truncated to ensure that the bounds are maintained. Is it a function of W(t) and dW(t) such that you update the weights without noise and then truncate the Gaussian at (0+W(t)+dW(t),W_max-W(t)+dW(t))? Or is it dependent on A_+ and A_-? In any case, it's not clear how the model results in noise from a truncated Gaussian random variable which will not violate the bounds.The way you've written it, it seems like you are claiming the model in Song et al. is infeasible because the weights are unbounded. But they do in fact bound synaptic weights from below at 0 and from above by some $g_{max}$ (in their notation).It's not clear to me that Eldawlatly et al. model time-varying synaptic parameters - as you claim - as much as causal influences that act at multiple time lags in a discrete-time model. The interpretation - depending on the bin size and length of the delay - could be either indirect (e.g. polysynaptic effects) or direct (e.g. delay due to axon length), but none of the model parameters are time-varying.Finally, I'm concerned about the tractability of the inference algorithm. It is not uncommon for optical recordings to contain 100s of neurons. How long would a network of that size take to fit? How much data would be needed to fit the model?


Top Rank Optimization in Linear Time
Bipartite ranking aims to learn a real-valued ranking function that orders positive instances before negative instances. Recent efforts of bipartite ranking are focused on optimizing ranking accuracy at the top of the ranked list. Most existing approaches are either to optimize task specific metrics or to extend the rank loss by emphasizing more on the error associated with the top ranked instances, leading to a high computational cost that is super-linear in the number of training instances. We propose a highly efficient approach, titled TopPush, for optimizing accuracy at the top that has computational complexity linear in the number of training instances. We present a novel analysis that bounds the generalization error for the top ranked instances for the proposed approach. Empirical study shows that the proposed approach is highly competitive to the state-of-the-art approaches and is 10-100 times faster.

The paper addresses the computational issue of bipartite ranking. The authors propose a new algorithm whose computational complexity is linear in number of training instances, and provide theoretical analysis of generalization error. The paper is rounded of with extensive experiments.Strong points- The paper is clearly written and might be of some value in case of bipartite ranking with large datasets. The generalization bound is novel and experiments section is detailed.However, i have multiple questions about this paper:1. Though bipartite ranking is well studied, it is restricted in scope, in the sense that it is in the domain of ranking but cannot handle queries. Considering there are already well established algorithms for bipartite ranking which have been well studied theoretically and tested empirically, is the study really very valuable? For eg., this will only be useful when m,n are really large. Is that very practical in domain of bipartite ranking? Admittedly, this is just my thought and i would like to hear authors' view on this (citing example or something).2. The main reason the paper gets an O(m+n/\sqrt(e)) error bound is because of the new defined target loss (2) and using a smooth convex surrogate which allows standard primal-dual trick to get a smooth dual, thereby allowing standard accelerated gradient descent optimization. According to target loss (2), a negative instance followed by all positive instances has greater loss than a positive instance followed by all negative instances. Can this be considered promoting good ranking at top, especially since there is no position based discount factor? Moreover, in the supplement it is stated that maximizing positives at top cannot be achieved by AUC optimization, but target loss (2) is an upper bound on rank loss. So why should someone try to optimize an upper bound on rank loss, if AUC optimization itself is not suitable for the purpose of pushing positives at top?  3. Once the convex surrogate is taken to be smooth, conversion to dual and applying Nestrov technique is neat but i do not think it is extremely novel.4. Looking at empirical section, i am confused as to what TopPush is gaining on LR (logistic regression). Computational power of the new algorithm is the USP of the paper; it does not seem to be doing any better than LR; infact LR takes less time than TopPush in  4/7 experiments. Nor is it gaining anything significant in AP and NDCG metrics, effectively metrics which are popular; it gains a little in position at top but loses in AUC.  So why should we consider TopPush over LR? Or am i reading the experimental results wrong?A side point: On comparing the computational complexity with SVM (Rank,MAP); it can be seen that all of them scale linearly with training data size. The gain in computation time in TopPush is because SVM consider hinge loss while TopPush considers a smooth surrogate. So computational complexity linear in number of training instances is not unique to TopPush.5. Theory- In the generalization bound, shouldn't the focus be on the cases when there are a large number of negative instances and few positive instances? The other way round is less practical and even an average ranking function would put a few positive instances on the top. However, if we focus on the negative instances, i am not sure what the bound is relaying. With growing n, the empirical loss is much more likely to keep increasing, since the normalizing factor is only 1/m (no dependence on n). Since \delta will become small, the L.H.S probability is likely to grow but the R.H.S is also likely to grow. Maybe i am not being able to understand the significance of the bound, from a more useful n >> m point of view.   My ratings later on are provisional. I would like the authors to address the questions i have raised. Specifically i would like the authors to shed more light on the comparison between TopPush and LR ( 4, question on empirical section). In my opinion, the accept/reject hinges on clarifying how TopPush gains on LR. I will be happy to review my decision after author feedback.Update after Author Feedback- 1. "Example of m,n large"- I am not an expert in bipartite ranking, so i will take the authors' words for it. However, from my knowledge of online advertisement, is it not the case that ranking online advertisements is in the learning to rank framework? (i.e query dependent?). I completely agree with the first reviewer that showing experiments for large datasets (possibly real datasets used in bi-partite ranking) will be very useful.2. "AUC optimization"- The authors dont really answer the question. They talk about advantage of the new loss, in terms of optimization and gen. bounds. However, it has nothing to do with AUC. In fact, independent of how the new loss compares with AUC, the advantages will hold. From the point of view that the new loss is an upper bound on AUC and AUC is not useful for the objective of "pushing positives at top", why should someone optimize the new loss?3."comparison with LR"- This is critical. I agree that TopPush is doing better than LR in Pos@Top metric. I think in the revised version, the authors should modify the introduction slightly. The USP of the paper is the computational advantage of TopPush over other algorithms. This is overselling the paper a little bit. TopPush has no (visible) computational advantage over LR. It can be seen as an alternate, with advantage when it comes to performance on a specific metric (and disadvantage on some others). 4."SVM"- I believe the advantage over SVM based methods is the quadratic convergence rate (O(1/T^2) as opposed to O(1/T)), not linear in "m+n"?  Both SVM based methods and top push are linear in "m+n", as the authors have clearly shown in Table 1.5."Gen bound"- Please include the discussion in the revised draft. This is critical. Overall, i like this paper. With revision, it will certainly be a very good paper. I have updated my decision to an accept.


Causal Inference through a Witness Protection Program
One of the most fundamental problems in causal inference is the estimation of a causal effect when variables are confounded. This is difficult in an observational study because one has no direct evidence that all confounders have been adjusted for. We introduce a novel approach for estimating causal effects that exploits observational conditional independencies to suggest "weak" paths in a unknown causal graph. The widely used faithfulness condition of Spirtes et al. is relaxed to allow for varying degrees of "path cancellations" that will imply conditional independencies but do not rule out the existence of confounding causal paths. The outcome is a posterior distribution over bounds on the average causal effect via a linear programming approach and Bayesian inference. We claim this approach should be used in regular practice to complement other default tools in observational studies.

The paper describes the so-called Witness Protection Program (WPP) algorithm to compute bounds on an inferred average causal effect (ACE) that takes possible violations of the standard faithfulness assumption in causal inference into account. It does so by introducing a set of relaxation parameters to constrain unfaithful interactions, and then uses a linear programming framework to solve for the resulting upper and lower bounds on the ACE for a given link by adjusting for an appropriate set of covariates, subject to these constraints.This problem currently receives a fair amount of attention, so the paper is both timely and interesting. I like the approach and ideas introduced in the paper: it is a principled attempt to explicitly quantify the impact of (weak) unfaithful interaction patterns, in combination with well-known mathematical tools to solve the corresponding maximization/minimization problems.Whether or not they found the right method to capture this impact remains to be seen. The resulting output is hard to interpret from a conceptual point of view: a distribution over bounds which are a function of 'unfaithful relaxations' relative to a given model ... but how likely these unfaithful instances are, and with it how relevant/realistic these bounds themselves are remains unclear. Many relaxations lead to wide bounds to the point where they can become less than useful. The method does not commit to any preferred point estimate within the interval ... even though people are likely to interpret it that way. Experimental evaluation is poor: the 'bias' evaluation metric is meaningless in comparing the WPP method with its competitors as they literally measure two completely different things. As a result it is unclear if the WPP output is a valuable addition to practitioners or merely a complicated distraction. Also the practical performance on the influenza data set is worryingly weak: ACE in (-0.23,0.64) implies next to no possible relevant conclusion ... although that could be the fairest assessment of them all.QualityThe approach introduced in the paper is technically sound, with many details explicated in the supplementary file (I did not check any details in there). Ideas are well introduced, although the overall context is lost on occasion. Important aspects like choosing the relaxation parameters are relegated to a future article, which makes it difficult to get a feel for potential issues with this part of the method. Experimental evaluation is not balanced or robust enough. ClarityDecently written (with exception of conclusion, see comments below). The algorithmic steps and subsequent modifications in terms of many subtly different parameters was quite hard to follow, in part due to lack of explanation of certain details of the method. (I think I understand what happens, but I am not sure I could fully implement it from just this description; supplement may just be sufficient).OriginalityAs far as I can tell the method is a novel combination of familiar techniques. It is closely related to, but sufficiently different from recent work on the same problem. As such it may stimulate a fruitful cross-fertilization of several approaches on the subject.SignificanceThe significance of the paper mainly lies in the ideas and direction it brings to the problem, showing how to incorporate other techniques in solving this highly challenging problem. In its current implementation it seems unlikely to represent the finished article for practitioners, although some of the ideas employed in the WPP algorithm could well end up playing an important role. Summary:Relevant and highly topical problem. Interesting method that will undoubtedly find an audience. Current application limited, but likely to extend to other, more general cases. Reasonably well written, though could be improved. Technically interesting solutions. Experimental evaluation is poor. Relevance and proper interpretation of output distribution over bounds remains unclear.----------------Other comments:p1/5: 'Witness Protection Program' - > great name for an algorithm p1/47: 'unstable' is not the right word (they are stable, but may not hold true) p2/74: 'cartoon representation'?p2/93-94: technically this also assumes causal sufficiency or knowledge W - > Xp3/127: explain that witness W is itself not needed in the computation of the ACE, only as a means to identify set Zp3/143: typo 'an useful' p3/144: 'gives greater control over violations of faithfulness' : too vague as I do not see the control aspect p4/164: might - > can;     idem: '(conditional)' - > remove or change to '(unconditional)' , as always W dep. U | X p4/172-180: very sudden transition in level of abstraction; hard to interpret for readers less familiar with the subject - > explain / give some intuition on what these parameters represent,p4/183: link to original introduction in (1),p4/185-189: again interpret the constraint parameters; (easily misread as weak interactions)p5/221: briefly describe the relation between the search part and the bounds computation,idem: explain how the identification of admissible sets Z for W depends on the relaxation parameters thetap5/239: 'Background ... used here.' - > sentence/purpose unclear: used how? important for the paper?p5/259: to be clear: you mean max. 10 free parameters to describe the entire set Z, not a set of 10 variables, right?p6/304: 'We know of no previous analytical bounds' - > This seems closely related to recent papers such as 'Estimating Causal Eff ects by Bounding Confounding' (Geiger, Janzing,  Sch olkopf, UAI2014)p6/305: as a last-minute reviewer I have not checked the supplementary material in detailp6/319-332; I like the method but I doubt the practical efficacy of the back-substitution process in refining bounds;  seems rare to be able to exploit such constraint violations, (can you give some idea of how effective this step is in section 5?)p7/350: 'more than one bound' - > this is due to different possible admissible sets for different witnesses right?p7/357: 'comparison is not straightforward' - > no: the subsequent trivial optimal example shows that comparison based on the bias metric is meaningless.p7/360: 'only about 30% of the interval (-1,1)' - > that still covers (0,0.6) which implies 'anywhere between a strong causal link and no causal link' ...p7/369: '5000 points' - > typical (medical) trials often have a lot less data available: worried about the impact on the size of the bounds for say 500 records,p8/384: it is completely impossible to gauge meaningful information from Table 1 on the benefit of using WPP over the standard point estimate using faithfulness.p8/391: 'WPP is quite stable' - > seems like a nice way of saying that the bounds derived by WPP are so loose that they don't even depend on whether the problem is solvable or not p8/406-8: 'This does not mean ... within the bounds.' - > this is a crucial statement for interpretation: many casual readers will read the bounds as a confidence interval around the best guess value. It also highlights the difficulty in using the WPP results in practice: what do the bounds we derive actually mean? p8/421+: 'risky Bayesian approaches', 'WPP bounds keep inference more honest' - > emotive language that suggests a biased evaluation and/or erroneous interpretation of resultsidem: 'providing a compromise' - > WPP is not a compromise but focusses on an additional aspect of the overall inference problem,idem: 'and purely theory-driven analyses that refuse to look at competing models' - > I have no idea what you refer to in this statement, but it definitely sounds unduly dismissivep9: typo ref 9: inuenza


Biclustering Using Message Passing
Biclustering is the analog of clustering on a bipartite graph. Existent methods infer biclusters through local search strategies that find one cluster at a time; a common technique is to update the row memberships based on the current column memberships, and vice versa. We propose a biclustering algorithm that maximizes a global objective function using message passing. Our objective function closely approximates a general likelihood function, separating a cluster size penalty term into row-and column-count penalties. Because we use a global optimization framework, our approach excels at resolving the overlaps between biclusters, which are important features of biclusters in practice. Moreover, Expectation-Maximization can be used to learn the model parameters if they are unknown. In simulations, we find that our method outperforms two of the best existing biclustering algorithms, ISA and LAS, when the planted clusters overlap. Applied to three gene expression datasets, our method finds coregulated gene clusters that have high quality in terms of cluster size and density.

This paper proposes a biclustering algorithm that has some advantages over related works. It is able to deal with overlapping clusters and the clustering problem is addressed by the maximization of a well-defined global objective function. I find the idea interesting and the paper technically sound and well written. Despite the description of related works is kind of short, it is enough for positioning the paper in the right context and to understand the novelty of the proposals. While the theoretical formulation presented in Section 2 seems to be correct, it would be very beneficial to have at the end of this section the pseudo-code of the proposed BCMP method. This would help to wrap up the results presented in this section and to make clearer the steps of the proposed algorithm.In the experimental section, the proposed method is compared with to related methods on simulated data with different degrees of noise and overlapping between clusters. Despite these results illustrate the suitability of the proposed algorithm; it would be nice to see its performance on real datasets. In the introduction of the paper, authors claimed that this kind of algorithms can be used in different kind of applications like clustering of microarray data, tumor prediction and document classification. However, no real data is used to test the performance (and compare with existing methods) of the proposed clustering algorithm. A final minor comment is regarding the abstract. In the current version is too long, at some point it looks like an introductory section. I think it can be reduced to half of the size.


Multiscale Fields of Patterns
We describe a framework for defining high-order image models that can be used in a variety of applications. The approach involves modeling local patterns in a multiscale representation of an image. Local properties of a coarsened image reflect non-local properties of the original image. In the case of binary images local properties are defined by the binary patterns observed over small neighborhoods around each pixel. With the multiscale representation we capture the frequency of patterns observed at different scales of resolution. This framework leads to expressive priors that depend on a relatively small number of parameters. For inference and learning we use an MCMC method for block sampling with very large blocks. We evaluate the approach with two example applications. One involves contour detection. The other involves binary segmentation.

The authors propose a way to define distributions over high-order image features, using a multiscale representation of binary image features, termed multiscale field of patterns (FoP). The paper discusses two example applications, contour integration and segmentation.- Originality and SignificanceThe idea that multiscale representations can be useful to capture global image properties is at least as old as computer vision. Nonetheless, the paper presents a thorough and original derivation of a prior over higher-order binary features. In particular, if I understand correctly, the nonlinearity (logical OR) used to build the binary image pyramid, and the use of nonlinear filters (binary pattern detectors), allows the model to capture higher-order image properties. For instance, with the right choice of cost (or energy) functions for the binary patterns at different scales, the authors demonstrate that samples from the model trained on contour images can capture long and curved contours. This is an interesting result, and so are the results on contour detection in noisy images and figure/ground segmentation. On the other hand, though, there are some important limitations. First, the authors only consider binary images and white pixel noise, therefore dealing with extremely simplified statistics, compared to natural images. For instance, given that the contour images are taken from the BSD (where both human-labeled contours and the original natural images are provided), it would be interesting to see if the model successfully detects contours also on those images (as opposed to just the binary contour image with white noise added on top). Neither do the authors experiment with white noise at different scales, which would be a slightly stronger test than adding white noise only at the finest level (since the noise averages out at coarser levels). Also, the authors compare the multiscale FoP model to a single scale FoP and to “baseline” models from the literature; but how well does the multiscale FoP compare to state of the art in the applications described?Another major limitation, at present, seems the computational complexity of the algorithm (Line 300 and following: 2 days of training over a cluster with 20 cores for a 200 images training set?), which makes the approach rather impractical. - QualityThe paper is technically sound and of good quality. It presents a simple and clear idea, develops the necessary math in great detail, and shows good results on interesting computer vision problems such as contour detection and figure/ground segmentation.Minor: Line 418: “picked the pairwise discontinuity cost minimizing the per-pixel error rate in the test set”. Doesn’t this produce overfitting?- ClarityThe paper is very well written and provides good coverage of relevant literature. I would suggest to consider also the work of Theis, L., Hosseini, R., & Bethge, M. (2012). Mixtures of conditional Gaussian scale mixtures applied to multiscale image representations. PLoS One, 7(7), e39857, which seems particularly relevant to this paper.Typos: Line 79: “fixed” is repeated twice.Line 264: y_i should be y_1


Clustered factor analysis of multineuronal spike data
High-dimensional, simultaneous recordings of neural spiking activity are often explored, analyzed and visualized with the help of latent variable or factor models. Such models are however ill-equipped to extract structure beyond shared, distributed aspects of firing activity across multiple cells. Here, we extend unstructured factor models by proposing a model that discovers subpopulations or groups of cells from the pool of recorded neurons. The model combines aspects of mixture of factor analyzer models for capturing clustering structure, and aspects of latent dynamical system models for capturing temporal dependencies. In the resulting model, we infer the subpopulations and the latent factors from data using variational inference and model parameters are estimated by Expectation Maximization (EM). We also address the crucial problem of initializing parameters for EM by extending a sparse subspace clustering algorithm to integer-valued spike count observations. We illustrate the merits of the proposed model by applying it to calcium-imaging data from spinal cord neurons, and we show that it uncovers meaningful clustering structure in the data.

The paper proposes a method for clustering multi-neuronal data based on a parametric latent variable model. The main contribution of the approach presented is the extension of previous mixture of factor analyzers models to a dynamic setup where a hidden latent model captures the temporal structure of the model. Latent variable models have proven very successful in recent years, and offer great flexibility and lead to efficient (usually approximate) estimation algorithms.  The basic building block of the model is a multinomial population model allowing each neuron to belong to one of M classes, where each latent variable is modeled using a linear dynamical system, and the observation model is based on a Poisson firing model. The main distinction compared to previous work is the incorporation of temporal dependencies in the latent variables. The general inference and parameter estimation is intractable within the present setting, leading to the introduction of a factorization assumption for the posterior distribution, and to a variational lower bound. This lower bound is optimized based on coordinate ascent, which is facilitated by using the dual formulation. The parameter update is interleaved with the update of the variational bound variables, leading to a variational EM type algorithm. In order to improve performance, an initialization scheme is proposed based on Poisson subspace sampling.  Additionally, in order to prevent certain undesirable properties of the solution (like assigning all neurons to a single cluster), a non-negativity constraint is introduced into the neuronal activity model. The authors conclude their paper by presenting experiments with both artificial and natural data, taken from calcium imaging of motor neurons activities. The analysis of the empirical data led to some interesting conclusions about the firing phase of neurons. Interestingly, validation using electro-physiological measurements was suggested, opening an interesting route for model validation. Overall this is a solid paper, extending earlier work in several promising directions and demonstrating good empirical results. However, as far as NIPS paper go, the contribution beyond previous work seems rather incremental.A question that should be addressed:The model described in eqs. (3-5) is linear and assumed a specific noise model. What are the implications of these assumptions? How robust is the model to misspecification? Following the authors' response I have re-evaluated the contribution of the paper and raised my original score.


Deep Convolutional Neural Network for Image Deconvolution
Many fundamental image-related problems involve deconvolution operators. Real blur degradation seldom complies with an ideal linear convolution model due to camera noise, saturation, image compression, to name a few. Instead of perfectly modeling outliers, which is rather challenging from a generative model perspective, we develop a deep convolutional neural network to capture the characteristics of degradation. We note directly applying existing deep neural networks does not produce reasonable results. Our solution is to establish the connection between traditional optimization-based schemes and a neural network architecture where a novel, separable structure is introduced as a reliable support for robust deconvolution against artifacts. Our network contains two submodules, both trained in a supervised manner with proper initialization. They yield decent performance on non-blind image deconvolution compared to previous generative-model based methods.

This paper presents a method for nonblind deconvolution of blurry images, that also can also fix artifacts (e.g. compression, clipping) in the input, and is robust to deviations from the input generation model.  A convolutional network is used both to deblur and fix artifacts; deblurring is performed using a sequence of horizontal and vertical conv kernels, taking advantage of a high degree of separability in the pseudoinverse blur  kernel, and are initialized with a decomposition of the pseudoinverse.  A standard compact-kernel convnet is stacked on top, allowing further fixing of artifacts and noise, and traned end-to-end with pairs of blurry and ground truth images.This is a nice approach that appears to obtain good results on the examples presented.  However, I feel the data could be better explained, and perhaps the test set expanded in size.  I didn't see any description of the sources for the training or testing data -- where did these images come from?  How were the blurry versions generated (in both train or test), and what kernels, noise, etc. were used exactly?  I feel this is a significant gap, and hope it might be addressed in the authors' rebuttal.Questions/comments:* l.150 "by taking a great number of pictures"  -- I'm not sure I understand this, could you explain a bit more?* l.166 "two million sharp patches"  -- from where were these drawn?* l.198: description of figure does not match the figure itself (there is just one example instead of two and the letters do not match)* l.214: extending with large conv kernels:  An alternative might be an FFT conv -- maybe this could be touched on in the text a bit?* l.262:  Is the nonlinearity sigma applied between horizontal and vertical conv layers?  It seems that it is.  If so, the operations are starting to differ from a linear decomposition of the original pseudoinverse convolution.  Is this good or bad in this case?  What if this nonlinearity is removed?* If there is a nonlinearity between horizontal and vertical, is there any benefit to stacking another layer of horizontal and vertical convs (with nonlin), and tweak the initialization either to stretch the pseudoinverse over all these applications or make the second set of kernels start from random?* l.268 "high-dimensional structure make the network more expressive than traditional pseudo-inverse and reasonably robust to outliers":  Which high dimensions are you referring to here?  Is it the 36 maps from the head of the SVD?  If so, would response of the largest eigenvectors potentially be truncated by the sigmoid?  Or does the network learn to work around this by moving away from the decomposition, and this is what makes it robust?* Likewise, I'm curious what might happen by initializing by duplicating the largest eigenvectors multiple times and adding a bit of gaussian noise to the weights?* l.306 "16 images"  -- this seems a bit small for a test set, though I'm not sure how large the images are; perhaps this could be augmented and/or explained a bit more?* l.342 "done by merging" kernels:  The merged kernel is not quite the same network as performing each in sequence:  there are more params.  But this probably is a benefit, allowing the network to perform even better, I suspect.* l.364 "This is the first attempt to make use of generative solvers to guide neural network learning" -- I don't think this is true, and think this sentence probably should be removed.  E.g. naive bayes corresponds to logistic regression; sparse coding informed the design of learned ISTA "Learning Fast Approximations of Sparse Coding" Gregor 2010; unrolled mean-field inference of RBMs in "Multi-prediction deep Boltzmann machines" Godfellow 2013.* sec 7:  Was the network retrained for each blur kernel in the experiments?


Kernel Mean Estimation via Spectral Filtering
The problem of estimating the kernel mean in a reproducing kernel Hilbert space (RKHS) is central to kernel methods in that it is used by classical approaches (e.g., when centering a kernel PCA matrix), and it also forms the core inference step of modern kernel methods (e.g., kernel-based non-parametric tests) that rely on embedding probability distributions in RKHSs. Previous work [1] has shown that shrinkage can help in constructing "better" estimators of the kernel mean than the empirical estimator. The present paper studies the consistency and admissibility of the estimators in [1], and proposes a wider class of shrinkage estimators that improve upon the empirical estimator by considering appropriate basis functions. Using the kernel PCA basis, we show that some of these estimators can be constructed using spectral filtering algorithms which are shown to be consistent under some technical assumptions. Our theoretical analysis also reveals a fundamental connection to the kernel-based supervised learning framework. The proposed estimators are simple to implement and perform well in practice.

This paper deals with the estimation of the kernel mean, an effective tool to design kernel-based learning algorithms for probability measures. Specifically, the authors propose the use of spectral filtering to obtain estimators of the kernel mean better than the standard empirical estimator. Their estimator builds upon recent work [13] which demonstrates  that the commonly used kernel mean estimator can be improved by shrinkage estimators. The authors provide a theoretical analysis of their spectral kernel mean estimator from an operator learning point of view and conducted artificial and real data experiments to test their spectral-based mean estimation under various settings and compare it with the standard empirical estimator and other shrinkage estimators.The paper is well written and the subject is well motivated in the introduction. The paper clearly explains a new mechanism that should have margin to improve kernel mean estimation. The underlying idea is quite simple, but it is proved through a rigorous analysis and an empirical evaluation.In the experimental section, it is true that Figure 2 shows that  the spectral estimator is less expensive than other shrinkage estimator in terms of computational cost; however, it is much more expensive than the standard estimator. The data used in the experiments seem to be relatively small. The computational efficiency of the proposed estimation for large data is not clear.Minor comment: - line 87: better than \mu —> better than \hat{\mu}- line 142: There is not (iii) in theorem 1Pros: 	- new kernel mean estimator         - well written paper         - the proposed method is justified by a theoretical analysis and experimental resultsCons: 	- The data used in the experiments seem to be relatively small


Diverse Sequential Subset Selection for Supervised Video Summarization
Video summarization is a challenging problem with great application potential. Whereas prior approaches, largely unsupervised in nature, focus on sampling useful frames and assembling them as summaries, we consider video summarization as a supervised subset selection problem. Our idea is to teach the system to learn from human-created summaries how to select informative and diverse subsets, so as to best meet evaluation metrics derived from human-perceived quality. To this end, we propose the sequential determinantal point process (seqDPP), a probabilistic model for diverse sequential subset selection. Our novel seqDPP heeds the inherent sequential structures in video data, thus overcoming the deficiency of the standard DPP, which treats video frames as randomly permutable items. Meanwhile, seqDPP retains the power of modeling diverse subsets, essential for summarization. Our extensive results of summarizing videos from 3 datasets demonstrate the superior performance of our method, compared to not only existing unsupervised methods but also naive applications of the standard DPP model.

This paper presents a framework for supervised video summarization. The proposed method includes a new Sequential Determinantal Point Process (seqDPP) model, which learns to select the best subsets of the raw input data as components of the output summarization. Unlike the most common approaches, that focus on unsupervised summarization by selecting most important/representative frames and features, this paper attempts to directly learn from human-created summaries.The technical formulation of the method seems sound and correct. The new model seqDPP extends DPP by incorporating sequential dependencies. The model, learning and inference formulations are updated to account for the extension.The performance evaluation is adequate, including quantitative results (Precision, Recall and F-Score) from 3 datasets (youtube, OVP, kodak consumer). The results are good and mostly outperform the related work.The readability of the document is good. However the order of the sections could be improved. For instance, the Related Work could be presented earlier to help understand the context of this paper.In terms of originality, the seqDPP model is an extension of DPP which is used to summarize text data. The extension is sound to tackle the problem of video summarization. Also, the authors use a novel way to derive ground truth, by combining multiple human-created summaries into a unique ‘oracle’ summary, which is used for learning and evaluation.


Convolutional Kernel Networks
An important goal in visual recognition is to devise image representations that are invariant to particular transformations. In this paper, we address this goal with a new type of convolutional neural network (CNN) whose invariance is encoded by a reproducing kernel. Unlike traditional approaches where neural networks are learned either to represent data or for solving a classification task, our network learns to approximate the kernel feature map on training data. Such an approach enjoys several benefits over classical ones. First, by teaching CNNs to be invariant, we obtain simple network architectures that achieve a similar accuracy to more complex ones, while being easy to train and robust to overfitting. Second, we bridge a gap between the neural network literature and kernels, which are natural tools to model invariance. We evaluate our methodology on visual recognition tasks where CNNs have proven to perform well, e.g., digit recognition with the MNIST dataset, and the more challenging CIFAR-10 and STL-10 datasets, where our accuracy is competitive with the state of the art.

This paper presents a kernel based approach to learning a model that is heavily based on convolutional networks. The standard types of layers of a convolutional network is replicated by a kernel that produces an equivalent or similar response to that of original. I think the main aspect of the paper is the approximation of the gaussian kernel by a learned formulation given in equation 4, however the exact procedure for training is not clear from the paper. The results presented on widely used datasets suggest that despite unsupervised training of the hierarchy of feature extractor, the system achieves competitive results.The method presented in the paper is algorithmically quite heavy compared to the simplicity of convolutional networks. On the other hand, it might provide useful insights into new models or better understanding of current models. I think the main drawback of this paper is that, it does not contain a clean explanation of an end-to-end process of training the model. Convolutional nets are popular since they achieve state of the art results on challenging datasets, but at the same time, the training procedure is very simple and scales to large datasets. I think it would be very helpful if the paper included a more clean explanation of the overall algorithm and computational requirements of training a CKN.


Sparse Multi-Task Reinforcement Learning
In multi-task reinforcement learning (MTRL), the objective is to simultaneously learn multiple tasks and exploit their similarity to improve the performance w.r.t. single-task learning. In this paper we investigate the case when all the tasks can be accurately represented in a linear approximation space using the same small subset of the original (large) set of features. This is equivalent to assuming that the weight vectors of the task value functions are jointly sparse, i.e., the set of their non-zero components is small and it is shared across tasks. Building on existing results in multi-task regression, we develop two multi-task extensions of the fitted Q-iteration algorithm. While the first algorithm assumes that the tasks are jointly sparse in the given representation, the second one learns a transformation of the features in the attempt of finding a more sparse representation. For both algorithms we provide a sample complexity analysis and numerical simulations.

Summary: This paper addresses the problem of multi-task reinforcement learning. The underlying intuition is that the value functions of all tasks can jointly be represented as a sparse combination of given basis functions. The paper introduces two algorithms, which are both variants of the Fitted Q-Iteration algorithm. The difference between these two algorithms is how they solve the regression problem at each iteration of FQI.To exploit the joint sparsity, Group LASSO Fitted Q-Iteration (GL-FQI) solves a group Lasso regression problem at each iteration. The regularizer in the Group Lasso encourages finding a sparse set of weights that are jointly used in all tasks.The other algorithm is Feature Learning FQI (FL-FQI). The underlying assumption of FL-FQI is that there exists a linear transformation of the features that make the tasks jointly sparse in the transformed space.With certain set of assumptions, the paper provides error upper bound for both algorithms.One may summarize the main finding of the theoretical guarantee for GL-FQI (FL-FQI) as that the joint sparsity level (the rank of weight matrix) appears in the upper bound. Moreover, if the number of tasks (T) is large, the effect of the number of features (d) becomes less important (the behavior would be log(d) / T for GL-FQI and d / T for FL-FQI). This latter point makes these methods different from a single-task Lasso FQI, which has log(d) behavior (no division by T).The paper also has experimental results comparing LASSO FQI, GL-FQI, and FL-FQI.This is an interesting paper. It is written clearly and has both theory and experiments (though please refer to my comments below regarding the theory, as some aspects of it are not clear). Benefitting from sparsity is not new in RL (both in the API and AVI contexts), but as far as I know, it is new in the multi-task RL setup. Even though GP-FQI might not seem to be a surprisingly novel algorithm, I found FL-FQI quite compelling as it tries to find a transformation that makes the problem sparse.Quality: The paper has both theory and experiments. The theory has some simplifying assumptions (especially Assumptions 1 and 2), but the message is clear: for some problems we might benefit from the joint sparsity of value functions. But there are some concerns about the theory, which I describe below.Clarity: The paper is well-written and well-organized. Some parts of it can be improved though.Originality: The general structure of the algorithms is not surprisingly novel (in particular, GL-FQI), but as far as I know, they have not been suggested before.Significance: The algorithms developed in this paper are good additions to the toolbox of RL practitioners.Detailed Comments (roughly ordered according to their importance):[Major] Please address these comments.* In Appendix C, the authors used a result from Lounici et al. 2011. Is it possible to be a bit more specific about the referred result?If it is Corollary 4.1 in that paper, then their noise model is for Gaussian noise. This is not the case here. Also the constants should be 128/k^2 instead of 1280/k^4.If it is Theorem 8.1, which is for non-Gaussian case, then the bound would be different (we have (log d)^{3/2 + delta} / sqrt{T} ) behavior instead of log d/T behavior).* If I understand Assumption 2 correctly, it indicates that the the i-th basis function phi_i is uncorrelated to (P phi_j). Later Lemma 7 shows that this along Assumption 1 imply that the application of the Bellman operator on a function f does not change its sparsity property. I have two main concerns here:1) Why the probability distribution for this correlation is chosen to be the Lebesgue measure? Why not mu or rho?2) In the proof of Lemma 7, w_1 is the weight that leads to a function f_{w_1} that has zero L_2-norm distance from T f_{w_0}. But this does not imply that they are pointwise equal. If my claim is correct and I did not misunderstood anything, isn't it possible that we have a large difference on a set of measure 0 (w.r.t. the Lebesgue measure)?I believe if we require that the design matrix does not sample from those small sets, the analysis should be fine. Please comment on this issue.* The value of k(s) in Assumption 4 is random, since the design matrix is random. This makes the upper bounds of both theorems random. Also the work of Lounici et al. 2011 is for fixed design. How does it affect the results?* One other issue is that you reuse the dataset in all iteration. It makes them dependent.[Minor] If you have enough space, please address these concerns.* L118:  Assumption 1 is much stronger than requiring that Q* belong to the function space. Is there any specific reason not to study the function approximation error? Is it because Proposition 1 in the supplementary material does not hold?* L095: Features are defined over the state space, but not state-action space.* L213: In the definition of Restricted Eigenvalues, should we have |J| \leq s or |J| \leq \tilde{s} ?* In both Theorems 1 and 2, the probability of the event is written in a non-standard form. Is it possible to simplify it in the usual (eps,delta) form?* What would happen if instead of l_{2,1}-norm on W, we use l_{1,1}? What is the motivation of using l_2-norm of the rows?* A short discussion on how to solve Equation (5)?* A relevant work is recently done by Brunskill and Li, 2013. Please compare with it.* L220: Is the distribution of samples rho or mu? Based on the definition of the concentrability coefficient, it seems that it should be mu.* L287: Typo: "Let X the" ---> "Let X be the"* In Assumption 5, is O^d orthogonal or orthonormal (as suggested on L102)?- Lounici, Pontil, Van De Geer, Tsybakov, et al, "Oracle inequalities and optimal inference under group sparsity," The Annals of Statistics, 2011.- Brunskill and Li, "Sample complexity of multi-task reinforcement learning," UAI, 2013.