Zero-Shot Recognition with Unreliable Attributes
In principle, zero-shot learning makes it possible to train a recognition model simply by specifying the category's attributes. For example, with classifiers for generic attributes like striped and four-legged, one can construct a classifier for the zebra category by enumerating which properties it possesses-even without providing zebra training images. In practice, however, the standard zero-shot paradigm suffers because attribute predictions in novel images are hard to get right. We propose a novel random forest approach to train zero-shot models that explicitly accounts for the unreliability of attribute predictions. By leveraging statistics about each attribute's error tendencies, our method obtains more robust discriminative models for the unseen classes. We further devise extensions to handle the few-shot scenario and unreliable attribute descriptions. On three datasets, we demonstrate the benefit for visual category learning with zero or few training examples, a critical domain for rare categories or categories defined on the fly.

This paper presents a method for attribute-based zero-shot learning using random forests.  Attribute classifiers are first trained using SVMs.  A random forest is trained on class-attribute signature vectors, while using a validation set of images with attribute-labels is used to estimate attribute reliability at each node of the tree.  An extension to few-shot learning is also proposed.Clarity:  The paper is very clear and well-written.  It does a good job summarizing related work.Originality:  The novelty of this paper is somewhat low, but not too low to be accepted.  One issue is that the claim that the proposed approach deals with unreliable attribute predictions while earlier work does not is a bit overstated or misleading.  For example, direct attribute prediction [7] (DAP)--the baseline method used in this paper and many others--has some similar abilities to deal with attribute uncertainty.  Both approaches use the basic approach: a) Train attribute classifiers on image features (identical for both papers)b) Train a 2nd class probability estimator as a function of attribute predictions, using a validation set to estimate the reliability of attribute classifiers ([6] uses Platt scaling and a probabilistic model to combine attributes, whereas the proposed approach uses a new random forest-based method)In this way, I don't find the proposed approach to be a fundamentally new way of handling attribute uncertainty.  What it does differently than earlier work is that it uses a more complex model for modelling attribute noise: 1) it uses an ROC-based estimate instead of a sigmoidal model like Platt scaling, and 2) whereas [6] treats attributes as independent, the proposed approach effectively models joint statistics in attribute errors, since the validation set is propagated down trees.  In this way, I think it would be a little more appropriate to describe the approach as a better way to model attribute correlation (I don't know this area well, but Scheirer et al. "Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search" might be one related paper that could be cited).At the same time, one possible weakness of the proposed approach in comparison to methods that treat attributes independently is that it seems like one would need a large validation set to obtain valid probability estimates as it gets subdivided when propagated down decision trees.  This issue was glossed over in the paper, and proposed approach for dealing with this (line 323) seems heuristic.  It would have been nice to see more experiments evaluating the effect of training and validation set size.Quality: In general, the quality of the paper is good and the technical choices and descriptions make sense.  The experiments are appropriate and support the claims of the paper, although the level of quantitative improvement isn't high enough to accept the paper solely on the merit of empirical results.  The extension to few-shot learning (Section 3.3) felt less polished than the rest of the paper.  Whereas the rest of the paper was clear and explanations were lengthy, this section was very brief.  To me, a method that could combine class-attribute priors and image attribute estimates when estimating probabilities p(a|...) might make more intuitive sense then a weighted combination of information gains (Eq. 6).


Global Belief Recursive Neural Networks
Recursive Neural Networks have recently obtained state of the art performance on several natural language processing tasks. However, because of their feedforward architecture they cannot correctly predict phrase or word labels that are determined by context. This is a problem in tasks such as aspect-specific sentiment classification which tries to, for instance, predict that the word Android is positive in the sentence Android beats iOS. We introduce global belief recursive neural networks (GB-RNNs) which are based on the idea of extending purely feedforward neural networks to include one feedbackward step during inference. This allows phrase level predictions and representations to give feedback to words. We show the effectiveness of this model on the task of contextual sentiment analysis. We also show that dropout can improve RNN training and that a combination of unsupervised and supervised word vector representations performs better than either alone. The feedbackward step improves F1 performance by 3% over the standard RNN on this task, obtains state-of-the-art performance on the SemEval 2013 challenge and can accurately predict the sentiment of specific entities.

This paper introduces "belief propagation recursive neural	networks". This kind of networks extends the "feedforward" (bottom-up)	recursive networks with a backward (top-down) step during	inference. This allows phrase level predictions and embeddings to	give feedback to word embeddings and labels.  	This paper is overall well written. The model is well	motivated and described. This extends the idea of the	Bidirectional Recursive Neural Networks introduced by Irsoy	and Cardie 2008 (note that the reference to that paper is	incomplete). For instance one contribution is the introduction of hybrid word vectors.           Maybe this is the most important difference with previous work ?          Moreover, experimental comparisons are not completely fair: were the RNN and B-RNN also use hybrid vectors ? how the "best" models are tuned and selected ? How the single best is selected ? It is worth noticing that the others methods do not use an ensemble. These points must be clarified.         The term "belief propagation" is misleading and	maybe the term forward-backward could be well suited. 	Section 3.5 (Training) is a little bit too short and could	provide more details. For instance, I guess that the training	use the back-propagation through structure algorithm (the paper	of Goller 1996 could be cited). This algorithm implies that	the recursive model is unfolded. In this case, this yields a	very deep network to "reach" the word embedding part that is	updated. If I understand correctly, I think that the authors	could provide more details on this very important step.  	Section 3.6 is a little bit confused and the difference	between this model and the wok of Irsoy and Cardie must be	clarified. 	Section 4.1 must be improved to be understood by readers	outside of the NLP community. In the task 2 of Semeval 2013	there are 2 subtasks: contextual and message polarity.  I	guess that you address the contextual polarity task. With a	better presentation of the task, one can even understand the	parser constraints. 	Maybe, sections  4.3 and 4.4 could be merged.


Learning Distributed Representations for Structured Output Prediction
In recent years, distributed representations of inputs have led to performance gains in many applications by allowing statistical information to be shared across inputs. However, the predicted outputs (labels, and more generally structures) are still treated as discrete objects even though outputs are often not discrete units of meaning. In this paper, we present a new formulation for structured prediction where we represent individual labels in a structure as dense vectors and allow semantically similar labels to share parameters. We extend this representation to larger structures by defining compositionality using tensor products to give a natural generalization of standard structured prediction approaches. We define a learning objective for jointly learning the model parameters and the label vectors and propose an alternating minimization algorithm for learning. We show that our formulation outperforms structural SVM baselines in two tasks: multiclass document classification and part-of-speech tagging.

This is a very inspirational and well-written paper that challenges a fundamental assumption in many supervised classification tasks, namely that class labels are always distinct, discrete units of meaning.Instead, the paper argues that class labels can expose common traits and that some labels are closer to each other than others, a fact that should be exploited during training. Of course, this viewpoint is not entirely new -- such "distributional" arguments have been made in the connectionist literature before, regarding both input and output units, as correctly pointed out in the manuscript.The main novelty in the present manuscript is then to extend such distributional formulations to structured prediction, where not only atomic labels but also compositional parts (such as labels of factors defined over multiple variables) can be modeled in a distributional manner.Towards this end, a model is proposed in which labels of atomic parts are represented as vectors, and compositional parts as well as the corresponding feature spaces are defined in terms of tensor algebra.  A traditional structured prediction model then corresponds to the special case where each atomic label is represented via one-hot encoding.What makes the approach interesting is that both labels and model weights can be learned together in a joint objective function, based on the structured hinge loss.  A rank-penalty regularization term is placed on the matrix of label vectors to encourage the labels to share weights.  The parameters constituting the label vectors and the weights on features are trained by alternating minimization, which seems to work well most of the time, though the objective is not jointly convex.The experiments demonstrate considerable improvements over a standard hinge loss formulation on both multiclass classification and sequence classification.  Interestingly, the approach also allows to learn labels in a lower-dimensional vector space than the standard one-hot encoding, while maintaining excellent performance even for substantially reduced dimensionality.Quality:The paper is technically sound.  The theory backing the approach is plausible and formally developed to a sufficient extent in the manuscript.  The experiments are on the synthetic side, but still demonstrate the plausibility of the intuition underlying the paper convincingly.Clarity:The paper is well-written and the authors take care to give pedagogical examples to make the material easier to understand. A background in tensor algebra is still helpful when reading the paper.Figure 1 runs over the right margin considerably and should be fixed.Originality:The extension of the distributional approach to compositional parts is novel to my knowledge.Significance:   At present, I think neither the theory nor the practical tools (such as the learning algorithm, which seems to struggle with non-convexity now and then) are sufficiently developed for the approach to find broad adoption in the near future.  Also, there needs to be further evidence that the proposed approach yields state-of-the-art results on established benchmark tasks.  Nonetheless, the proposed approach is inspiring and seems to have a lot of potential.


Optimal decision-making with time-varying evidence reliability
Previous theoretical and experimental work on optimal decision-making was restricted to the artificial setting of a reliability of the momentary sensory evidence that remained constant within single trials. The work presented here describes the computation and characterization of optimal decision-making in the more realistic case of an evidence reliability that varies across time even within a trial. It shows that, in this case, the optimal behavior is determined by a bound in the decision maker's belief that depends only on the current, but not the past, reliability. We furthermore demonstrate that simpler heuristics fail to match the optimal performance for certain characteristics of the process that determines the time-course of this reliability, causing a drop in reward rate by more than 50%.

Optimal decision-making with time-varying evidence reliabilityIn this article, the authors formulate an ideal observer model for making a decision based on accumulated evidence over time, where the evidence reliability varies over time.  They assume the reliability of the evidence follows a stochastic process (a Cox-Ingersoll-Ross process) and that there is a cost for waiting to act that is linear in time. The resulting computational model is complex, and so they apply Dynamic Programming and Partial Differential Equation solvers to approximate the model given some data. They then explore the behavior of their model for different scenarios (e.g., increasing/decreasing the standard deviation of the reliability) and outline some intuitions for how the parameters affect model behavior. Finally, they compare model behavior to some simple heuristics to outline why those heuristics fail when evidence reliability is actually variable.The article tackles a difficult problem that is of interest to many within the NIPS community and makes theoretical advances, although they may be incremental advances. It uses mathematical techniques that are not currently prominent within the NIPS community (e.g., applying partial differential equation solvers to stochastic processes), which could have a positive impact on other areas with equivalent mathematical models. As a cognitive scientist with a moderate background in stochastic process theory, the mathematics in the article are beyond my ability to evaluate properly (beyond cursory reasonableness and consistency). So I will leave the critique of the mathematics to other reviewers. From this perspective, I found the article difficult to follow at times, not because it is poorly written (it is actually fairly well written), but because it is dense and the mathematics are difficult. I would have benefited from some more thorough explanations of the underlying assumptions (why a Cox-Ingersoll-Ross process as opposed to other continuous stochastic processes?) and some more thorough explanations of the figures (which are quite dense). For future work, it might be useful for the authors to explore using a more discontinuous stochastic process for the evidence reliability. Intuitively, it seems like evidence reliability is likely to stay constant except for when there is a change in the environment (in which case there would be a large jump). This is not meant as a criticism (the authors’ approach seems reasonable and to the best of my knowledge, the first of its kind), but rather as a suggestion for future work.Also, the article feels incomplete because they setup and develop a nice formalism, but never test it or compare it to previous empirical data. However, there is more than enough theoretical work in the article to justify its worth, and so it reflects my own bias. In fact, presenting the article at NIPS could result in useful collaborations to test the model.Minor comments:“stead” -> “steady” (in many places, it says “stead” state rather than “steady” state)


Probabilistic ODE Solvers with Runge-Kutta Means
Runge-Kutta methods are the classic family of solvers for ordinary differential equations (ODEs), and the basis for the state of the art. Like most numerical methods, they return point estimates. We construct a family of probabilistic numerical methods that instead return a Gauss-Markov process defining a probability distribution over the ODE solution. In contrast to prior work, we construct this family such that posterior means match the outputs of the Runge-Kutta family exactly, thus inheriting their proven good properties. Remaining degrees of freedom not identified by the match to Runge-Kutta are chosen such that the posterior probability measure fits the observed structure of the ODE. Our results shed light on the structure of Runge-Kutta solvers from a new direction, provide a richer, probabilistic output, have low computational cost, and raise new research questions.

1. SummaryThis paper presents a probabilistic approach to ODE solvers based on Gaussian processes so that the posterior mean matches exactly the Runge-Kutta estimates. The main contribution of the paper is that it shows that Runge-Kutta solutions of order 1, 2, 3 can be obtained from integrated Weiner processes. 2. QualityThe paper is technically sound as it exploits the closeness property of GPs under linear operators to come up with covariance functions and, consequently, predictive means that match the Runge-Kutta solutions. The claims regarding the matching of these solutions are well supported by the analytical derivations in the paper and in the supplemental material. The claim regarding the low computational cost is also well supported, as it falls directly from the resulting model being a Gauss Markov process. One possible weakness of this work is the claim in the main paper regarding previous ODE solvers that they do not match exactly the Runge-Kutta estimate. This contradicts what is presented in the supplementary material where the authors (line 448 onwards) that Equation (42) in the supplemental material can be used to derive 2nd order and 3rd order Runge-Kutta means. Although such a match is only achieved when \lambda tends to infinity, there should be more discussion in the main paper regarding both approaches. In fact, the authors do recognize (line 414) that the “application of different GP models might still be favorable in practice”, but fall short in providing more discussion or more experimentation comparing the proposed method with previous work.  One additional technical comment is the setting of the “scale” parameter \sigma^2, which seems to be done ad-hoc. Why haven’t  he authors pursued a more principled way to provide point-estimates for this since it ultimately calibrates the the resulting posterior distribution?3. ClarityThe paper is very well written and organized and it contains about the right balance of background and new material. 4. OriginalityThe paper addresses an existing problem, that of providing probabilistic interpretations for ODE solvers. It approaches this problem in a very different way to previous work as it goes beyond simply using GPs and exploiting their closeness under linear operators by showing equivalence results when starting from the kernel for the Wiener process. 5. SignificanceThe results are important as they provide a better understanding of Runge-Kutta methods from a probabilistic point of view. However, the paper will benefit greatly from a more careful comparison wrt to previous probabilistic approaches to the same problem. 6. Minor comments(a) line 268: "Its finite values that this posterior indeed defines a proper GP". I don't think this sentence quite makes sense(b) line 295: "an correction" --> "a correction"


Tree-structured Gaussian Process Approximations
Gaussian process regression can be accelerated by constructing a small pseudodataset to summarize the observed data. This idea sits at the heart of many approximation schemes, but such an approach requires the number of pseudo-datapoints to be scaled with the range of the input space if the accuracy of the approximation is to be maintained. This presents problems in time-series settings or in spatial datasets where large numbers of pseudo-datapoints are required since computation typically scales quadratically with the pseudo-dataset size. In this paper we devise an approximation whose complexity grows linearly with the number of pseudo-datapoints. This is achieved by imposing a tree or chain structure on the pseudo-datapoints and calibrating the approximation using a Kullback-Leibler (KL) minimization. Inference and learning can then be performed efficiently using the Gaussian belief propagation algorithm. We demonstrate the validity of our approach on a set of challenging regression tasks including missing data imputation for audio and spatial datasets. We trace out the speed-accuracy trade-off for the new method and show that the frontier dominates those obtained from a large number of existing approximation techniques.

1. SummaryThis paper proposes a sparse method to (approximate) large scale inference in Gaussian process (GP) regression models. The main idea builds upon the inducing-point formalism underpinning most sparse methods for GP inference. As the computational cost of traditional sparse methods in GPs based on inducing points is O(NM^2), where N is the number of observations and M is the number of inducing points, the paper addresses the problem of large-scale inference by making conditional independence assumptions across inducing points. More specifically, the method proposed in the paper can be seen as a modified version of the partially independent conditional (PIC) approach, where not only the latent functions are grouped in blocks but also the inducing points  are clustered in blocks (corresponding to those latent functions) and statistical dependences across inducing point blocks are modeled with a tree.These additional independence assumptions make the resulting inference algorithm much more scalable as it only scales (potentially) linearly with the number of observations and the number of inducing points.  The method is evaluated on 1D and 2D problems showing that it outperforms standard sparse GP approximations.  2. QualityThe paper is technically sound in that it builds upon well-know technically sound approaches such as GP sparse approximations based on inducing points. The claim of linear computational cost as a function of the number of observations and/or the number of inducing points is correct when having very small block-sizes, as the cost is cubic on the size of the block. However, it is surprising that the authors do not discuss what is potentially lost by the conditional independence assumptions underlying their approximation. The whole idea of GPs is to be able to model highly coupled observations, i.e. long-term dependencies.  There is a lot to loose by assuming a “block-tree” structure in the inducing points, which is equivalent to making sparsity assumptions in their covariance. Their 1D and 2D experiments are not entirely convincing of the effectiveness of their approach to more general problems (see item 5 below).There are also fundamental parts in the inference algorithm that are not obtained from first principles in a single probabilistic framework. For example:(a) The partitioning of the data (and the inducing inputs) into blocks(b) The selection/learning of the inducing points (c) The learning of the tree structure using distances (although the algorithm for learning is sound)All these  are performed in an ad-hoc manner, which is uncharacteristic of a NIPS paper. A fundamental criticism to this paper is that it appears to refer to traditional sparse GPs such as FITC, etc, as if they were state-of-the-art in sparse GP approximations. Clearly this is not the case. For example, (i) The learning of inducing points have done before in sparse GP models and has been shown to be advantageous [Snelson and Ghahramani, 2006; Titsias, 2009]. (ii) The work in [Hensman et al, 2013] has shown that scalability at very large datasets is possible and that these should be the new benchmarks for sparse GPs. In terms of completeness,  although the technique developed is not specific to 1D or 2D, the experiments are only carried out on these types of applications. Hence, the paper will greatly benefit from experiments on higher dimensional spaces. 3. ClarityOverall, the paper is well written and organized. The introduction of sparse GP approximations as a three-step process and the use of the KL divergence in the 3rd step is at best uninformative. For example, there is nothing mysterious (line 110) to sparse GPs understood within a common probabilistic framework  [Quinonero-Candela et al, 2005]. The introduction of the KL divergence is unnecessary and distracting of the main objective of the paper. Approximations such as FITC, PITC, PIC make some assumptions in the prior and of course these can also be written as a KL between the approximate prior and the true prior. Variational methods (such as VFE) and EP are, by definition, optimizing a KL divergence  but such methods are more concerned with dealing with analytically intractable posteriors. In terms of reproducibility, the paper does not provide enough detail for an expert reader to reproduce  the experimental results. There is not detail regarding how the benchmark algorithms were executed (optimization details, etc) and vague sentences are used when referring to the execution of the proposed algorithm (e.g. “until a convergence criteria or a maximum number of function evaluations was reached”).4. OriginalityNeither the problem or the approaches presented in the paper is new. Standard regression problems with Gaussian noise are addressed, and the method proposed builds upon well-know sparse approximation techniques for GPs. The main computational advantage comes from making additional conditional independence assumptions across inducing points, whose impact is not discussed in the paper.  As mentioned in item 2 (Quality) above, the authors seem unaware of the state of the art regarding GP approximations. 5. SignificanceThe results presented in the paper are very limited as they only concern 1D and 2D regression problems. the paper will greatly benefit from experiments on higher dimensional spaces. It is not clear how the paper will advance the state of the art as it does not compare with the most recent advances in GP approximations. 6. References [Hensman et al, 2013]. Gaussian processes for big data. In UAI, 2013.[Quinonero-Candela et al, 2005] A Unifying View of Sparse Approximate Gaussian Process Regression. in JMLR 2005.[Snelson and Ghahramani, 2006] Sparse Gaussian Processes using Pseudo-inputs. In NIPS 2006.[Titsias, 2009] Variational Learning of inducing variables in Sparse Gaussian Processes. In AISTATS 2009.


Ranking via Robust Binary Classification
We propose RoBiRank, a ranking algorithm that is motivated by observing a close connection between evaluation metrics for learning to rank and loss functions for robust classification. It shows competitive performance on standard benchmark datasets against a number of other representative algorithms in the literature. We also discuss extensions of RoBiRank to large scale problems where explicit feature vectors and scores are not given. We show that RoBiRank can be efficiently parallelized across a large number of machines; for a task that requires 386, 133 × 49, 824, 519 pairwise interactions between items to be ranked, RoBiRank finds solutions that are of dramatically higher quality than that can be found by a state-of-the-art competitor algorithm, given the same amount of wall-clock time for computation.

In this paper, the authors proposed a ranking algorithm named RoBiRank, which is motivated by their interesting observation of the connection between evaluation metrics for learning to rank and loss functions for robust classification. The proposed RoBiRank shows competitive performance comparing with recent advances, and further the authors extend the algorithm in order to be applied for large scale problems. Quality: this paper brings the connection between two seemly unrelated problems -- robust classification and learning to rank under a general framework, and proposes to address the problem of learning to rank from a robust binary classification perspective. The proposed loss function is not convex, and the authors propose to optimize the lower bound of the metrics, which is continuous and twice differentiable. The standard l2 regularizer is applied to the model, and the optimization is performed via the standard gradient-based optimization approach. The empirical results are extensive given the use of multiple real-world datasets and comparing with 8 existing algorithms. In addition, the authors further proposed the linearization trick leading to parallelization for the optimization, and demonstrated the performance on the Million Song Dataset; by comparing with Weston et al, the authors demonstrated that their algorithm outperforms the earlier work. Clarity: the paper is overall well written, and there are just a few very minor problems that I have mentioned below. Originality: the proposed new loss function for ranking can be seen as (1) applying logistic loss on the difference between the item-context utility between an observed pairs, and (2) further applying a transformation function with slower rate that can lead to robustness. Note that the transformation function used in the Step 2 has been proposed in Ding [9] as cited in the paper. Given these facts and the use of the standard l2 norm on the weight, I found that the model itself is incremental. But I feel that the parallelization for the optimization algorithm is non-trivial. Significance: the proposed model and algorithm enjoyed better performances evaluated with different metrics (e.g., precision, NDCG), based on which I think the paper is making significant contributions to the machine learning community. I have some questions/comments below: 1. The notation \Y_{x} does not seem to be define when first introduced in line no. 119. It seems that this notation is defined later in line no. 255. and I suggest that the authors correct this minor problem. 2. Could the authors provide some analyses concerning how tight is bound for the objective function in (10)?3. Concerning the evaluation metrics, I am wondering if the NDCG is the correct metric used to report the results. The reason is that the proposed model in this paper is optimization this metric, while for Rank-SVM the objective function is not NDCG.  4. could the author provide further analysis regarding the convergence rate for the proposed algorithm?


Shaping Social Activity by Incentivizing Users MPI for Software Systems †
Events in an online social network can be categorized roughly into endogenous events, where users just respond to the actions of their neighbors within the network, or exogenous events, where users take actions due to drives external to the network. How much external drive should be provided to each user, such that the network activity can be steered towards a target state? In this paper, we model social events using multivariate Hawkes processes, which can capture both endogenous and exogenous event intensities, and derive a time dependent linear relation between the intensity of exogenous events and the overall network activity. Exploiting this connection, we develop a convex optimization framework for determining the required level of external drive in order for the network to reach a desired activity level. We experimented with event data gathered from Twitter, and show that our method can steer the activity of the network more accurately than alternatives.

Quality:The paper is technically sound, well-written, and addresses a solid problem which one could connect to practical applications. Activity shaping is at least as hard as influence maximization.The main weakness of the paper are the experimental results. The results must necessarily be done via simulation, because activity shaping is an interventional problem, and they do not have a social network into which they could intervene. The metric for comparing heuristics is not all that compelling, but the authors take a lot of care to show their approach competes against reasonable baselines, on simulated data.Clarity:The paper is well written, and accessible to a reader who is not familiar with Hawkes processes. Section 2 provides a good, self-contained, explanation of the necessary background material on point processes and branching processes. Significance:The foundational assumption appears to be that the exogenous events (shaping) happens only at the beginning of the process. All subsequent evolution in the point process is endogenous. The paper’s assumptions diverge from reality if, for example,1. The shaping incentives are doled out throughout the evolution of the process.2. There are multiple competing or interacting incentives (though, I believe the Hawkes process construction could handle this case).The authors do get something for making this assumption: "a linear relation between the expected intensity of the network and the intensity of the exogenous events" (L192), which turns activity shaping into a convex optimization.I’d welcome some commentary on whether the Hawkes process construction would work if incentives were doled out throughout the process, e.g., in a sequential decision making setting.Originality:As the authors point out, Hawkes processes have had some prominence in the ML literature in the last couple of years. This paper is a solid extension that line of thinking: i.e., taking a self-exciting process, and applying it to a social network which exhibits self-exciting behavior (cascades). Solid incremental work, which describes most papers accepted at NIPS.Once the problem is framed as a convex optimization in Section 4, the optimization itself is a straightforward projected gradient method (Algorithm 1). The main significance of the paper is the formulation of activity shaping as a convex optimization, using a Hawkes process.Questions:L262: Is the matrix exponential in Section 5 a consequence of the specific kernel chosen for this paper, or is it required for any kernel G ?Misc:Bibliography: Check your Bibtex (e.g., “hawkes” vs. “Hawkes”)


Discriminative Unsupervised Feature Learning with Convolutional Neural Networks
Current methods for training convolutional neural networks depend on large amounts of labeled samples for supervised training. In this paper we present an approach for training a convolutional neural network using only unlabeled data. We train the network to discriminate between a set of surrogate classes. Each surrogate class is formed by applying a variety of transformations to a randomly sampled 'seed' image patch. We find that this simple feature learning algorithm is surprisingly successful when applied to visual object recognition. The feature representation learned by our algorithm achieves classification results matching or outperforming the current state-of-the-art for unsupervised learning on several popular datasets (STL-10, CIFAR-10, Caltech-101).

This paper presents a new approach to discriminative unsupervised feature learning. Given an unlabelled set of images, 'seed' patches are extracted which act as surrogate classes for training. For each of these seed images, a variety of transformations is applied to generate training examples within the seed classes. A convolutional network is trained on top of the patches to predict the seed classes. After training, feature are extracted by forward propagating a new image, extracting the feature maps at each layer and pooling over them. Experimentation is performed on several standard image classification benchmarks, with additional experimentation to determine the effectiveness of transformations and invariance properties of the learned features.The proposed approach is simple and demonstrates strong performance on each of the classification tasks. The additional analysis, particularly the effects of removing transformations, was most welcome. The paper is clearly written for the most part with enough information given for the reader to reproduce the results.Detailed comments / questions:- Can you go into more detail about patch extraction? Are you just sampling a large number of patches, computing the gradients for each one, and choosing the top-N?- You should mention that the STL-10 dataset uses 96x96 images earlier in the paper, in order to give context for the choice of 32x32 patch size.- Was the 32x32 patch size arbitrary chosen? Do you have any intuition on how to select patch sizes as a function of the image sizes?- section 4.1: A picture or two, perhaps one illustrating the convnet during surrogate training and another illustrating feature extraction would be really helpful. The description is fairly clear for a reader familiar with this line of work but may be challenging to understand for those that are not.- From a practical point of view, given a dataset with a small amount of labels, one of the simplest things to try is feeding the images through a pre-trained convolutional net on ImageNet and training a linear classifer on top of these features. I think it would beneficial to include these results in table 1, even through they fall under a different category of algorithms (you could include this in a separate block of the table). It would be interesting to see how your results compare (at least on Caltech 101, your results are on par to DeCAF)- How much does validation on the surrogate task act as a proxy for validation on the classification task? Does the network with the best validation performance on the surrogate task also have the best validation performance on the classification task? Figure 3 hints that this might be true. Table 1 in the supplementary material presents classification accuracies with several networks. I would highly recommended including (maybe in a separate table) both the surrogate validation scores as well as the classification validation scores for each of these tasks, to see how well they correspond.- Supplementary, table 2: How come the diagonal is different than the results in table 1 with the same network?- To my best knowledge, the best published result on STL-10 is 70.1% +- 0.6% from "Multi-Task Bayesian Optimization" (Swersky et al, NIPS 2013). They achieved their results by first extracting k-means feature maps and training a convolutional network on top of these. By fixing the k-means features, the network is much less prone to overfitting. A strong baseline that I'd recommend the authors try, if time permits, is to use the same approach they do but include each of the transformations you use in the paper. This give a relatively straightforward way of running convnets on small labelled datasets which I suspect would be very competitive with your approach.


Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such minimizations, and it is often thought that a main source of difficulty for these local methods to find the global minimum is the proliferation of local minima with much higher error than the global minimum. Here we argue, based on results from statistical physics, random matrix theory, neural network theory, and empirical evidence, that a deeper and more profound difficulty originates from the proliferation of saddle points, not local minima, especially in high dimensional problems of practical interest. Such saddle points are surrounded by high error plateaus that can dramatically slow down learning, and give the illusory impression of the existence of a local minimum. Motivated by these arguments, we propose a new approach to second-order optimization, the saddle-free Newton method, that can rapidly escape high dimensional saddle points, unlike gradient descent and quasi-Newton methods. We apply this algorithm to deep or recurrent neural network training, and provide numerical evidence for its superior optimization performance.

---------------------Basically, I think it's really important to show that |H| is better than the Fisher/Natural Gradient matrix in an "apples to apples" comparison.  Same setup, same learning rate heuristics, only different curvature.   It's important because the Fisher matrix is known to be much better for nonlinear optimization than the Hessian.  The experiment that you mention show that |H| is also much better than the Hessian.  So the big question is, which is better:  |H| or F, where both matrices are used in the same setup.---------------------Optimization methods assume that they are given convex approximations to the objective function.  However, this assumption is clearly false due to the presence of the large number of saddle points in high dimensional spaces.  The idea of explicitly modelling and dealing with saddle points is very interesting, and the results are intriguing:  working with the absolute value of the eigenvalues really does seem to result in better performance on the tasks considered.However, I feel that the experiments do not quite prove the point of the paper as well as it could have.  Specifically,  the experiments do not show convincingly that working with the absolute value of the Hessian is better than the Natural Gradient, or the Gauss Newton.  To be precise, I noticed that Saddle Free Newton was initialized by SGD.  In contrast, previous work on second order methods, including Martens (2010), did not initialize from SGD.  And while you report better results, it is possible that most of the improvement was caused by the use of SGD as a “pretraining” stage of the second order.  Thus the paper would be much more convincing if alongside SFN it included the same experiment (with the same optimizer, Krylov Subspace Descent), with the Gauss-Newton matrix.   At present, there is only a comparison with damped newton. The second issue, which is small yet significant, is that too much space and emphasis was placed on unnecessary formal justification of the method, introducing a new trust region method (which isn’t new --- the classical derivation of natural gradients minimizes the linear gradient over a quadratic constraint, which is precisely equivalent to an elliptical trust region method).  Likewise, the |H| matrix has positive eigenvalues, and therefore dividing by this positive definite matrix is a proper optimization procedure.  I strongly recommend the authors to replace the Hessian with the Gauss-Newton / Natural Gradient matrix while keeping everything else fixed (and spending the appropriate amount of time tuning the new setting), in order to see whether it is the Hessian that contributes to the better performance, or whether it is the SGD pre-training.


Learning convolution filters for inverse covariance estimation of neural network connectivity
We consider the problem of inferring direct neural network connections from Calcium imaging time series. Inverse covariance estimation has proven to be a fast and accurate method for learning macro-and micro-scale network connectivity in the brain and in a recent Kaggle Connectomics competition inverse covariance was the main component of several top ten solutions, including our own and the winning team's algorithm. However, the accuracy of inverse covariance estimation is highly sensitive to signal preprocessing of the Calcium fluorescence time series. Furthermore, brute force optimization methods such as grid search and coordinate ascent over signal processing parameters is a time intensive process, where learning may take several days and parameters that optimize one network may not generalize to networks with different size and parameters. In this paper we show how inverse covariance estimation can be dramatically improved using a simple convolution filter prior to applying sample covariance. Furthermore, these signal processing parameters can be learned quickly using a supervised optimization algorithm. In particular, we maximize a binomial log-likelihood loss function with respect to a convolution filter of the time series and the inverse covariance regularization parameter. Our proposed algorithm is relatively fast on networks the size of those in the competition (1000 neurons), producing AUC scores with similar accuracy to the winning solution in training time under 2 hours on a cpu. Prediction on new networks of the same size is carried out in less than 15 minutes, the time it takes to read in the data and write out the solution.

This paper addresses the reconstruction of network topology from calcium imaging data, using inverse covariance matrix estimation. It is shown empirically that a simple convolution filter (to be applied to the calcium traces) can be learned (once and for all) that substantially improves the reconstruction performance, and the time it takes to infer connectivity on new datasets.----QUALITYOverall, what is done seems solid. There are a couple of wrinkles that I would like the authors to clarify.First: the authors restricted the length of the convolution filter to 10 time steps. The learned filter peaks at time step 8, which left me to wonder whether there was enough room for the filter to "converge" (in "time lag space"). For example, had it been restricted to 5 time steps, the authors would have missed this presumably crucial peak. Was that a computational restriction? If not, is there a biophysical (or empirical?) reason why correlations would fall off anyway after 10 time steps? The authors should probably discuss this.Second: although I understand that the study is focused on improving the inverse covariance method specifically, I would have liked to hear more of a discussion regarding the inherent limitations of the method. For example, the fact that only an undirected graph can be extracted seems like a big restriction as far as neural circuits are concerned.Third, how crucial is it to get the value of $\chi$ in Eq.7 in the right ballpark? In particular, how does reconstruction generalize to datasets of very different sparsity than the one assumed for training?CLARITYThe paper is well-written and well structured.It'd be great if the Kaggle dataset (e.g. the fact that the ground truth connectivity is known) could be described upfront (at the beginning of the methods) in 1/2 sentences.  I would also like the authors to clarify the timescales for the non-experts (e.g. what does "one time step" in the convolution filter mean? what does "time (20 ms)" mean exactly in the x-axis labels? that 20ms are shown in total, or 1000*20ms = 20 sec?). It would also help the readers to appreciate the difficulty of the task.Also, AUC is nowhere defined (not even spelled out -- Area Under the (ROC) Curve?).ORIGINALITY I'm not an expert in this specific calcium imaging literature, but it seems surprising that nobody had tried (even heuristic) convolution filters on top of calcium data prior to covariance estimation before... Anyhow, the work presented here is original, and clearly improves on the current leaderboard for Kaggle.SIGNIFICANCEEstimating network topology has very important implications for the neurosciences, especially with the advent of whole-network imaging techniques. There definitely is a need for statistical methods. While this study provides state-of-the-art performance and speed, I believe it remains essentially a simple (though important) addition to a known algorithm (L2-regularized inverse covariance estimation) which unfortunately does not address its most inherent limitations (e.g.  underlying, implicit Gaussian assumption, undirected graph recovery, ...).


Analysis of Variational Bayesian Latent Dirichlet Allocation: Weaker Sparsity than MAP
Latent Dirichlet allocation (LDA) is a popular generative model of various objects such as texts and images, where an object is expressed as a mixture of latent topics. In this paper, we theoretically investigate variational Bayesian (VB) learning in LDA. More specifically, we analytically derive the leading term of the VB free energy under an asymptotic setup, and show that there exist transition thresholds in Dirichlet hyperparameters around which the sparsity-inducing behavior drastically changes. Then we further theoretically reveal the notable phenomenon that VB tends to induce weaker sparsity than MAP in the LDA model, which is opposed to other models. We experimentally demonstrate the practical validity of our asymptotic theory on real-world Last.FM music data.

In this paper, authors analyze sparsity of the posterior parameters in LDA using a variational Bayesian algorithm. They derive an expression for the VB free energy which shows its asymptotic behaviour with respect to number of words (N), number of documents (M), vocabulary size (L) etc. Their results suggest that, for certain settings of L,M,N,  the sparsity behaviour changes drastically at a particular hyper-parameter setting. These changes differ from those of MAP and partial-Bayes algorithms.The problem discussed in this paper is original, interesting, and is perhaps useful too.  The arguments are supported by rigorous theorems. I found the paper to be really hard to read and understand. The paper is not written clearly, and there are many steps in the derivations that are not outlined. Therefore, I was unable to understand the derivation the theorem. I have few suggestions below to improve this part. Another big problem that I find with this work is the context. I think authors need to discuss the significance of their results in the context of real data. Perhaps, a more extensive experiment section will be useful. I have some suggestions on this below.Detailed comments:The paper discusses, at several occasion (e.g. second paragraph), sparsity in general latent variable models (e.g. mixture model etc.). If the notion of sparsity is comparable in these models, then it will be good to include a general definition of sparsity in latent variable models. Otherwise, it will be good to include a discussion of what exactly is meant by weaker or stronger sparsity in these models, e.g. mixture model, and how does it compare to sparsity in LDA.The LDA notation used in the paper is not-standard and makes it difficult to read for people familiar with LDA. I highly recommend to change the notation. Also, please include a graphical model in Section 2.1. Use q for approximate distribution instead of r. Paragraph at line 185 is not clear. It is perhaps clearer to state that you assume A* and B* of rank K* that can model the data well.Define O_p for people who are not familiar with it.Explain more clearly the average number of nonzero entries in line 201.In Line 253, you say "Note that Theorem 1 combined with Lemma 1 implies the consistency". This isn't trivial (at least to me), although I can see how to get there. It is perhaps a good idea to write these results as theorem rather than the Lemma that you have. At least it gives a clear statement to the reader about what is being proved in the paper. You should then include a detailed proof (really detailed) in the Appendix. I will suggest the same for Corollary 1 and 2. I do not see how you arrive at these results so trivially. It will be good have details on this in the Appendix. The proofs in Appendix need to be more detailed too. Currently, there are statements that are not easy to derive, or at least I could not derive them myself. E.g. line 511, "because of the independence between A and B, the posterior distribution must converge to a point... ". I don't know why? In line 522, "we can show that...". Unfortunately, I wasn't able to show this, so some help will make the experience for readers less frustrating. It was so discouraging that I couldn't make it beyond the first page of Appendix.Please provide some context for Corollary 1 and 2. You have discussed two settings of L,M,N where you have clear statements. What are the significance of these cases? when they are likely to occur? do real data have one setting more than others? If we only know about sparsity of B, can we still conclude something general about the whole situation?Please make a table comparing PBB, PBA, MAP and VB vs. settings of c_a and c_b. This will improve clarity in summarizing the results. It seems right now that integrating A out gives the sparsity at c_a=1/2 behaviour. Do you think that will be true in general?Line 305, MAP tends to induce stronger sparsity. Please clarify this further. I assume you mean that VB does not include c_a >1/2 cases therefore will have less sparsity? Line 323. L and M would be comparable to N in standard text analysis. I don't think it's correct to say that.Results section. Why don't you include other text datasets on which LDA has been applied more frequently rather than LastFM? Also provide a reference that applies LDA on lastFM.Please include more datasets. Remove c_b from plots if it doesn't affect the sparsity. Rather include more datasets and more different scenarios for different values of L,M, and N.


Weakly-supervised Discovery of Visual Pattern Configurations
The prominence of weakly labeled data gives rise to a growing demand for object detection methods that can cope with minimal supervision. We propose an approach that automatically identifies discriminative configurations of visual patterns that are characteristic of a given object class. We formulate the problem as a constrained submodular optimization problem and demonstrate the benefits of the discovered configurations in remedying mislocalizations and finding informative positive and negative training examples. Together, these lead to state-of-the-art weakly-supervised detection results on the challenging PASCAL VOC dataset.

The goal of this paper is to learn an object detector that produces tight bounding boxes around objects given training data that contains only weak annotations of presence/absence of objects without bounding box supervision. The paper develops an initialization for the latent SVM detector of Felzenszwalb et al. The paper first obtains many candidate part bounding boxes in all training images using selective search [28]. Then the method proceeds by selecting a set of distinctive object parts that occur (are matched) on the positive images and not on the negative data, but do not significantly overlap each other. This is formulated as a constrained discrete optimization problem and approximately solved using a greedy algorithm. Finally, to better localize the object in the image the paper selects pairs of discriminative parts that tend to co-occurr in a consistent mutual geometric configuration (position, scale and aspect ratio). Bounding boxes obtained from such consistent pairs are then used to train the LSVM object detector.Novelty:The work is somewhat incremental w.r.t. (Song et al., ICML'14 [27]).The main differences are:1.  finding multiple non-overlapping discriminative parts per image. This is in contrast to a single part per image found in [Song et al. ICML'14] and seems as a step in the right direction. 2. finding pairs of parts that occur in a certain configuration that is based on the algorithm of (Li et al., CVPR 2012 [17]) and involves voting in a 4D part-to-part transformation space.Pairs of co-occurring parts are used to better localize the object in the image.While the paper is incremental, the novelty seems sufficient for publication. Experiments:- Results are shown on the Pascal VOC 2007 detection dataset.Results on the test data slightly outperform the method of Song et al. (mAP 24.6 vs. mAP 22.7). The paper also experimentally demonstrates the (slight) improvement of the proposed hard-negative mining procedure that extracts (hard) negative bounding boxes from the positive images.  - The paper also shows qualitative (fig.5) demonstrating the output object localization for several object classes. The qualitative results seem reasonable. Impact:While this is a solid work, given the somewhat incremental nature w.r.t. Song et al. its impact is probably going to be limited. In general, the practical impact of weakly supervised learning methods is going to be determined by the scalabality of the method to large datasets with many images and many classes. What is the complexity of the algorithm? How does the algorithm scale to many images and many labels?  Detailed questions:- L296: It seems that each iteration adds only one additional positive example.How many iterations are performed? How is the algorithm stopped?- The paper claims that the algorithm easily extends beyond pairs of parts to three or more parts. This is not clear to me. The authors should elaborate on this in the rebuttal.- The authors should discuss the scalability of the proposed method to large datasets and many object categories.


Scalable Inference for Neuronal Connectivity from Calcium Imaging
Fluorescent calcium imaging provides a potentially powerful tool for inferring connectivity in neural circuits with up to thousands of neurons. However, a key challenge in using calcium imaging for connectivity detection is that current systems often have a temporal response and frame rate that can be orders of magnitude slower than the underlying neural spiking process. Bayesian inference methods based on expectation-maximization (EM) have been proposed to overcome these limitations, but are often computationally demanding since the E-step in the EM procedure typically involves state estimation for a high-dimensional nonlinear dynamical system. In this work, we propose a computationally fast method for the state estimation based on a hybrid of loopy belief propagation and approximate message passing (AMP). The key insight is that a neural system as viewed through calcium imaging can be factorized into simple scalar dynamical systems for each neuron with linear interconnections between the neurons. Using the structure, the updates in the proposed hybrid AMP methodology can be computed by a set of one-dimensional state estimation procedures and linear transforms with the connectivity matrix. This yields a computationally scalable method for inferring connectivity of large neural circuits. Simulations of the method on realistic neural networks demonstrate good accuracy with computation times that are potentially significantly faster than current approaches based on Markov Chain Monte Carlo methods.

Summary:The authors consider the problem of neural connectivity inference from calcium imaging observations. Their model follows the approach of [7] (Mischenko et. al 2011), a network of spiking neurons coupled together through a matrix of weights that is to be inferred. The important difference is that the spiking neurons are modeled as leaky integrate-and-fire (LIF) neurons instead of generalized linear model. Similarly to [7], they propose an EM approach where at the E step the hidden variables (spikes, membrane potential, calcium concentration) are estimated and the M step is used to estimate the connectivity matrix W and other parameters. The main contribution of the paper is the efficient implementation of the E step using a generalized approximate message passing algorithm (GAMP), which scales linearly both with the number of timesteps and the number of neurons. They also present a sparse probit regression initialization approach for W, that arises naturally from the structure and dynamics of the LIF neurons. The algorithm is applied on synthetic data and the authors claim to achieve performance at least similar to that of previous methods (e.g. [7]) but with significantly less computation.Comments- Why are all the nonzero connectivity weights excitatory? The authors cite simplicity but it looks to me that allowing negative weights shouldn't change anything. Why isn't this the case?- The authors do not address the "common input" problem, according to which connectivity inference is limited from the fact that only subsets of the whole network can be observed at any given point in time. It seems to me that the methods proposed here could be connected with the approach presented in [24] (Keshri et al. 2013) to address this issue. It would be nice to see a discussion about this point, since a potential application of the GAMP algorithm to the case of unobserved units could bring even more substantial computational gains, and address the major limitation of [24].QualityThe method presented in the paper is technically sound and is based of the GAMP algorithm that has been very successful in the efficient statistical estimation of sparse/structured signals. ClarityThe paper is at times dense but mostly clearly written, with a lot of useful information presented in the appendix. There are several typos (see below).Originality & SignificanceThe approach of the authors is novel and very likely to be of use in future work since it establishes an efficient for approaching this problem. Since the dimensionality of the connectivity problem can be extremely high, computationally efficient methods will be of greater importance. Even though all the tools that are presented here have been presented before, the application is original since the authors consider a LIF based setup that allows them to apply the GAMP algorithm in a non-trivial way.Typos:line 21: "methods" possibly missing after "Bayesian inference"line 107: The second instance of \tilde{v}_i^k should be without the \tilde?line 107: The bias term b_{IF,i} should probably be moved outside the definition of q_i^k since q is subsequently expressed as q = Ws.line 109: eq (2) v_i^k < \mu --->  \tilde{v}_i^k < \muline 148: \tau_{CA} and \tau{y} in eq. (4) is the same noise variance?line 183: v^k_{T-1} should be changed to v^k_{N}line 312: add "to" before a global maximaline 404: smaller than the… ??


Optimal prior-dependent neural population codes under shared input noise
The brain uses population codes to form distributed, noise-tolerant representations of sensory and motor variables. Recent work has examined the theoretical optimality of such codes in order to gain insight into the principles governing population codes found in the brain. However, the majority of the population coding literature considers either conditionally independent neurons or neurons with noise governed by a stimulus-independent covariance matrix. Here we analyze population coding under a simple alternative model in which latent "input noise" corrupts the stimulus before it is encoded by the population. This provides a convenient and tractable description for irreducible uncertainty that cannot be overcome by adding neurons, and induces stimulus-dependent correlations that mimic certain aspects of the correlations observed in real populations. We examine prior-dependent, Bayesian optimal coding in such populations using exact analyses of cases in which the posterior is approximately Gaussian. These analyses extend previous results on independent Poisson population codes and yield an analytic expression for squared loss and a tight upper bound for mutual information. We show that, for homogeneous populations that tile the input domain, optimal tuning curve width depends on the prior, the loss function, the resource constraint, and the amount of input noise. This framework provides a practical testbed for examining issues of optimality, noise, correlation, and coding fidelity in realistic neural populations.

Sensory systems in the brain are faced with the challenge of representing noisy signals with finite resources. Since these questions were first formalised and discussed over 50 years ago, many have pursued the optimal way of allocating resources to this task, and asked whether the brain's allocation of resources reflects this challenge. This paper attempts to further the body of literature devoted to this question. In particular, it attempts three goals: (1) to replace previous assessments of optimal coding that were based on numerical simulations with analytical results; (2) to study the implications of several components of these models (stimulus priors, the choice of objective function, the integration time, and the input noise) for optimal coding; and (3) to make predictions about how real neurons should behave under the kinds of noise considered here.Quality: Under the (mostly reasonable) assumptions made by the authors, I believe they make some headway on the first goal, namely to analytically derive some of the previous numerical results on optimal coding. There is no discussion on how robust these results are to deviations from these assumptions. The parameter dependence is a bit of a mixed bag -- while the results on adding noise lead to generally intuitive conclusions (e.g. under noisy conditions you can't make the tuning widths arbitrarily small), the time window and objective function results I believe are already mostly known (to some extent covered across Bethge et al 2002, Brunel & Nadal 1998, Wang et al 2012, and others perhaps, eg Berens 2011 for neurometrics). The part that worries me the most though is that the third goal of this paper (presented in Section 4), seems completely tangential to the remaining work, is poorly explained, and lead to predictions that are simply not borne out by the data. The authors are able to cherry-pick two data examples which only roughly follow the predicted patterns; even in this case the predictions are a complete order of magnitude out of proportion. While the authors admit this is a weak correspondence (Line 357: "[the results] are more an exception than a rule"), the authors don't evaluate the implications of this objectively (rather, they cite an unpublished conference talk [16], as a rather weak defence). The paper would be better if Section 4 were omitted, and the extra space devoted to better explanations of the derivations and intuitions.Some other comments:- The tiling property in eqn (3) is *only* an approximation, and does not hold exactly (as asserted) with finite numbers of neurons. What is the implication of the discrepancy? Does it make the decoder biased? Also this means that "the relevant stimulus range" should be defined as being well away from $\star{s}_1$ and $\star{s}_N$- Equation (11). A reference is necessary for this equation: it took me some digging. Also some comment is necessary about the fact that this is a complex-argument extension of the incomplete gamma function, since the second argument is negative, and the $-\lambda$ factor is raised to a fractional power (which is thus complex-valued). Things always have the potential to get very messy in the complex domain (which root of $-\lambda$? do we cross a branch point?), and I can't verify that something funky hasn't happened here.- No information was provided on how to obtain the bound in equation (12). I can't verify this.- Also, $\sigma_{tc}$ as a symbol is a stumbling block, and would be much cleaner as $\sigma_t$.There were several typographical errors which can be corrected.- eqn (4): $\exp^{-\lambda}$, not $\exp^{-R}$- Line 180: missing a factor of $2 \pi e$ inside the logarithm.- eqn (11): exponent on $-\lambda$ should be negative. Same for equation 22.- Fig 5 caption: s* = -5? not -2? Clarity: While the abstract and introduction are clear, I found that the mathematics were often poorly explained. In particular, many of the results were reiterating previous findings of others (e.g. Bethge et al 2002, Berens et al 2011) that had been obtained through other analyses, but these weren't cited throughout. For example, Fig 2B seems to parallel Berens 2011 Figs 2A-B (though why is the FI different as $\sigma_{tc}$ goes to zero?). The comments on pp 185-186 are well known (and indicated as such in the intro), and this should again be stated here. I really could not follow the logic in Section 4 particularly well, not for want of trying; it didn't help that crucial information on Lines 370-372 appeared in the discussion rather than here. Also, very importantly -- there really needs to be consolidation at the end of every section/subsection otherwise this appears as an enumeration of random results.- Line 121: what is S? The whole example is very confusing and unexplained.- Line 124: remind that the rate is independent of s.- Line 132: This is very unclear. Rather: "Since $\Prod_i \exp^{-f_i(s)} = \exp^{-\lambda}$ is constant, the likelihood on $s$ in (1) is thus proportional..."- Line 134: the normal would be clearer as $\mathcal{N}( s | 1/R ... )$- Line 146: The Fisher information, given by $FI(s) = E[ ... ] $- Fig 5: the top line and bottom line need to be distinguished within the figure itself. The interpretations of the left and middle panels of the top line are impossible to see.- Cramer -> CramérOriginality: This paper does extend the previous work on this question with a new treatment. As mentioned above, it often wasn't clear what was a new contribution and what was a re-derivation or re-presentation. For those results which had previously been numerically derived, there is little comment on whether these lead to the same conclusions as the previous results obtained analytically (especially since the authors emphasise that this is a key contribution of the paper).Significance: While the analytic treatment provides an extra degree of rigour above the previous numerical results, I am not convinced that this contribution alone is substantial. The extension to the input noise provide some results that may of use to some. However, the overall value of this paper is tainted by the lack of clear explanations or useful intuitions going along. Previous published work on this material is certainly clearer. Given that these optimality arguments have generally failed to make accurate predictions about coding in the brain in the past (with a few exceptions), and the predictions here are rather shoddy, this work convinces me more that this is a failed project than one worth pursuing. I doubt that is the authors' intention.


Searching for Higgs Boson Decay Modes with Deep Learning
Particle colliders enable us to probe the fundamental nature of matter by observing exotic particles produced by high-energy collisions. Because the experimental measurements from these collisions are necessarily incomplete and imprecise, machine learning algorithms play a major role in the analysis of experimental data. The high-energy physics community typically relies on standardized machine learning software packages for this analysis, and devotes substantial effort towards improving statistical power by hand-crafting high-level features derived from the raw collider measurements. In this paper, we train artificial neural networks to detect the decay of the Higgs boson to tau leptons on a dataset of 82 million simulated collision events. We demonstrate that deep neural network architectures are particularly well-suited for this task with the ability to automatically discover high-level features from the data and increase discovery significance.

The paper describes an application of deep neural networks to offline high energy physics particle collider event detection. The data used for training and testing were generated by simulating collisions. The paper describes experiments using a small number of features that correspond to the actual detector measurements as well as engineered features previously proposed for this task.The paper describes a reasonable machine learning approach to an important problem.As an application paper, in order to add substantial value to the conference, the paper needs to provide a practically important improvement over previous approaches to the problem. Furthermore, the choices made when applying the model to the problem need to be clearly explained, justified, and ideally supported empirically. How are the discovery significance numbers in table 3 calculated and what evidence is there that the improvement over the baseline will be exciting to the physicists who would make use of the classifier? Any citation or argument understandable to people outside HEP that supports the usefulness of the improvement would strengthen the paper dramatically.The caption for figure 2 and section 2.1 need clarification. By my count, there are 6 momenta inputs for the charged leptons and 1 transverse momentum imbalance input. However, it isn't completely clear how the jets are dealt with. Are there 3 more jet momenta inputs that hold the sum of the momenta of all jets? And then one more input holding the number of jets? Presumably the momentum angular information mentioned in the caption for figure 2 is not used. How exactly are the inputs encoded?The first sentence of section 2.2 would be improved by some sort of citation to an example of something in the physics literature that deals with some of these high level features.If at all possible, the exact dataset used should be released publicly.The empirical investigation of different neural net architectures and metaparameter settings feels somewhat preliminary and unfinished, especially since the largest networks got the best results and there is no obvious reason why larger networks couldn't be tried relatively quickly with a couple of the machines used. Were any other momentum schedules tried other than ramping up to 0.99? Or a learning rate annealing schedule based on the training progress? How about rectified linear units? The text mentions that there is no overfitting, but to be maximally clear it should mention somewhere that there is no gap between training set and validation set error or provide a training curve. A training curve with training set and validation set progress as a function of epoch number would help the reader judge whether stopping after 50 epochs was reasonable.How large is the test set? Adding a count scale to the ordinate of the plots in figure 4 would have partially answered this question, but it would probably be better just to say how large the test set is.Figure 5 is not particularly valuable and could be eliminated if more space is needed.In general, in an application paper that uses a deep neural net classifier, there should be a strenuous effort to squeeze the most performance possible out of the neural net, possibly by using Bayesian optimization to set the metaparameters.


Learning with Pseudo-Ensembles
We formalize the notion of a pseudo-ensemble, a (possibly infinite) collection of child models spawned from a parent model by perturbing it according to some noise process. E.g., dropout [9] in a deep neural network trains a pseudo-ensemble of child subnetworks generated by randomly masking nodes in the parent network. We examine the relationship of pseudo-ensembles, which involve perturbation in model-space, to standard ensemble methods and existing notions of robustness, which focus on perturbation in observation-space. We present a novel regularizer based on making the behavior of a pseudo-ensemble robust with respect to the noise process generating it. In the fully-supervised setting, our regularizer matches the performance of dropout. But, unlike dropout, our regularizer naturally extends to the semi-supervised setting, where it produces state-of-the-art results. We provide a case study in which we transform the Recursive Neural Tensor Network of [19] into a pseudo-ensemble, which significantly improves its performance on a real-world sentiment analysis benchmark.

This paper proposes the notion of a pseudo-ensemble, that of dependent child models ensembled together, which is an nice and unified way of several related techniques prominently including dropout. Then presented the pseudo ensemble variance regularizer and tested it with convincing empirical results. The presented technique seems much easier to apply to multi-layer, semi-supervised setting than the technique of [23].Some objections are that the notion of this pseudo-ensemble as an expectation is already in references [5] and [23], so the contribution of naming it and generalizing to multi-layer is not a significant leap. The authors made an effort of gaining more understanding by writing down the boosty and baggy forms, but nothing was really done with them and the connection to boosting is vague. Finally, while it is okay to introduce an regularizer and show that it is empirically successful, it would be nice if it corresponds to, or is approximating something that we understand. The structure of paper would have me believe that this is done, as sections 1-3 talks about a general framework, and section 4 should be testing it. Instead, section 4 introduces a new regularizer that has little to do with the previous sections.  I would have given the paper a higher score based on its empirical strength and the simplicity of the proposed regularizer compared to the alternatives, if not for the fancy presentation that made it harder to read but did not add much insights. Please explain in the response if I am wrong on this and I am willing to change my scores.Detailed comments:the formalism of f(x; \xi) is much more general than the situation the paper seek to deal with: that of subsampling child models through \xi. Without alluding to that \xi represents something like dropout, this formalism does not mean much. Unfortunately, this seems hard to remedy without making the notations more complicated, so perhaps it is okay.Line 106 "Note how the left objective moves the loss L inside the expectation over noise"...this contradicts the equation, where the loss L is outside the expectation over noiseEq 2): baggy PE does not seem to make sense to me, are you summing over i? or is one of the expectations supposed to be taken over i as well?While there is some intuitive resemblance (final prediction is made by the sum of weak learners, though the sum is weighted in boosting), it is unclear to me how these boosty PE concretely connect to boosting. The flavor of sequentially adding more classifiers is not there, nor is the boosting weights assigned to each example. Can you make this more concrete? If not, perhaps it would be good to say this is how you define boosty in the first place and the reader should not be looking for rigorous connections.In line 129-140, you compared a few existing approaches and say that they only deal with the input space. I do not think this is entirely true. My perspective on this:A: [5] Learning with MCFs: non-convex (convex in some cases) lower bound bymoving the expectation inside the log but not inside the exp.B: [23] adaptive regularization: delta method (second-order central limit)expansion (relies on 4th moments small). This has the advantage of giving an explicit regularizer that's interpretable. Non-convex.C: Wang and Manning, Fast Dropout: relies on central limittheorem and sum of noise converging to Gaussian. Non-convex and deals with model noise as well as input noise.D: Baldi and Sadowski, Understanding dropout paper, analyzes geometric mean, also deals with model noise. Eq 3): why is there a subscript on the variance notation? it is the same operation regardless of i (correction: not so in line 210: is it worth the subscript though). You should perhaps call \mathcal{V} scale-invariant variance penalty, so the reader does not think you are talking about the actual variance. The font helps, but you still said its variance. What is the motivation of this regularization method? can you show that this converges to [23] for linear models somehow? or can you provide more understanding by somehow deriving/approximate it instead of just defining it?


Quantized Kernel Learning for Feature Matching
Matching local visual features is a crucial problem in computer vision and its accuracy greatly depends on the choice of similarity measure. As it is generally very difficult to design by hand a similarity or a kernel perfectly adapted to the data of interest, learning it automatically with as few assumptions as possible is preferable. However, available techniques for kernel learning suffer from several limitations, such as restrictive parametrization or scalability. In this paper, we introduce a simple and flexible family of non-linear kernels which we refer to as Quantized Kernels (QK). QKs are arbitrary kernels in the index space of a data quantizer, i.e., piecewise constant similarities in the original feature space. Quantization allows to compress features and keep the learning tractable. As a result, we obtain state-of-the-art matching performance on a standard benchmark dataset with just a few bits to represent each feature dimension. QKs also have explicit non-linear, low-dimensional feature mappings that grant access to Euclidean geometry for uncompressed features.

The paper provides a new algorithm for learning quantized kernels: kernels based on piecewise constant mappings of the features into a finite index space. The main motivation and application is identifying similar pairs in vision tasks. The method achieves significant improvement over state-of-the-art on a task of identifying patch pairs between images.Pros:1. The experimental results, while confined to a specific task, are very strong. The method, while slightly slower, improves ginificantly over state-of-the-art in several metrics and experimental conditions (e.g. explicit feature maps, compact codes).2. The experimental analysis of the learned kernels is deep and insightful to the nature of the specific problem and the way quantized kernels help approach it.3. The approach is novel. Quantized and compact descriptors have been hugely successful in vision applications. This paper takes these ideas into the realm of kernels and offers an interesting way to adaptively learn a quantized kernel.Cons:1. The paper is hard to follow, especially the technical details in section 3. Some examples:a. The definition of a quantizer is cumbersome, instead of just a function q:R^D -> {1,...,N}.b. The optimization algorithm is presented for the one-dimensional case with 2 labels. That is in itself a very easy problem to solve, with no need for the more complicated machinery as brought in the paper, which made it harder to follow for me. Later, there is mention of how to (very approximately) reduce the multi-dimensional, multi-label case to this simple case. This whole line of thought should be made more explicit and easier to follow.c. Last line of page 4: isn't b_1 the same for all t? If so, which t to choose? d. Should I conclude from lines 246-247 that the AQK kernel is actually a diagonal kernel?e. The logic of the subsection about sharing kernel matrices is particularly hard to follow, at least for someone not very familiar with these methods.I think the paper could benefit from an "algorithm" box stating explicitly how to solve the full dimensional, arbitrary N case, for both AQK and BQK. Misc.:1. The problem attempts to classify pairs in negative and positive pairs. Obviously not all pairs can be used for large datasets, especially since most will be negative. How are the pairs chosen in the experiments?2. In other similarity learning tasks it is common to learn a bilinear form such that a triplet order is preserved instead of positive and negative pairs. For example in "Learning a Distance Metric from a Network" by Shaw et al., NIPS 2011. Has this been tried in the context of the tasks used in this paper?3. Line 315 and table 1: it seems that for Adaptive+ the Optimized version does NOT improve performance significantly. It is on the same order as in the Adaptive case, where the authors claim there is no improvement.


On Iterative Hard Thresholding Methods for High-dimensional M-Estimation
The use of M-estimators in generalized linear regression models in high dimensional settings requires risk minimization with hard L 0 constraints. Of the known methods, the class of projected gradient descent (also known as iterative hard thresholding (IHT)) methods is known to offer the fastest and most scalable solutions. However, the current state-of-the-art is only able to analyze these methods in extremely restrictive settings which do not hold in high dimensional statistical models. In this work we bridge this gap by providing the first analysis for IHT-style methods in the high dimensional statistical setting. Our bounds are tight and match known minimax lower bounds. Our results rely on a general analysis framework that enables us to analyze several popular hard thresholding style algorithms (such as HTP, CoSaMP, SP) in the high dimensional regression setting. Finally, we extend our analysis to the problem of low-rank matrix recovery.

This paper presents a theoretical framework for analyzing hard thresh-holding methods for high dimensional inference. These methods take the general recipe of (in one interpretation) (1) taking a gradient step and (2) projecting this result onto a space such that the result is sparse (typically, the s largest entries of the result). The framework proceeds in a similar fashion as Negahban et al (2012), with smoothness and restricted strong convexity assumptions placed on the objective. The authors first present some general results, and next applications of the results to various relevant problem areas. A small section on related methods to projected gradient (which add an additional 'corrective' step of maximizing the objective on a restricted set). The paper concludes with some simulation studies comparing some thresholding methods to greedy and L_1 methods, which are popular sparsity approaches.On quality:Overall, extremely good. The results here are very interesting, and very general. The methods covered by these results are practical, and the applications covered argue strongly for the relevance of the general framework. I can only make a small criticism that the sub-section on GLMs was left as "should be possible"; my intuition is that this should not be a serious challenge, but might also not enough on its own for another paper.On clarity:This paper is quite dense, with a large number of both results and pages of supplemental material. The results are substantial and insightful, but there is a lack of text spent on exposing their meaning, or comparing these results to those obtained for other popular approaches (basically I'm thinking of regularized approaches). There is a little of this sort of discussion in the introduction. As far as the framework is considered, the authors did well giving insight in the proof sketch of Theorem 1. However, some key details are left to the appendix for Theorem 3. In particular, they claim that the loss need not be convex is not expanded upon. The application to noisy data is a key use case of this property, but that part of the proof is given an implication of another proof in the appendix: is there an insight to be gained from it this that could be stated there? There could be a reference that would give insight into where we have restricted convexity but not convexity.The experiments mostly showed improvements on run-time versus other common classes of approaches. This sort of analysis seems a little out of place given the area of study of the rest of the paper. While many of the results were given in terms of a iteration of a certain order of the algorithm, these ideas can't exactly be lined up with ideas from regularized methods (I admit to not looking up the FoBa results, which might have a similar quality). I'm not really sure that anything better could be done here anyway.On originality and significance:Overall, the paper offers highly original and highly significant results. Some parts of the main approach of the framework here are not entirely original. I do not see this as a major detraction from the ideas: I find it sensible that the notions of smoothness and restricted convexity are fundamental here as well.


Probabilistic Differential Dynamic Programming
We present a data-driven, probabilistic trajectory optimization framework for systems with unknown dynamics, called Probabilistic Differential Dynamic Programming (PDDP). PDDP takes into account uncertainty explicitly for dynamics models using Gaussian processes (GPs). Based on the second-order local approximation of the value function, PDDP performs Dynamic Programming around a nominal trajectory in Gaussian belief spaces. Different from typical gradientbased policy search methods, PDDP does not require a policy parameterization and learns a locally optimal, time-varying control policy. We demonstrate the effectiveness and efficiency of the proposed algorithm using two nontrivial tasks. Compared with the classical DDP and a state-of-the-art GP-based policy search method, PDDP offers a superior combination of data-efficiency, learning speed, and applicability.

This paper proposes a DDP formulation for models represented as Gaussian processes. The derivations of the required equations that connects the Gaussian Process model with the DDP framework are provided.The method allows the solution of local optimal trajectory problems for systems whose dynamics are not known in advance.As such this work shares several characteristics with PILCO. The simulated results show that solutions are found much faster than PILCO. Quality and clarityAlthough the concept of the paper is strong, the paper needs more work to be done, particularly regarding the presentation of the results.Presentation of the results is currently too shallow, especially the discussion, which is basically non-existent for figures 1 and 2.Claims such as "safe exploration" lacks support and the computational complexity conclusion is questionable.The writing could be refined in general; I am not referring to specific sentences, but it feels this work is lacking some additional number of polishing iterations on the writing.Originality and SignificanceThe fact that DDP is being proposed as the optimization method has an impact for the DDP/iLQG practitioners and also for the model-based optimization community. Unarguably the presented work shares several similarities with PILCO both in the philosophy and also the methodology. However, while the work of PILCO basically kept the choices of the optimizer open, this work suggest DDP and develops the full framework thus providing a self-contained solution.This work can be much improved if extensive and careful analyses between PDDP and PILCO are provided.Questions . Line 33: "... Compared to global optimal control approaches, the local optimal DDP shows superior scalability to high-dimensional problems..." is an incomplete statement. The big trade-off is that DDP solution is only local and prone to provide a poor a local optima. I agree that for high-dimensional problems a local solution is usually the only option. If there was no trade-off there would be no reason for the full dynamic programming.. Line 212: The sentence " ... where the variance of control distribution and the off-diagonal entries of the state covariance matrix can be neglected because of no impact on control performances." is not obvious especially regarding the control distribution. Noise in control can have a large impact on how the states integrate over time. It may also influence the final cost. Are you assuming a deterministic system?. Line: 294. The conclusion on the computational complexity is precipitated. Yes, although the complexity of the learning policy does not relate directly to the number of states, for the usual case of fully actuated systems, the number of control inputs is proportinal to the number of states which makes the complexity of the algorithm O3.. The formulation of the "safe exploration" feature is not clear in the text. Although Figure 3(a) is a cartoon that shows the intuition, where is this intuition implemented in the algorithm? . Line 322: "To keep the system stable" seems to be a bad sentence, unless you are talking about unstable system such as an inverted pendulum. Do you mean "final zero velocity"?. The variance in (a) seems simply to small, almost a deterministic system. As the GP is learning the system from samples one can infer that the variance during the initial iterations are large and decrease with the number iterations. It would be interesting to see such a plot.. I do not see the point of proposing another method (GPDDP) that  does not bring any benefit. What is the motivation for that? If what you want to show is the effect of the propagation of uncertainty then you can just say that without the need of introducing the GPDDP as a new second method.. The paper must also compare the traditional DDP results when dynamics are fully known  in Figs 1 and 2.Minor questions:. Legends for Figure 1 are too small. The y axes of (a) and (c) should be on the same range.. Why not showing the costs for PILCO on figures 1(b) and 2(b)?


Learning Chordal Markov Networks by Dynamic Programming
We present an algorithm for finding a chordal Markov network that maximizes any given decomposable scoring function. The algorithm is based on a recursive characterization of clique trees, and it runs in O(4 n ) time for n vertices. On an eight-vertex benchmark instance, our implementation turns out to be about ten million times faster than a recently proposed, constraint satisfaction based algorithm (Corander et al., NIPS 2013). Within a few hours, it is able to solve instances up to 18 vertices, and beyond if we restrict the maximum clique size. We also study the performance of a recent integer linear programming algorithm (Bartlett and Cussens, UAI 2013). Our results suggest that, unless we bound the clique sizes, currently only the dynamic programming algorithm is guaranteed to solve instances with around 15 or more vertices.

This work develops a new exact algorithm for structure learning of chordal Markov networks (MN) under decomposable score functions. The algorithm implements a dynamic programming approach by introducing recursive partition tree structures, which are junction tree equivalent structures that well suit the decomposition of the problem into smaller instances so to enable dynamic programming. The authors review the literature, prove the correctness of their algorithm and compare it against a modified version of GOBNILP, which is implements an state-of-the-art method for Bayesian network exact structure learning.The paper is well-written, relevant for NIPS and technically sound. As far as I can tell, it is also novel and presents an important contribution to the area. The only main issue I find is the lack of a proper motivation for learning _chordal_ Markov networks. I assume the restriction to chordal networks is to allow for efficient exact inference and smaller sample complexity. The author could enlighten us with a discussion on that. Below are some minor issues:page 2, lines 67: There are recent publications on learning Bayesian networks of bounded treewidth which should be added as related work:P. Parkaviainen et al. Learning Bounded Tree-width Bayesian Networks using Integer Linear Programming. In AISTATS 2014.J. Berg et al. Learning Optimal Bounded Treewidth Bayesian Networks via Maximum Satisfiability. In AISTATS 2014.In particular, the first publication shows an integer programming formulation of the problem which could be adapted to learn chordal MNs, similarly to the way the authors adapted the work of Barlett and Cussens (probably even easier. As the implementation of the code is freely available, the authors are suggested to compare against it.)There is a similarity in learning maximum likelihood BNs of bounded treewidth to learning chordal maximum likelihood MNs of bounded treewidth. If that is the case, then algorithms for the former could be tested against the proposed method as well. It would also provide a better connection between these two tasks.page 3, lines 108-110: By assuming that the scores are given as input, you implicitly assume that w (the width of the network decomposition) is small. Hence, I see little value later on in the experiments section when w is taken to infinity; the way I see it, the current approach is designed for handling cases of "small" w (perhaps up to 30 or so). A warning note here would be helpful.same page, definition 1: it is better to give names to the conditions (1), (2) and (3), otherwise it gets confusing when following the proofs, which also contains enumerations of their own (something like RPT1, RPT2, RTP3 would do).page 5, line 245: the enumeration (0), (1) and (3) slightly confuses the reader (which expects to see a (2) somewhere). Try renumbering the conditions.page 7, line 370: better or additional references for score pruning rules areC.P. de Campos, Q. Ji: Properties of Bayesian Dirichlet Scores to Learn Bayesian Network Structures. In AAAI 2010C.P. de Campos, Q. Ji: Efficient Structure Learning of Bayesian Networks using Constraints. Journal of Machine Learning Research 12: 663-689 (2011)pages 7-8, Experiments sec.: drawing CPTs uniformly at random seems unrealistic. It would be better to draw distributions from a symmetric Dirichlet with hyper-parameter < 1 so that it gets relatively high entropy and mimics real-world distributions. As the performance of GOBNILP is strongly affected by the parameters, using unrealistic models might bias the comparison.page 9: write full name of JMLR in Ref. 13.