Approximating Hierarchical MV-sets for Hierarchical Clustering
The goal of hierarchical clustering is to construct a cluster tree, which can be viewed as the modal structure of a density. For this purpose, we use a convex optimization program that can efficiently estimate a family of hierarchical dense sets in high-dimensional distributions. We further extend existing graph-based methods to approximate the cluster tree of a distribution. By avoiding direct density estimation, our method is able to handle high-dimensional data more efficiently than existing density-based approaches. We present empirical results that demonstrate the superiority of our method over existing ones.

The paper introduces a new method for estimating cluster trees by direct estimation of the MV-sets rather than separate density estimation and recursive estimation of the tree using level sets. This avoids the issues inherent in producing stable high-dimensional density estimates. The method uses a graph cutting technique (with the full dataset being the graph nodes) to produce a set of high-density clusters.Essentially the q-OCSVM method is used to create a different method for specifying the graph edge weights between observations. After this point the algorithm largely follows the cluster tree estimation method set out by Stuetzle and Nugent. The issue of bandwidth selection is an important one and the authors mention it only briefly. Their suggestion of specifying the bandwidth which gives the number of modes that the user would expect does allow for expert knowledge to be incorporated into the procedure which is an advantage (and could be emphasised more in the paper) but only when such knowledge is available. They suggest no alternative for cases where a more objective solution might be required.The simulations are acceptable and the methods chosen for comparison are reasonable. Quality: The motivation for the new method is clear and the analysis/results sections are reasonable. It is an interesting approach that could be of use.Clarity: The paper is very well written, with a good summary of the background of all elements of the approach.Originality: This method is the application of an already developed method for MV estimation to an existing graph-based approach to building cluster trees. The combination is novel but the different elements are not.Significance: The results seem to suggest that the new graph edge weight metric has merit but there are outstanding issues with the choice of bandwidth that I suspect will limit the impact of this work on practitioners.Small details:If you have space I would suggest emphasising the advantage of this method over existing density-estimation based methods in high-dimensional datasets in the abstract.The authors give a very nice summary of the concept of a (non-parametric) cluster tree and level sets. Figure 1 level set lines should really have (light) vertical lines dropping to x-axis at points of intersection with the density to illustrate the level sets properly. Line 082: The phrase "feature vectors" (and "features") seems a little confusing here and elsewhere. Perhaps "observation vectors" would be better. Since "features" often refers to variables.Line 232, expected is used too many times. Perhaps change the second to the "number specified/recommended by the user".The description of the quality measure in section 4.1 needs more detail. Better clarifying what it meant by cluster and class here would improve the readability of this part. Are clusters defined just for a single level or over all levels? Similarly are classes defined over a tree or a single level? Presumably clusters refers to the partitions resulting from the tree estimated by the method in the paper?There is an issue in formatting with the caption of Figure 3 overlapping the line on the bottom of page 6.There needs to be a line to space things out between the bottom of the captions for Figures 4 and 5 and the main text.Line 370 "1, 092" should be "1,092"


Deep Joint Task Learning for Generic Object Extraction
This paper investigates how to extract objects-of-interest without relying on handcraft features and sliding windows approaches, that aims to jointly solve two subtasks: (i) rapidly localizing salient objects from images, and (ii) accurately segmenting the objects based on the localizations. We present a general joint task learning framework, in which each task (either object localization or object segmentation) is tackled via a multi-layer convolutional neural network, and the two networks work collaboratively to boost performance. In particular, we propose to incorporate latent variables bridging the two networks in a joint optimization manner. The first network directly predicts the positions and scales of salient objects from raw images, and the latent variables adjust the object localizations to feed the second network that produces pixelwise object masks. An EM-type method is presented for the optimization, iterating with two steps: (i) by using the two networks, it estimates the latent variables by employing an MCMC-based sampling method; (ii) it optimizes the parameters of the two networks unitedly via back propagation, with the fixed latent variables. Extensive experiments suggest that our framework significantly outperforms other state-of-the-art approaches in both accuracy and efficiency (e.g. 1000 times faster than competing approaches).

Summary:This paper proposes a model that jointly trains two networks. One network triesto do localization. The second network does segmentation.  The first networkoutputs the coordinates of a bounding box. The image is cropped using thesecoordinates and resized to a fixed square size. The resulting image is given as inputto the second network. The second network predicts a pixel-wise binary mask tosegment out the foreground. The model is supervised i.e., pixel-wise segmentationlabels are given.  The authors argue that if the two tasks are decoupled and atight bounding box around the object is used as the target for the firstnetwork, then the predicted bounding boxes may be too tight and not provideenough context to do segmentation. They may also cut across some region of theforeground.To prevent this, the authors introduce latent variables which are perturbationsin the coordinates of the bounding box. The localization net now has to predictthe perturbed bounding box. However the optimal perturbation is not known.Therefore the model optimizes for the perturbation using a Metropolis-Hastings sampling strategy that uses super-pixels to figure out the goodness of any perturbation.Strengths- Shows improvements over decoupling the tasks and using no perturbation.- The model is very efficient.- The performance is compared to many other models. These models include ones based on hand-crafted features alone as well as ones that involve some form of learning. The model is convincingly better than these other models.Weaknesses- A simple baseline would be to just add a 5 or 10 pixel boundary around the predictions of the localization net. The authors do not compare to this baseline.- A slightly stronger baseline would be to modify the predicted bounding boxes using super-pixels as a post-processing step (maybe using the same idea that is used in the data-dependent term of the MH proposal distribution). This baseline is not used either.- In the absence of strong baselines, it is not clear what the joint optimization buys us.- The proposal distribution's data-dependent term uses super-pixel cues in a very ad-hoc way. The term is designed to avoid cutting through super-pixel regions. However, unless the super-pixel regions are aligned with the axes of the bounding box rectangle, it seems that it would be hard to avoid cutting through super-pixels. Moreover, there may be a lot of background region going across the edges of the bounding box. It is perfectly reasonable for the bounding box to cut through super-pixels in such regions.Quality:The paper does not compare to simpler baseline models (as mentioned in the weaknesses section) which makes it hard to assess it contribution.Clarity:There seem to be many typos and grammatical errors, some of which are mentioned here.Minor typos -- Eq(1) The equality should probably be a proportionality.- Eq(2) There seems to a square missing over the L2 norm.- '... may lead to unsatisfied pixel-wise segmentation ..' unsatisfied -> unsatisfactory ?Originality:The idea of using Metropolis-Hastings to combine two tasks seems interesting. However, this particular application is not very compelling.Significance:The main idea is interesting but more experiments with stronger baselines are needed to assess the potential impact of this model.


Exact Post Model Selection Inference for Marginal Screening
We develop a framework for post model selection inference, via marginal screening, in linear regression. At the core of this framework is a result that characterizes the exact distribution of linear functions of the response y, conditional on the model being selected ("condition on selection" framework). This allows us to construct valid confidence intervals and hypothesis tests for regression coefficients that account for the selection procedure. In contrast to recent work in high-dimensional statistics, our results are exact (non-asymptotic) and require no eigenvalue-like assumptions on the design matrix X. Furthermore, the computational cost of marginal regression, constructing confidence intervals and hypothesis testing is negligible compared to the cost of linear regression, thus making our methods particularly suitable for extremely large datasets. Although we focus on marginal screening to illustrate the applicability of the condition on selection framework, this framework is much more broadly applicable. We show how to apply the proposed framework to several other selection procedures including orthogonal matching pursuit and marginal screening+Lasso.

The purpose of this paper is to derive exact confidence intervals for linear regression coefficients post model selection. The authors first introduce the notation of the regression model under study and the marginal screening as one simple algorithm for variable selection. They illustrate empirically that the commonly used z-test interval is not correctly calibrated and thus not valid. The authors then construct the distribution of the target variable conditioned on the selection event, which leads to confidence intervals for, e.g., the marginal screening. Finally, the confidence intervals are studied empirically on two real-world datasets.* Quality: Overall the paper is technically sound. However, it fails to evaluate all strength and weaknesses of the proposed method. Specifically:- In hypothesis testing (confidence intervals), there is always the trade-off between type I and II error. The type I error is, unfortunately, not well-studied. This could be done, e.g., with simulated data where you have control over the ground truths (correlation between features and target label).- You state that the POSI [2] method is computational expensive and thus not applicable for d larger than 30. But that means that you can use POSI in Section 7.1 or study its behaviour on artificial data. - There are missing baselines: What is the performance (power and correctness) compared to other baselines as greedy algorithms, POSI,... mentioned in Section 2?- The depending on the number of dimensions and the coverage rate depending on n would also be worth to be studied. * Clarity: Overall, the paper is comprehensibly structured, but the context is often rarely described, which makes it hard to follow. Furthermore, some key concepts and notation are ambiguous or not introduced. Specifically,- The model is introduced immediately in the first sentence but the notation is actually formally defined in Section 3. At the first part some aspects are incompletely presented. E.g., what is the number of examples? What is \mu (in contrast to \mu(x))?- The notation for the observation and the noise free label is confusing: "(X^TX)^{−1} X^T\mu, which is the best linear predictor of \mu" (line 039) makes not really sense to me. Isn't \mu the best estimator for \mu? Do you mean "(X^TX)^{−1} X^T y" is the best predictor of \mu? The same holds for "\mu = X \beta^0".- \mu(x) is assumed to be arbitrary in the beginning but only the linear case is studied. To clarify things, why don't you restrict yourself to linear models at all?- While the need for variable selection is motivated the need of confidence intervals is not. Can you clarify this in more detail?- 7.1 studies the dependance between empirical and theoretical coverage. However, Section 7.2 present only one outcome of your method on the genomic dataset. What is the significance of that statement? Is that already known and does it confirm the correctness of your approach or is that a new outcome and gives new insight to the community?- "Z-test", "Adjusted",  and "Nominal" in Figure 3 not introduced* Originality: The paper applies the idea of [12] to derive the exact distribution of the target variables by conditioning on the selection event to marginal screening.* Significance: The empirical study fails to evaluate all strength and weaknesses and the theoretical results are quite similar to [12].


Weighted importance sampling for off-policy learning with linear function approximation
Importance sampling is an essential component of off-policy model-free reinforcement learning algorithms. However, its most effective variant, weighted importance sampling, does not carry over easily to function approximation and, because of this, it is not utilized in existing off-policy learning algorithms. In this paper, we take two steps toward bridging this gap. First, we show that weighted importance sampling can be viewed as a special case of weighting the error of individual training samples, and that this weighting has theoretical and empirical benefits similar to those of weighted importance sampling. Second, we show that these benefits extend to a new weighted-importance-sampling version of offpolicy LSTD( ). We show empirically that our new WIS-LSTD( ) algorithm can result in much more rapid and reliable convergence than conventional off-policy LSTD( ) (Yu 2010, Bertsekas & Yu 2009).
1 Importance sampling and weighted importance sampling Importance sampling (Kahn & Marshall 1953, Rubinstein 1981, Koller & Friedman 2009) is a wellknown Monte Carlo technique for estimating an expectation under one distribution given samples from a different distribution. Consider that data samples Y k 2 R are generated i.i.d. from a sample distribution l, but we are interested in estimating the expected value of these samples, v
, under a different distribution g. In importance sampling this is achieved simply by averaging the samples weighted by the ratio of their likelihoods ⇢
, called the importance-sampling ratio. That is, v g is estimated as:ṽ
This is an unbiased estimate because each of the samples it averages is unbiased:
Unfortunately, this importance sampling estimate is often of unnecessarily high variance. To see how this can happen, consider a case in which the samples Y k are all nearly the same (under both distributions) but the importance-sampling ratios ⇢ k vary greatly from sample to sample. This should be an easy case because the samples are so similar for the two distributions, but importance sampling will average the ⇢ k Y k , which will be of high variance, and thus its estimates will also be of high variance. In fact, without further bounds on the importance-sampling ratios,ṽ g may have infinite variance (Andradóttir et al. 1995, Robert & Casella 2004).
An important variation on importance sampling that often has much lower variance is weighted importance sampling (Rubinstein 1981, Koller & Friedman 2009). The weighted importance sampling (WIS) estimate v g as a weighted average of the samples with importance-sampling ratios as weights:
This estimate is biased, but consistent (asymptotically correct) and typically of much lower variance than the ordinary importance-sampling (OIS) estimate, as acknowledged by many authors (Hesterberg 1988, Casella & Robert 1998, Precup, Sutton & Singh 2000, Shelton 2001, Liu 2001, Koller & Friedman 2009). For example, in the problematic case sketched above (near constant Y k , widely varying ⇢ k ) the variance of the WIS estimate will be related to the variance of Y k . Note also that when the samples are bounded, the WIS estimate has bounded variance, because the estimate itself is bounded by the highest absolute value of Y k , no matter how large the ratios ⇢ k are (Precup, Sutton & Dasgupta 2001).
Although WIS is the more successful importance sampling technique, it has not yet been extended to parametric function approximation. This is problematic for applications to off-policy reinforcement learning, in which function approximation is viewed as essential for large-scale applications to sequential decision problems with large state and action spaces. Here an important subproblem is the approximation of the value function-the expected sum of future discounted rewards as a function of state-for a designated target policy that may differ from that used to select actions. The existing methods for off-policy value-function approximation either use OIS (Maei & Sutton 2010, Yu 2010, Sutton et al. 2014, Geist & Scherrer 2014, Dann et al. 2014 or use WIS but are limited to the tabular or non-parametric case (Precup et al. 2000, Shelton 2001). How to extend WIS to parametric function approximation is important, but far from clear (as noted by Precup et al. 2001).

Summary=============The paper brings the idea of weighted importance sampling (WIS) to off-policy policy evaluation. The core of the work is built around the observation that WIS can be formulated as a weighted regression problem and the corresponding formulation is extended to the problem of linear approximation in off-policy policy evaluation. The soundness of the method is discussed in detail and its empirical effectiveness is evaluated in a simple chain problem.Quality==============The way the paper is written may give the impression that the application of WIS is very simple and almost obvious. Nonetheless, as remarked by the authors, bringing WIS to off-policy evaluation was not trivial and it required a deep understanding of the structure of WIS.The main contributions of the paper are:1- I found the formulation of OIS and WIS in eq. 2 and 3 and their extension to linear regression very simple but elegant and very insightful. Eqs. 5 and 6 clearly reveal how the idea of OIS and WIS estimators smoothly extend to the case of linear regression. This is further supported by the analysis in Section 3. There are only two aspects that I would like the authors to clarify further:a- The definition of the OIS and WIS linear estimators only correct for the unbalance in the output distribution, while the possible difference in the input distributions is not taken into account (starting from the definition of J^{MSE}). Why? How would you correct differences in the input distribution?b- In the analysis in Section 3 no discussion is provided about whether WIS-LS may actually benefit from similar variance-reduction properties as for WIS. It would be helpful to report in sections 1/2 a more clear overview of the theoretical results available for WIS and discuss in section 3 how much these results may extend to the linear regression case.2- The paper then proceeds in extending the WIS-LS idea to the off-policy evaluation problem, notably reformulating the LSTD(lambda) algorithm. Furthermore, a fully online version of the algorithm is provided with constant computational complexity over time. There are two aspects of this part which are not fully clear:a- In Thm.7 it is shown that WIS-LSTD(lambda) reduces to WIS-LS in the Monte-Carlo, finite-horizon case. Given that WIS-LS corrects only for the output distribution (in this case the sum of rewards), I am not sure I fully understand how this is sufficient in the off-policy case, when the state-action distribution (which corresponds to the input distribution) may vary because of the different policies.b- The incremental definition of WIS-LSTD(lambda) still requires the inversion of a matrix of the dimensionality of the number of features (m^3), while usual online implementations of TD only have a linear complexity per step. Do you think such a linear-complexity implementation may be actually possible?3- Finally, the paper reports preliminary empirical results in a simple exact and approximate chain problem. Although the test is very simple, it illustrates the performance of the proposed method along multiple dimensions: number of steps, lambda, and regularizer. Results show a clear advantage of the weight importance sampling solution both in performance and robustness in the parameter selection. The only drawback of this part of the paper is that the problem is really simple and a more extensive empirical evaluation on more challenging problems would definitely benefit to the paper.4- Although the authors reference most of the relevant literature, no discussion and comparison is provided w.r.t. the recent paper:Doubly Robust Policy Evaluation and Learning Authors: John Langford, Lihong Li and Miroslav DudíkConference: Proceedings of the 28th International Conference on Machine Learning (ICML-11)which study a very much related problem. Another very much related paper isHirotaka Hachiya, Takayuki Akiyama, Masashi Sugiyama, Jan Peters: Adaptive importance sampling for value function approximation in off-policy reinforcement learning. Neural Networks 22(10): 1399-1410 (2009)which seems to have an overlap with the current submission.Clarity==============The paper is very well written.Originality==============As reviewed by the authors, the idea of importance sampling has been already applied in off-policy learning in the past. The way the authors bring weighted importance sampling to RL through the clean and elegant formulation of WIS-LS in LSTD is original. Nonetheless, I encourage the authors to have a more thorough discussion with existing literature point out similarities and differences in order to understand better the originality of the contribution. Significance==============Off-policy evaluation is a crucial problem in a huge variety of problems and effective solutions to this problem may have a significant impact.***************************************** AFTER REBUTTALThanks to the authors for their reply. The comparison sketched in the rebuttal is fair and seems convincing. Thus I confirm my positive impression about the paper and I strongly encourage them to revise the paper including the reference and discussion.


Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
We present techniques for speeding up the test-time evaluation of large convolutional networks, designed for object recognition tasks. These models deliver impressive accuracy, but each image evaluation requires millions of floating point operations, making their deployment on smartphones and Internet-scale clusters problematic. The computation is dominated by the convolution operations in the lower layers of the model. We exploit the redundancy present within the convolutional filters to derive approximations that significantly reduce the required computation. Using large state-of-the-art models, we demonstrate speedups of convolutional layers on both CPU and GPU by a factor of 2×, while keeping the accuracy within 1% of the original model.

The paper addresses the problem of speeding up the evaluation of pre-trained image classification ConvNets. To this end, a number of techniques are proposed, which are based on the tensor representation of the conv. layer weight matrix. Namely, the following techniques are considered (Sect. 3.2-3.5):1) SVD decomposition of the tensor;2) outer product decomposition of the tensor;3) monochromatic approximation of the first conv. layer - projecting RGB colors to a 1-D space, followed by clustering;4) biclustering tensor approximation - clustering input and output features to split the tensor into a number of sub-tensors, each of which is then separately approximated;5) fine-tuning of approximate models to (partially) recover the lost accuracy.Also, the authors consider two approximation criteria (Sect. 3.1):1) diagonal Mahalanobis metric, defined by the importance of the individual weights, computed by backprop;2) data covariance metric, which takes into consider the distribution of the layer inputs.The proposed speed-up methods are applied to the first and second convolutional layers of the state-of-the-art ConvNet architecture [8], and they are shown to speed-up the computation of these layers (but not the whole network) by a factor of 2-3, with a little loss of accuracy. SVD decomposition is also applied to the fully-connected layers of the network to reduce the number of parameters.== Positive points ==1. The problem of speeding up the computation of large ConvNets is relevant.2. The proposed methods for speeding-up ConvNets are novel.== Negative points ==The main problem of the paper is that even though a multitude of methods is proposed, their evaluation is superficial, which makes it hard to understand how these methods compare to each other. Also, the effect of methods' parameters is not properly explored. It would be really helpful to have a "take home" message for deep learning practitioners, which provides clear recipes on how to use the proposed methods. For that purpose, the authors might want to add an "accuracy vs speed-up" plot for the whole net (not just two speeded-up layers), with all approximations combined. Also, the description of some of the methods is too concise. A detailed list of issues and questions is given below.== Detailed comments ==1. Sect. 3.1: the proposed distance metrics are not compared to each other (and the Frobenius norm baseline) in the experiments. Fig. 4 only contains the results of the data covariance distance metric, the Mahalanobis metric (and the Frobenius norm) results do not seem to be reported. By the way, what is denoted as "Original" in Fig. 4?2. l.128, (1): which value of h did you use? Why not to compute the derivatives of the correct class score w.r.t. the approximated weights?3. l.144-153: the description is too concise, it should be extended (especially if this is the method used in the experiments).4. l.236, l.201 mention that the biclustering approximation is used for higher convolution layers. However, it's only applied to the first two layers. If the approximation was evaluated on higher levels, these results should be reported; otherwise, these statements should be corrected.5. l.236-242: is it true that each sub-tensor corresponds to a subset of input-output channels, and the subsets do not overlap, i.e. each channel is "covered" by a single sub-tensor? The authors should elaborate on how the result is computed by including a formal equation, instead of just referring to Fig. 1(b).6. l.241-242 ("each sub-tensor contains similar elements"): do you mean similar in the sense of the distances (2) or (3)? Are there any theoretical reasons why each cluster lives on a low-rank manifold, and is thus easier to approximate than the original tensor?7. Sect. 4.1.2: two approximations of Sect. 3.2.2 are not compared to each other without biclustering. It would be helpful to see the benefits of biclustering.8. l.362-365 ("we explored various configurations"): only the results corresponding to 2 combinations of G and H are given in the paper and the supp. material, which is not sufficient. In Fig. 4, why are different values of G, H, K, K1, and K2 used for the CPU and GPU implementations? How are they chosen?9. l.376 ("performance is restored"): there is no guarantee that the performance can be fully restored by fine-tuning. It would be helpful to see the results with and without fine-tuning.10. Sect. 4.3: how does SVD on the fully-connected layers affect the evaluation speed?11. l.269 ("4 convolutional layers"): isn't it supposed to be 5, if the configuration of [8] is used?12. l.285 ("20K validation images"): why not to report the performance on all 50K validation images, as typically used in the literature?13. Speed-up of the whole net: it would be helpful to see the speed-up of the whole net, rather than just the first two layers. According to Table 1 of the Supp. material, it seems that x2 speed-up of the first two conv. layers will result in only 22% reduction of the evaluation time of the whole net on the GPU?14. Relation to [6]: it is understood that this work was developed in parallel to [6] (recently published at BMVC-2014). While the comparison with [6] is not strictly required, it would be interesting to see it in the final version, if this paper gets accepted. Interestingly, [6] report 2-4 times speed-up for the *whole* network, but obviously the ConvNet configurations considered in this submission and [6] are different, so a direct comparison is not possible at the moment, and the authors are encouraged to add one.== Typos ==l. 214: "consider consider"l. 211, l.315 "Figure 4.1.1." -> "Figure 2"


Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning
Stochastic gradient descent algorithms for training linear and kernel predictors are gaining more and more importance, thanks to their scalability. While various methods have been proposed to speed up their convergence, the model selection phase is often ignored. In fact, in theoretical works most of the time assumptions are made, for example, on the prior knowledge of the norm of the optimal solution, while in the practical world validation methods remain the only viable approach. In this paper, we propose a new kernel-based stochastic gradient descent algorithm that performs model selection while training, with no parameters to tune, nor any form of cross-validation. The algorithm builds on recent advancement in online learning theory for unconstrained settings, to estimate over time the right regularization in a data-dependent way. Optimal rates of convergence are proved under standard smoothness assumptions on the target function as well as preliminary empirical results.

Added post rebuttal: After considering the rebuttal and taking another look at the paper, I may have been a bit harsh in my criticism. I still think the paper relies a lot on techniques from previous papers, but the connections between them are indeed novel. I thank the author\s for the clarifications.---------------------------------------------This paper proposes a parameter-free algorithm for stochastic gradient descent of linear and kernel predictors. In particular, the step size is automatically determined and does not rely on the (initially unknown) norm of the competitor, nor on a doubling trick. The algorithm builds on recently developed online convex optimization algorithms with similar behavior, with the contributions being:- Convergence bounds depending on the magnitude of the subgradients rather than the time horizon. In particular, for smooth losses, this leads to bounds which depend on the risk of the competitor, and interpolate between O(T^{-1/2}) and O(T^{-2/3}) rates.- Convergence to the risk of the "Bayes-optimal" function (w.r.t. the loss function used), assuming this function lies in a function space with an appropriate spectral decay.- Convergence bounds under a Tsybakov noise condition.- A few experiments showing the competitiveness of the proposed algorithm and cross-validated SVM.The question of how to avoid cross-validation and parameter tuning is certainly an interesting one and has practical implications. Moreover, the algorithm is simple and easy to implement. On the flip side, there has already been a string of recent works on online learning methods with such behavior, which can be readily applied to stochastic and batch problems (e.g. [14,16,25] which the paper cites). The proposed algorithm also builds on those methods, so the conceptual novelty is perhaps not as strong as the abstract and introduction seem to suggest. As far as I understand, the main improvement over them consists of the refined bound depending on the subgradient magnitudes, which for smooth losses lead to bounds depending on the loss of the competitor. This is an interesting contribution, although the bounds are probably suboptimal (e.g. when the loss is zero, we only get O(T^{-2/3}) rates, but with standard algorithms can get O(T^{-1}). A more minor issue is that it isn't clear if the refined bound and algorithmic change leads to anything practically noticeable (some experiments would be useful in that regard). The results regarding convergence to the Bayes-optimal and Tsybakov noise condition are nice and technically non-trivial, but unless I missed something, appear to consist mostly of plugging the new bound into existing techniqus. Overall, I would say that the contributions are solid but a bit incremental in light of previous work.  In terms of presentation, the paper is mostly well-written, but I felt it was unnecessarily complicated at times. For example:- In section 2, the definitions of the function spaces are very abstract, and would probably be hard to grasp for non-experts. Some specific examples would be very useful. - Theorem 1 could be written in a much simpler way by choosing a specific value for "a", e.g. a=2.25L, and replacing \phi(x) by the resulting constant. I failed to see what additional insight is gained by keeping "a" as free parameter. Technical and Other Comments----------------------------- The setting focuses on classification losses, which are a function of y*prediction. Can similar results be shown for regression or more general losses?- Line 46: "While in finite hypothesis spaces simple optimal strategies are known, in infinite dimensional spaces..." - should "finite" be "finite dimensional"?- Line 82: Since you focus on scalar-input loss functions, perhaps it's best to present things w.r.t. scalars rather than vectors, and use "sub-derivative" instead of "sub-gradient". For example, I was confused at first by algorithm 3, which talks about s_t as subgradients but are in fact used as scalars.- Line 125: "By the Mercer's theorem"- LIne 203: "We will present such algorithm"- Line 315: "that is low optimal \ell-risk" -> "that has low optimal \ell-risk" (?)- Line 408: "run few" -> "run a few"


A Unified Semantic Embedding: Relating Taxonomies and Attributes
We propose a method that learns a discriminative yet semantic space for object categorization, where we also embed auxiliary semantic entities such as supercategories and attributes. Contrary to prior work, which only utilized them as side information, we explicitly embed these semantic entities into the same space where we embed categories, which enables us to represent a category as their linear combination. By exploiting such a unified model for semantics, we enforce each category to be generated as a supercategory + a sparse combination of attributes, with an additional exclusive regularization to learn discriminative composition. The proposed reconstructive regularization guides the discriminative learning process to learn a model with better generalization. This model also generates compact semantic description of each category, which enhances interoperability and enables humans to analyze what has been learned.

The paper presents a multi-task learning based technique for jointly accommodating information conveyed by mid-level attributes and category semantics. A sparse coding approach is utilized to infer discriminative information to identify categories while permitting generative modeling to reason out new categories. The method is evaluated on the Animals with attributes dataset and shows improved classification performance, with an emphasis on one/few-shot learning circumstances.The paper, as mentioned in the manuscript, is a culmination of different existing ideas pertaining to semantic models, discriminative embedding, multitask learning and sparse learning. While the paper does argue how they are different, in my opinion there isn't much contribution in terms of a new approach/algorithm.Moreover, the paper seems to overly use/rely on recent "terminologies" in the name of attributes/semantics, while in crux all its doing is taking different pieces of information about the entities (object categories, here) one is interested in. While one can argue, for instance, attributes are generic while category specific information is not, the approach is more like an hierarchical one where such information can be integrated. In that spirit, existing deep learning methods do the same thing without having to handcode these pieces of information. Also of relevance is the coarse-to-fine strategies that vision community has looked at for many years. Besides dealing with these features, the claim of jointly performing discriminative and generative modeling dates back atleast three decades in the pattern recognition literature eg [Ref1-4]. And in doing do, the paper resorts to existing sparse coding schemes - while it argues [14] learns dictionary in unsupervised way, there are efforts that learn dictionaries discriminatively (eg. [Ref5-6]). Thus there is not much new/interesting contribution in the paper.References:[Ref1] P. A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach (Prentice-Hall International, Englewood Cliffs, NJ, 1980).[Ref2] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd Ed. (Academic Press, New York, 1990. [Ref3] R. Schalkoff, Pattern Recognition: Statistical, Structural and Neural Approaches (John Wiley & Sons, New York, 1992)[Ref4] Lasserre, Julia A., Christopher M. Bishop, and Thomas P. Minka. "Principled hybrids of generative and discriminative models." Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on. Vol. 1. IEEE, 2006.[Ref5] Jiang, Zhuolin, Zhe Lin, and Larry S. Davis. "Learning a discriminative dictionary for sparse coding via label consistent K-SVD." Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011.[Ref6] Yang, Meng, D. Zhang, and Xiangchu Feng. "Fisher discrimination dictionary learning for sparse representation." Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 2011.


Efficient Structured Matrix Rank Minimization
We study the problem of finding structured low-rank matrices using nuclear norm regularization where the structure is encoded by a linear map. In contrast to most known approaches for linearly structured rank minimization, we do not (a) use the full SVD; nor (b) resort to augmented Lagrangian techniques; nor (c) solve linear systems per iteration. Instead, we formulate the problem differently so that it is amenable to a generalized conditional gradient method, which results in a practical improvement with low per iteration computational cost. Numerical results show that our approach significantly outperforms state-of-the-art competitors in terms of running time, while effectively recovering low rank solutions in stochastic system realization and spectral compressed sensing problems.

** I have read the authors’ response and they have provided satisfactory comments addressing my major concerns. I discuss some of these below (denoting added comments with **). I have read the other reviews and my rating of the paper hasn’t changed; I think it is a strong paper and should be accepted. I do not feel that the theoretical or experimental contribution pushes it into the oral/spotlight category.**The paper addresses the problem of finding structured low rank matrices for the case when the structure is encoded using a linear map and nuclear norm regularization is employed. The authors propose a modified formulation of the problem which makes it amenable to application of the generalized conditional gradient method. They provide an algorithm that incorporates other methods that accelerate convergence and show that it converges. Empirical studies suggest that the algorithm is significantly faster than its competitors and can scale to much larger problems. Strengths:(1) The paper addresses an optimization problem that lies at the core of multiple machine learning and signal processing tasks and the authors provide a novel procedure that can identify a suitable solution much faster than state-of-the-art approaches. The empirical studies demonstrate the efficacy of the approach for two useful application problems. Weaknesses:(1) The authors achieve the significant reduction in computational cost by modifying the objective function so that it is no longer directly penalizes the rank of the structured matrix. The authors make three reasonable arguments for doing so shortly after stating the objective function in (7).  The first two argue that the linear constraint could be satisfied asymptotically through a homotopy scheme or exactly through a projection, but the paper doesn’t explore either of these options, either empirically or theoretically. The theoretical result really addresses the convergence of the algorithm in terms of minimizing the modified objective function; it doesn’t provide a sense of the discrepancy between the identified solution and the more natural original formulation. Graphs 2(c) and 2(f) show some slightly strange behaviour in the rank of the solution as it converges. This behaviour is not discussed at all and it is not clear when the algorithm should be terminated to identify an appropriate solution.  ** The authors have argued that “any formulation that achieves this goal should be sufficient for the application”. I think this is a reasonable argument, but at the moment the paper is presented as though (7) is a relaxation/approximation of (5) – and is effectively trying to find the same solution. I think it would be useful to add a sentence or two making it clear that this is an alternative utility function that targets the same overall objective (low-rank model). **** With regard to the experimental results and the behaviour of the algorithm, it would be good to add some extra discussion either in the main paper or in the supplementary material expanding on what the authors have provided in their response” **Comments:Why does Theorem 1 cite [28]? Although there is a similar theorem in [28], with a similar proof approach, this doesn’t seem to be taken directly from there? What does ([28]) mean?** The authors acknowledge that the citation should appear in the proof, after a statement that the proof technique is inspired by the method in [28]. **


Difference of Convex Functions Programming for Reinforcement Learning
Large Markov Decision Processes are usually solved using Approximate Dynamic Programming methods such as Approximate Value Iteration or Approximate Policy Iteration. The main contribution of this paper is to show that, alternatively, the optimal state-action value function can be estimated using Difference of Convex functions (DC) Programming. To do so, we study the minimization of a norm of the Optimal Bellman Residual (OBR) T * Q − Q, where T * is the so-called optimal Bellman operator. Controlling this residual allows controlling the distance to the optimal action-value function, and we show that minimizing an empirical norm of the OBR is consistant in the Vapnik sense. Finally, we frame this optimization problem as a DC program. That allows envisioning using the large related literature on DC Programming to address the Reinforcement Leaning problem.

The paper presents a new technique for solving MDPs. The new technique, presented as an alternative to approximate policy/value iteration, consists in directly minimizing the Optimal Bellman Residual (OBR). The authors first motivate their method by showing that the loss bound of OBR is often tighter than the loss bound of policy/value iteration, which is a known result [9,15]. The authors then show that an empirical estimate of OBR is consistent in the Vapnick sense, i.e. minimizing the empirical OBR is equivalent to minimizing an upper bound on the true OBR, which is unknown when the MDP model is unknown. Finally, the authors show that OBR can be decomposed into a difference of two convex functions, and a standard Difference of Convex Functions (DC) optimization method can be used for finding a local optimum. The authors argue that the obtained optimum is usually not very different from the global one. Finally, the authors show that the performance of their method is similar to the performance of standard techniques, such as LSPI and Fitted-Q learning. The authors also show that the values obtained from OBR minimization have a smaller variance.Pros: The paper is very clear, well-written and well-organized. Technically, the paper is strong and sound. The proofs are not trivial and quite original. Theorem 4 (decomposition of OBR into a difference of convex functions) is probably the most useful result in this paper from a practical point of view. The motivation from a loss bound (Sec 2.2) is appreciated. Moreover, the authors prove the consistency of empirical risk minimization when the value function is approximated with basis functions (features). Cons: The empirical evaluation is very weak. The authors experiment only on a simple artificial MDP, and their method is not better than LSPI or Fitted-Q learning. If it will turn out that OBR minimization is never better than LSPI, then this whole contribution becomes questionable. Moreover, the authors did not report the computational effort needed for solving multiple convex optimization problems. My intuition is that if the DC decomposition is not well-done, than DC programming can be slower than DC programming.However, I agree that the main contribution of this paper is pointing to this new way of solving MDPs, which seems intriguing. I would like to see how the DC iterations are related to the policy/value iterations. Questions/comments:1) What is \mu in line 120? How is it different from the state-action distribution v? This should be introduced before Equation (1).2) Equation (3) is mostly a repetition of Equation (2)3) Typos in line 158: "better that", "manly", in line 313 "is currently is"4) The proof of Theorem 3 is straightforward, you can cut it off, remove Equation (3) and bring the more important proof of Theorem 4 to the main paper.5) In your experiment, is the computational cost of DC programming in the same order of magnitude as dynamic programing?


Analog Memories in a Balanced Rate-Based Network of E-I Neurons
The persistent and graded activity often observed in cortical circuits is sometimes seen as a signature of autoassociative retrieval of memories stored earlier in synaptic efficacies. However, despite decades of theoretical work on the subject, the mechanisms that support the storage and retrieval of memories remain unclear. Previous proposals concerning the dynamics of memory networks have fallen short of incorporating some key physiological constraints in a unified way. Specifically, some models violate Dale's law (i.e. allow neurons to be both excitatory and inhibitory), while some others restrict the representation of memories to a binary format, or induce recall states in which some neurons fire at rates close to saturation. We propose a novel control-theoretic framework to build functioning attractor networks that satisfy a set of relevant physiological constraints. We directly optimize networks of excitatory and inhibitory neurons to force sets of arbitrary analog patterns to become stable fixed points of the dynamics. The resulting networks operate in the balanced regime, are robust to corruptions of the memory cue as well as to ongoing noise, and incidentally explain the reduction of trial-to-trial variability following stimulus onset that is ubiquitously observed in sensory and motor cortices. Our results constitute a step forward in our understanding of the neural substrate of memory.

The paper returns to the venerable problem of associative memory in neural networks with a new fresh perspective, adding much new insight to an old problem. The authors observe that much previous work in the field ignored biological aspects such as Dale’s law, used binary memory representations in rate based models, and used non-sparse representations for memories. While all these issues have been dealt with individually in other work, the authors develop a strategy which allows them to address all issues. More concretely, they make use of a recent control-theoretic approach, based on spectral bounding techniques, to construct networks which deal with all these issues. In this approach the authors propose a cost function balancing three terms to achieve the desired goal through gradient descent. The first term in the cost contributes to the fixed point nature of the desired memories, the second term to the stability of the fixed points, and the third term encourages small weights which prevent saturation. By directly enforcing Dale’s principle in the weight parameterization, the authors are able to address all the above shortcomings of previous work, leading to a network where excitations and inhibition are naturally balanced. Interestingly, while they do not enforce saturation in the single neuron firing, this arises naturally through the solution of the optimization problem. Importantly, the authors only make use of the excitatory neurons to implement the memory (which is consistent with the fact it is usually the excitatory neurons that send output to other regions), and view the inhibitory neurons as variables to be optimized over.  The authors validate their results through numerical simulations, and demonstrate its noise robustness. In addition to demonstrating the good associative memory properties of the network, the authors are also able to explain experimental findings showing that the trial to trial variability is reduced following stimulus onset (although this seems to be a general feature of associative memories with spontaneous baseline firing). This is explained through convergence to the attractor closets to the initial condition. In summary, this is a clear and well written paper. It tackles a classic problem in neural computation using novel control-theoretic tools, and demonstrates how a biologically plausible associative memory may function. They do not propose a biologically plausible learning mechanism at this point, but hint at some recent proposals that may lead to such rules. I found the utilization of the inhibitory neurons as resources to be interesting and beneficial. Overall, the paper presents a clear, concise and important contribution to neural computation. A few minor issues:The authors use \gamma=0.04 in eq. (3). It would good to discuss the sensitivity of the results to this choice (although I expect it to be small)A log-normal distribution for memory states is proposed on line 115. Please motivate this. The cost function (4) may yield solutions which violate the exact fixed-point nature of the memories. How severe is this violation? This should be quantified and discussed. Line 254-256: The statements here were not clear to me. Please clarify.The authors do not mention capacity issues. It would be nice to make some statement, even preliminary, about this important computational aspect. It would be nice if the authors could discuss the computational benefits of Dale’s law. Namely, can this law be shown to enhance computational power under specific constraints? This is not necessarily the case, and may be a result of evolutionary and/or biophysical constraints, but it would be nice to relate to this.


Design Principles of the Hippocampal Cognitive Map
Hippocampal place fields have been shown to reflect behaviorally relevant aspects of space. For instance, place fields tend to be skewed along commonly traveled directions, they cluster around rewarded locations, and they are constrained by the geometric structure of the environment. We hypothesize a set of design principles for the hippocampal cognitive map that explain how place fields represent space in a way that facilitates navigation and reinforcement learning. In particular, we suggest that place fields encode not just information about the current location, but also predictions about future locations under the current transition distribution. Under this model, a variety of place field phenomena arise naturally from the structure of rewards, barriers, and directional biases as reflected in the transition policy. Furthermore, we demonstrate that this representation of space can support efficient reinforcement learning. We also propose that grid cells compute the eigendecomposition of place fields in part because is useful for segmenting an enclosure along natural boundaries. When applied recursively, this segmentation can be used to discover a hierarchical decomposition of space. Thus, grid cells might be involved in computing subgoals for hierarchical reinforcement learning.

Beyond representing the position of an animal in a given environment, the activity of neurons in the hippocampus (areas CA1,3) is known to be influenced by a range of task-dependent factors, for instance the presence of a reward at a specific location in the environment; yet we don't fully understand how these representations emerge and what they are good for. The present paper proposes that these observations are a reflection of the circuit implementing a specific algorithm (using a successor representation, SR, initially proposed by Dayan in 1993) for learning state values for reinforcement learning; moreover it suggests that the representation in an upstream region (medial EC) may provide a basis for a hierarchical decomposition of space.  Overall, some of the ideas put forward here are intriguing and potentially interesting for theoretical neuroscientists studying hippocampal coding, however the link to the neural data is relatively weak and the presentation of the material is difficult to follow in places. Detailed comments1. Content:Since the algorithmic part of the paper is not new, the key contribution of this work is the link between the SR representation  and the activity of neurons in the hippocampus. Unfortunately, this link between the two is not clear in several aspects:a) it is never spelled out exactly  how does the matrix M relate to the firing of the neurons in the corresponding hippocampal circuit.If there is a one-to-one map between firing rates and M(s,s'), how can a downstream circuit compute V(s)?One would need to include at least a cartoon version of how the whole algorithm would work at the level of the circuit.b) in SR the representation is a direct reflection of the geometry of the space on one side and the policy on the other. It is the second component that is particularly important for explaining task-dependent effects on neural activity; still, in section 3 it is barely stated why policy corresponds to which experimental result (there is a general description of the different learning algorithms but it is never spelled out what exactly was used when).  This is particularly important from the perspective of the dynamics of learning in the circuit: in an online setting, the policy changes in parallel to the state representation, which presumably could influence the final result (depending on the time constants for learning the different components).c) On a more computational note, the parallel learning of policy and representation can actually slow down learning in a some settings, for instance when the rewards contingencies change (see Dayan 93). In a constantly changing environment, it is not clear to me why it would be a good idea to have a policy-dependent representation rather than just doing proper model-based RL. Presumably it is the ease of learning that makes such a solution still appealing; these issues should be at least mentioned in the discussion. c) I find the link to the Stevens and Coupe behavioural results somewhat tenuous: in particular the notion of a boundary as defined there is slightly different than a simple spatial boundary (be it a 'semi-permeable' one); also I would not ascribe the processing of state boundaries to the hippocampus in particular; if nothing else, this bit needs to be justified more carefully.d) the activity of grid cells in MEC looks nothing like the fields in fig 6. Although the responses are periodic, the geometry is different (hexagonal grid) and border cells have elongated fields, these are restricted to the actual border and do not show any periodicity; I am unaware of any experimental data corresponding to panel c. More generally, I am unaware of any experimental work showing reward-dependent changes in these responses, which would be expected should these neural responses correspond to an eigendecomposition of the SR.Small technical note: the convolution with the Gaussian filter is used in the data analysis to smoothen out some of the noise due to limited sampling of the neural responses. I don't see a reason to use it here, and in fact smoothening the fields may prove misleading. Also, was this smoothing used consistently? (there does not seem to be any in the plots with boundaries)2. Clarity: I find the structure of the paper is somewhat difficult to follow, in particular the fact that important technical details are lumped together in the supplements, often without even being referenced in the main text. Also some of the details are missing, making it impossible to know exactly what was done to obtain the results in individual figures. 3. Significance: once the link to the neural circuit is made explicit and the presentation is improved, this could be a very interesting paper for the hippocampal coding subfield.


Making Pairwise Binary Graphical Models Attractive
Computing the partition function (i.e., the normalizing constant) of a given pairwise binary graphical model is NP-hard in general. As a result, the partition function is typically estimated by approximate inference algorithms such as belief propagation (BP) and tree-reweighted belief propagation (TRBP). The former provides reasonable estimates in practice but has convergence issues. The later has better convergence properties but typically provides poorer estimates. In this work, we propose a novel scheme that has better convergence properties than BP and provably provides better partition function estimates in many instances than TRBP. In particular, given an arbitrary pairwise binary graphical model, we construct a specific "attractive" 2-cover. We explore the properties of this special cover and show that it can be used to construct an algorithm with the desired properties.

Making Pairwise Binary Graphical Models AttractiveQuality - there are some surprising aspects to the experiments that are not discussed. See below. Clarity - paper covers a lot of ground many of the subparts are relatively well written, with references where necessary. But I found it surprisingly hard to get quickly a succinct overview of what is achieved. What would help is to emphasize in abstract and intro which claimed improvements are wrt approximating the Bethe free energy (in the direction of recent work by Weller and Jebara) and which wrt the true free energy. E.g. just from the introduction we get the statement "We then show that the computation of the Bethe partition function can approximated or in some cases found exactly by computing the Bethe partition function over this special cover." and " In these experiments, the proposed scheme converges signiﬁcantly more frequently than reweighted BP and provides a better estimate of the partition function than TRBP." The first is about approximations of the Bethe free energy, the second about the true free energy. Similarly for the assumptions about the model. Is the assumption of switching log-submodular models / signed graphs necessary? Originality - the paper follows a recent direction, but appears to make clear new contributions.Significance - finding better approximate inference algorithms is one of the most important challenges in the field. This paper makes advances and increases our understanding in a relatively fresh direction. Questions:- In Fig 5 in the appendix the BP-2 curve is non-smooth (assuming it is occluded by the other curve in the left part). Why is this? - Fig 3 seems to show a weird asymptote for BP-2 this does not seem to be discussed in the text. Did I miss this? What is going on here?


Learning Mixtures of Ranking Models *
This work concerns learning probabilistic models for ranking data in a heterogeneous population. The specific problem we study is learning the parameters of a Mallows Mixture Model. Despite being widely studied, current heuristics for this problem do not have theoretical guarantees and can get stuck in bad local optima. We present the first polynomial time algorithm which provably learns the parameters of a mixture of two Mallows models. A key component of our algorithm is a novel use of tensor decomposition techniques to learn the top-k prefix in both the rankings. Before this work, even the question of identifiability in the case of a mixture of two Mallows models was unresolved.

This paper considers the problem of the estimation of a mixture of two Mallows models. Though this problem has already attracted significant attention in the literature, the present paper is the first to introduce a polynomial time algorithm that provably solve it. It shows in particular the identifiability of the model, which was not known previously. The algorithm relies on two main ingredients:•	Explicit combinatorial formulas for the probability of the set of the k items ranked first under the mixture, for k = 1, 2, 3.•	The tensor decomposition of the matrix of these probabilities in terms of the representative vectors of the two Mallows models of the mixture.The statement of the algorithm is rather technical but it is well segmented into subroutines. Broadly speaking, the algorithm first randomly partition the set of items to obtain the mixture parameters as well as the diffusion parameters and the first elements of the central rankings of the two Mallows models, via a tensor decomposition. Then it completes the central permutations via a fine analysis of the different possible cases. The algorithm is proved to find the exact parameters of the mixture with high probability. At last, numerical experiments on synthetic data show that the proposed algorithm largely outperforms a classic EM based algorithm of the literature.This paper introduces explicit calculations on the Mallows model that, up to our knowledge, were never carried out before, highlighting a particular tensor-based structure. They are then exploited with a fine technical analysis in the design of a specific, novel algorithm. Finally the technical complexity of the approach is rewarded by convincing numerical experiments.


Real-Time Decoding of an Integrate and Fire Encoder
Neuronal encoding models range from the detailed biophysically-based Hodgkin Huxley model, to the statistical linear time invariant model specifying firing rates in terms of the extrinsic signal. Decoding the former becomes intractable, while the latter does not adequately capture the nonlinearities present in the neuronal encoding system. For use in practical applications, we wish to record the output of neurons, namely spikes, and decode this signal fast in order to act on this signal, for example to drive a prosthetic device. Here, we introduce a causal, real-time decoder of the biophysically-based Integrate and Fire encoding neuron model. We show that the upper bound of the real-time reconstruction error decreases polynomially in time, and that the L 2 norm of the error is bounded by a constant that depends on the density of the spikes, as well as the bandwidth and the decay of the input signal. We numerically validate the effect of these parameters on the reconstruction error.

The paper introduces a model and algorithms for decoding spikes from an integrate and fire decoder. Theorems that are providing upper bounds on the reconstruction error are given and the parameters and and error of the model are studied. The difference to prior work by the Lazar group does not become clear from the ms to the reviewer. Potentially the contribution comes from this lab. The simulations and numerical studies are on the weaker side. What I do not like is that the authors build up an expectation that their methods may be used in BMI decoding and then do not show any data. As a theoretical contribution, I consider it solid, but as said above, the margin to prior work does not become clear.


Fast Training of Pose Detectors in the Fourier Domain
In many datasets, the samples are related by a known image transformation, such as rotation, or a repeatable non-rigid deformation. This applies to both datasets with the same objects under different viewpoints, and datasets augmented with virtual samples. Such datasets possess a high degree of redundancy, because geometrically-induced transformations should preserve intrinsic properties of the objects. Likewise, ensembles of classifiers used for pose estimation should also share many characteristics, since they are related by a geometric transformation. By assuming that this transformation is norm-preserving and cyclic, we propose a closed-form solution in the Fourier domain that can eliminate most redundancies. It can leverage off-the-shelf solvers with no modification (e.g. libsvm), and train several pose classifiers simultaneously at no extra cost. Our experiments show that training a sliding-window object detector and pose estimator can be sped up by orders of magnitude, for transformations as diverse as planar rotation, the walking motion of pedestrians, and out-of-plane rotations of cars.

This work presents a method to efficiently train object detectors in the presence of geometric transformations that can be represented as vector-matrix multiplications. This has recently been developed for the case of translation transformations ([1,8,14]) but has not been too obvious for other transformations, such as rotations, let alone non-rigid deformations.  The authors propose to adapt the Fourier-based method that originally allowed the development of efficient algorithms for the case of translations so that we can now deal with rotations and other `cyclic' signal transformations (e.g. the walking pattern of a pedestrian). The condition for this to hold is that the  transformation is norm-preserving, can be represented as a matrix multiplication, x_transformed = Q x,  has an inverse Q^{-1} = Q^T and for some s Q^s = I.  The starting point for the previous works was the fact that the 'data matrix' obtained by 'stacking' together all translated versions of a signal is a circulant  NxN matrix, where N is the length of the signal, and as such can be diagonalized using the discrete harmonic basis (or, Discrete Fourier Transform-DFT matrix), Eq. 3, and reference [5].The authors  note that in the case of general transforms (not translations) the 'data matrix' may not be circulant in general - but as long as the conditions above hold, the Gram matrix will be.  As such, one can exploit the DFT-based diagonalization to efficiently train a detector. Using these, the authors obtain closed-form solutions for ridge regression that only involve inversions of diagonal Gram matrixes  - and present also a nice extension for the simultaneous training of multiple pose detectors. Turning to applying this idea to visual data, we are presented with two different methods for constructing the Gram matrix, one involving an analytic construction of the transformation matrix Q, and another being more `data-driven', where there is in principle no underlying linear transformation, but rather a cyclic pattern, as in the walking cycle of pedestrians, or even out-of-plane rotations of cars. Result-wise the authors demonstrate remarkable acceleration in terms of training time, albeit with an occasional loss in performance, even when comparing to a ridge regression baseline.   This work presents a method to efficiently train object detectors in the presence of geometric transformations that can be represented as vector-matrix multiplications. This has recently been developed for the case of translation transformations ([1,8,14]) but has not been too obvious for other transformations, such as rotations, let alone non-rigid deformations.  The authors propose to adapt the Fourier-based method that originally allowed the development of efficient algorithms for the case of translations so that we can now deal with rotations and other `cyclic' signal transformations (e.g. the walking pattern of a pedestrian). The condition for this to hold is that the  transformation is norm-preserving, can be represented as a matrix multiplication, x_transformed = Q x,  has an inverse Q^{-1} = Q^T and for some s Q^s = I.  The starting point for the previous works was the fact that the 'data matrix' obtained by 'stacking' together all translated versions of a signal is a circulant  NxN matrix, where N is the length of the signal, and as such can be diagonalized using the discrete harmonic basis (or, Discrete Fourier Transform-DFT matrix), Eq. 3, and reference [5].The authors  note that in the case of general transforms (not translations) the 'data matrix' may not be circulant in general - but as long as the conditions above hold, the Gram matrix will be.  As such, one can exploit the DFT-based diagonalization to efficiently train a detector. Using these, the authors obtain closed-form solutions for ridge regression that only involve inversions of diagonal Gram matrixes  - and present also a nice extension for the simultaneous training of multiple pose detectors. Turning to applying this idea to visual data, we are presented with two different methods for constructing the Gram matrix, one involving an analytic construction of the transformation matrix Q, and another being more `data-driven', where there is in principle no underlying linear transformation, but rather a cyclic pattern, as in the walking cycle of pedestrians, or even out-of-plane rotations of cars. Result-wise the authors demonstrate remarkable acceleration in terms of training time, albeit with an occasional loss in performance, even when comparing to a ridge regression baseline.   There is in my understanding a substantial amount of originality in this paper, proposing to combine recent advances in fast learning of object detectors in the presence of translations, with more challenging transformations. Some 'out of the box' thinking was required here, as it is most common to deal with rotations through polar coordinates - which would make everything harder; it turns out that by using the Gram matrix we are back to the setting of [14], which is a not-too-evident result; so I was very interested when originally reading the paper.   Still, the paper leaves more to be desired to make the paper significant to the learning/vision community. The most obvious critique is that the detection results are not compared to a stronger baseline, e.g. an SVM-based detector, so we do not know to what extent the presented method will be practically useful. Along the same lines, the authors could build a pose-invariant detector out of their system (by using the max over poses, as they discuss) and compare it to the methods that are currently out there for detection in the presence of pose variation (e.g. [24] or [c] below).  It is also disappointing to see that there is no discussion about how this method could be extended to other problems, beyond ridge regression. In [14], and (a) below, a combination of similar ideas with the hinge loss has been pursued; it should at least be discussed, and ideally tried out.  This is, in my opinion, the most important development that would be needed to make this paper have high impact. As it stands, the work would be of use only to practitioners who want a 'quick and dirty' detector, which may be only as good as one trained with ridge regression. In particular it is not true that ridge regression (+ whitening) performs similar to SVMs for detection; it merely serves as an 'ok' proxy. This was made more pronounced  in the DPM training work of R. Girshick, J. Malik 'Training Deformable Part Models with Decorrelated Features' in ICCV 2013  - there was quite a big difference in performance with and without the final SVM training stage.  I found it also disappointing  that the presentation is a bit blurred,  leaves several gaps to be filled in by the reader's imagination. I provide below a (partial) list of  points where this was the case.  Eq. 3: The DFT matrix F is never defined; so even if the reader has a signal processing background, one still needs to think about the period of the DFT (is it s or m?) whether F^H is the inverse of forward DFT (I understand it is the inverse) and what the normalization factors are; a single line equation would have made all of this clear.    l. 157-158: 'the matrices involved in a learning problem become diagonal, which drastically reduces the needed computations': There are many learning problems, and many ways in which matrices may be involved. Unless the authors pin down their optimization problem, show how matrices are involved in it,  and clarify to what extent the computations become reduced, this is a vague statement.  l. 182: 'structure of the covariance matrix is unknown': we do know the structure of the covariance matrix (we have an equation for it in l. 174 - and it is diagonalizable); we simply cannot express it in terms of the DFT matrix. l. 182: 'the diagonalization trick ... offers a way out, by computing the Gram matrix, and solving the dual problem'->The Gram matrix can be computed anyway, and the dual problem can be solved irrespectively of the diagonalization trick. So maybe the authors should be a bit more precise, stating e.g. 'The diagonalization technique allows us to express the Gram matrix in a form which makes an analytical solution of the dual problem possible.'l. 190: 'a regularized risk': The authors have to provide the form of the regularized risk. It is very common indeed, but presentation-wise it cannot be that the single most important thing (the optimization objective) is never written down. Strictly speaking the paper does not make sense from here on, since there are no 'primal', 'dual' objectives provided anywhere, but the authors keep referring to them. Lack of space is not a real excuse, several more verbose parts of the text can easily become condensed. l. 191: why should we have 's' dual variables? Where do they show up? In general, it is bad practice to leave all technical details to other references [5]/[23], since this practically requires that the reader reads three papers rather than one. A paper needs to be as self-contained as possible, and we are talking about things that can be explained with a couple of equations. l. 198-205: this could be placed right after theorem 1. Here it  breaks the presentation's flow l. 209: the reason why Eq. 6 follows from applying the circulant decomposition to G is not entirely obvious: to a casual reader it may seem as if you were saying F(a/b) = F(a)/F(b), with F being the Fourier transform, which does not make too much sense. The result is correct indeed (it is a one line proof), but some hand-holding would help. Stating a few central properties of circulant matrices in Section 2.2 would be useful. Section 4.1: this is, in my understanding, the same story as [14] (they also pass through the gram matrix in their optimization, and finally end up solving many simpler problems). If this is not the case, the difference should be clarified, it it is, it should be stated.   Section 5.1: from the style of the presentation, it is not too clear what exactly the authors do; we read about 'possibilities' (l.299,l.322), but it would be better if the authors state in advance that we use 'such and such a method for this experiment because of this reason'  and then go on to describe what the method consists in. Section 5.3: I could not understand this section. Apparently the authors are describing a technical problem that they faced, but they describe it in a somehow esoteric way : 'we have no principled way of obtaining negative samples for different poses for the same transformation ... lacking a definitive answer a simple strategy is .. - we can further optimize this by realizing that..'I understand that it is about negative samples not coming with a pose annotation - but I would guess that if one knows the transformation (e.g. rotation) all one has to do is apply all transformations to each negative sample, and label all transformed samples as negatives - so I do not see what is special about negative samples.I cannot see why the samples are constant along the DFT dimension. The labels may be (all zeros), but the features will be changing if we transform the image.  l. 371: rephrasing is needed (break into sentences, describing the setup: Pascal criterion assigns ground truth object to hypothesis; pose error is measured with respect to that object, as #i/#poses where #i is discretized pose difference) Table 2: what do you mean by 'with calibration'? Was the proposed method not calibrated?  Appendix: to prove that the data matrix 'will not be circulant in general', we would also need a counterexample for the case of non-translations (I understand that there are plenty - but providing one is a formal requirement).  One work that seems to me relevant (in the sense that it is learning the transformation Q) is  [a] Transformation Equivariant Boltzmann Machines, Jyri J. Kivinen and Christopher K. I. Williams, ICANN 2011,   The authors should also cite a precursor to the more recent works [1,14]: [b] Maximum Margin Correlation Filter: A New Approach for Simultaneous Localization and Classification Andres Rodriguez, Vishnu Naresh Boddeti, B.V.K Vijaya Kumar and Abhijit Mahalanobis  IEEE Transactions on Image Processing 2013 Finally, this is one of the works working on detecting rotated cars, the authors could compare to it (or [28])  [c] Andrea Vedaldi, Matthew B. Blaschko, Andrew Zisserman: Learning equivariant structured output SVM regressors. ICCV 2011


Signal Aggregate Constraints in Additive Factorial HMMs, with Application to Energy Disaggregation
Blind source separation problems are difficult because they are inherently unidentifiable, yet the entire goal is to identify meaningful sources. We introduce a way of incorporating domain knowledge into this problem, called signal aggregate constraints (SACs). SACs encourage the total signal for each of the unknown sources to be close to a specified value. This is based on the observation that the total signal often varies widely across the unknown sources, and we often have a good idea of what total values to expect. We incorporate SACs into an additive factorial hidden Markov model (AFHMM) to formulate the energy disaggregation problems where only one mixture signal is assumed to be observed. A convex quadratic program for approximate inference is employed for recovering those source signals. On a real-world energy disaggregation data set, we show that the use of SACs dramatically improves the original AFHMM, and significantly improves over a recent state-of-the-art approach.

This paper develops a new method of performing blind source separation, by formulating the problem as an additive factorial HMM (AFHMM), and then applying signal aggregate constraints (SACs). The motivation behind this is that additional domain knowledge can be incorporated to improve the separation of the time series into components. The example used throughout the paper is energy disaggregation, where the components of domestic energy use (relating to individual appliances) can be better separated, when information relating to total (expected) usage of each appliance in a time period is incorporated. The objective function that is maximized to perform the separation (which is the log of the posterior distribution of the hidden chains given the observed data) is then transformed into a convex optimization problem. The efficacy of the method is demonstrated in practice with both a toy data set, and with real data from domestic energy output readings.The paper is technically sound with a strong simulation section. The method, as expected, performs favorably in contrast to regular AFMM and also AFAMAP (a recently developed alternative method) - neither of which incorporate the SACs. The improvements are shown to be significant using t-tests. The author(s) also show that the speed of optimization is comparable to AFAMAP. My only question on the results section is why the author(s) used such a low noise variance in their toy example - is this to make the illustration of Figure 2 clear that they can separate the signal very well and other methods cannot? Some clarification would be good here otherwise the problem setup seems "overly" toy and rather easy. Or are these signal to noise ratios common in real data? On this note, the authors could also comment on the signal/noise ratio between the mains readings and the aggregate appliance output in the energy application.The theoretical aspects of the paper are overall well presented and structured. The paper would perhaps benefit from a proof that the final procedure in equation (6) is indeed convex, as it is not quite clear to me using the reasoning provided. In addition, theoretical scaling arguments (with respect to data length and number of hidden states) would benefit the paper, and compliment the simulation findings. Are the methods tractable if used for much larger data sets and state spaces, and how would this contrast with AFAMAP?The paper is well written and structured. The problem is nicely motivated in the introduction, with a clear description of relevant literature, though I am not that familiar with this problem, so cannot definitively state that nothing has been omitted here. I point to a small (potential) error on page 4 (line 202), in the unnumbered equation before (6), where I believe there should be no commas in the subscripts proceeding the object p, to be consistent with earlier notation on page 3 (unless new notation has been introduced here?)To my knowledge the work is novel and original, though I am not an expert so cannot comment on the significance of these results to the HMM community and to the energy application studied.Addendum: The authors satisfactorily answered my questions in the feedback phase, and I encourage them to make similar clarifications in the next version of the paper.


Model-based Reinforcement Learning and the Eluder Dimension
We consider the problem of learning to optimize an unknown Markov decision process (MDP). We show that, if the MDP can be parameterized within some known function class, we can obtain regret bounds that scale with the dimensionality, rather than cardinality, of the system. We characterize this dependence explicitly asÕ(
where T is time elapsed, d K is the Kolmogorov dimension and d E is the eluder dimension. These represent the first unified regret bounds for model-based reinforcement learning and provide state of the art guarantees in several important settings. Moreover, we present a simple and computationally e cient algorithm posterior sampling for reinforcement learning (PSRL) that satisfies these bounds.

The paper presents a general theoretical analysis of model-based reinforcement learning algorithms. The presented regret bounds depend on the Eluder dimensions of the function class that we want to learn for the transition function and the reward function. The eluder dimension captures the complexity of the model by, said intuitively, the number of samples that are needed to learn the real function well. This  theoretical analysis is applicable for many model-based RL algorithm, including the commonly used posterior sampling for reinforcement learning (PSRL) algorithm. Quality:The theoretical analysis seems to be well done in the paper, although the proofs are very hard to follow or check. While the theoretical contribution seems to be promising, the presentation of the paper is rather poor and I think it is almost impossible to understand the paper in a reasonable amount of time. For more comments on the clarity of the paper see below. Another maybe not so crucial point is the missing evaluations. I know it is not common for theoretical bounds, but I am always a bit sceptical about these papers where only bounds are presented without any evaluation. You never know how useful the bounds are. I think adding an evaluation of the presented bounds, for example, on a linear quadratic Gaussian (LQG) system  would highly upgrade the paper. For example, the authors could evaluate the regret of the PSRL algorithm for randomly sampled LQG problems with different dimensionalities and show the tightness of the bound. I think such an experiment would also help considerably to understand the contribution and it should be easy enough to do. Significance:Theoretical bounds for a very general set of model-based RL algorithms can have a high impact in the field. However, it is always the question how tight the bounds are in comparison to the results achieved with the real algorithms. Hence, even just a small experimental evaluation of the algorithm on a toy task would considerably increase the significance of the paper. Clarity:-The clarity is the main weakness of the paper. The authors assume too much knowledge of the reader about complex mathematical concepts that are not really common. I think the authors need to concentrate more on transmitting the message of the main result and put more of the very long derivations in the appendix. In the paper it would be more important to explain the intuition and motivation of each step that is important to obtain the bounds.- I think it is necessary to reduce Equation 4 to its most important parts. I do not think that not all the terms are relevant to understand the main message of the paper. The presented bounds go over two full lines. The authors should spend more time on explaining the more relevant parts of the equation (but put the full equation in the appendix)- Explaining the Eluder dimension and the Kolmogorov complexity right in the beginning would greatly simplify the understanding of the paper- define alpha covering and sub-Gaussian noise- define eluder dimension (section 4) before presenting the main results (section 3)- For each of the sections, a small description why the following steps are needed would be really helpful- The paper would also benefit a lot from proofreading. Many sentences are hard to understand due to grammatical mistakes and some of the used words are quite strange (see minor)Originality:The presented analysis using the Eluder dimension is a new contribution. Minor Points:- line 47: falls victim -> reformulate- line 48: exploit MDP structure -> the MDP structure- interedependence -> typo - "what is more, ..." -> reformulate (additionally, moreover, furthermore...)- Define distribution Q for Eq 2 in more detail (distribution over what, is it a conditional?)- 103: "contant"- Corollary 2: Typically in a continuous linear MDP, the action dimension is smaller than the state dimension (half for dynamical systems). How would that change the result?- Conclusion: reformulate "what is more". Quagmire is a quite uncommon word


Consistent Binary Classification with Generalized Performance Metrics
Performance metrics for binary classification are designed to capture tradeoffs between four fundamental population quantities: true positives, false positives, true negatives and false negatives. Despite significant interest from theoretical and applied communities, little is known about either optimal classifiers or consistent algorithms for optimizing binary classification performance metrics beyond a few special cases. We consider a fairly large family of performance metrics given by ratios of linear combinations of the four fundamental population quantities. This family includes many well known binary classification metrics such as classification accuracy, AM measure, F-measure and the Jaccard similarity coefficient as special cases. Our analysis identifies the optimal classifiers as the sign of the thresholded conditional probability of the positive class, with a performance metric-dependent threshold. The optimal threshold can be constructed using simple plug-in estimators when the performance metric is a linear combination of the population quantities, but alternative techniques are required for the general case. We propose two algorithms for estimating the optimal classifiers, and prove their statistical consistency. Both algorithms are straightforward modifications of standard approaches to address the key challenge of optimal threshold selection, thus are simple to implement in practice. The first algorithm combines a plug-in estimate of the conditional probability of the positive class with optimal threshold selection. The second algorithm leverages recent work on calibrated asymmetric surrogate losses to construct candidate classifiers. We present empirical comparisons between these algorithms on benchmark datasets.

The paper provides a study and accompanying algorithms for binary problems where performance metrics go beyond the mere classification accuracy; such performances are F-measure, AM-measure, Jaccard similarity coefficient, and, of course, precision and recall measures. The findings of the authors are a few:a) they show that there are many performance measures that can be expressed as ratio of linear combinations of the false positive (FP) rates, true positive rates (TP), true negatives (TN) and false negative (FN) rates. Working out a bit these ratio, it turns out that they can be expressed with respect to TP and probability for the classifier to predict a +1 class.b) a consequence of a) is that it is possible to show the Bayes classifier (which optimizes the performance metric at hand) is simply of the form sign(eta_x - \delta) where eta_x is the conditional probability for Y=1 given x.c) 2 consistent learning algorithms building upon b) (and therefore a)) are provided and proof of consistency (of one of them) are provided (and correct, as far as I have checked)d) numerical simulations on both controlled and benchmark datasets are provided that illustrate the soundness of the algorithms. Finally, a few directions for future research such as multiclass classification and multilabel classification are outlined.Quality------The paper is of very good quality, both from the mathematical and writing viewpoints. It makes reference to the pivotal works in the same family of contributions that help understand the proposed contributions. The structure of the paper is very sound and there is a nice balance between the depiction of basic things (cf. the linear ratios), the mathematical part (on consistency), the algorithmic part and the experimental part.Clarity------The paper is very clear. The authors made a good choice in postponing the most technical part to the appendix.Originality---------It is very nice to discover that there is still very interesting (and not just technicall) work to do in the realm of binary classification. This is all the more so that the topic addressed (richer performance measure than the accuracy) are of very high practical interest.Significance----------It is no doubt that this paper will be of very high interest for people working in information retrieval or for those looking for new learning framework to chew from the statistical standpoint.Minor comments--------------- insert a section 2.1 (named, e.g. "Metrics as ratios of linear combinations...") before the current section 2.1, or remove section title 2.1. (If there is only one subsection then there is no need to have any numbering.)- l. 387: "the methods, the methods"


Fast Multivariate Spatio-temporal Analysis via Low Rank Tensor Learning
Accurate and efficient analysis of multivariate spatio-temporal data is critical in climatology, geology, and sociology applications. Existing models usually assume simple inter-dependence among variables, space, and time, and are computationally expensive. We propose a unified low rank tensor learning framework for multivariate spatio-temporal analysis, which can conveniently incorporate different properties in spatio-temporal data, such as spatial clustering and shared structure among variables. We demonstrate how the general framework can be applied to cokriging and forecasting tasks, and develop an efficient greedy algorithm to solve the resulting optimization problem with convergence guarantee. We conduct experiments on both synthetic datasets and real application datasets to demonstrate that our method is not only significantly faster than existing methods but also achieves lower estimation error.

The authors consider the task of interpolation (cokriging) and forecasting on spatio-temporal tensor data. They show that both tasks can be posed as a (non-convex) low-rank tensor optimization problem, and proceed to present a learning method that combines a forward greedy selection with an optional orthogonal projection step. Experiments consider simulated data, two climatological datasets, and another containing FourSquare checkins.  I would not consider myself an expert in this particular domain, and am not in a position to comment on novelty or the presented proofs. That said, I find the paper clearly written. The presented greedy approach to low-rank learning seems reasonable, and the experiments show clear gains relative to other methods. In all this seems a solid paper with no obvious faults.Minor Points - While the discussion in 2.1 and 2.2 is easy to follow, it is not obvious to me how equation (4) in section 2.3 is equivalent to equation (1), how  equation (5) is equivalent to (3), and how both (4) and (5) are equivalent to (6). A reference or more verbose explanation in the supplementary material would be helpful.  - When discussing Fig 1c the authors write that the "run time of ADMM increase[s] rapidly with data size while the greedy algorithm stays steady".   This does not seem an accurate representation, as the run time clearly increases for all algorithms. If the longest performed run is indeed only 1000 seconds, then I would encourage the authors to further test the scalability of their algorithm with additional longer runs. - Is the RMSE used to characterize cokriging and forecasting performance normalized in some way? If so, it would be interesting if the authors could provide some insight into why there are comparatively large gains in the CCDS dataset, whereas all algorithms appear to perform well on the Foursquare dataset- Line 34: From machine learning perspective -> From [a] machine learning perspective- Line 305: five folds cross-validation -> five-fold cross-validation


A Boosting Framework on Grounds of Online Learning
By exploiting the duality between boosting and online learning, we present a boosting framework which proves to be extremely powerful thanks to employing the vast knowledge available in the online learning area. Using this framework, we develop various algorithms to address multiple practically and theoretically interesting questions including sparse boosting, smooth-distribution boosting, agnostic learning and, as a by-product, some generalization to double-projection online learning algorithms.

This work describes an approach for obtaining boosting algorithms from the mirror-descent algorithm for (online) convex minimization. The algorithms that are derived have provable guarantees in the PAC framework (that is given a weak learner with advantage \gamma it produces an \epsilon-accurate hypothesis). The framework is useful when boosting needs to be performed with additional (convex) constraints on the distribution since those can be obtained by changing the Bregman projection step. In particular, smoothness and sparsity are analyzed. The paper also claims to contribute new analysis of mirror-descent with lazy updates.This paper is a nice generalization of Kivinen-Warmuth 99 and Barak-Hardt-Kale 09 papers that only dealt with KL divergence to any Bregman divergence. The applications are nice but none of them seems both interesting and novel. Smoothness is certainly a useful property (e.g. for agnostic learning) but this case the approach to obtaining smoothness is identical to that in the BHK09 paper. In the case of sparsity the discussion of it is rather non-specific. No clear motivation and also no theoretical analysis of sparsity are given. The result only shows that it is possible to add an additional l_1 constraint into the boosting analysis. Without motivation and analysis it's impossible to tell for example how it would compare with subsampling.At the technical level the results seem nice but rely primarily on the well-known connection of boosting and online learning/optimization and known analysis of mirror-descent. The paper mentions a number of "second-order" improvements to aspects of the analysis but I'm not familiar enough with those to judge the importance and novelty. Overall I think that the paper has interesting technical content which should be published. At the same time I expect more novelty (either conceptually or technically) for a solid recommendation to NIPS.Some additional comments and typos:053 - The term smooth boosting was first defined by Servedio, COLT 01 not Gavinsky (but smoothness was used even earlier by Jackson, FOCS 94)095 - The bound on the number of steps lacks the \gamma_t term098 - Smoothness is also shown to be useful for agnostic boosting by Gavinsky, ALT 02 and by Feldman, ICS 10 for agnostic and PAC learning models.138 - It seems that later on you are using a different definition of entropy156 - is -> are169 - "norm of"170 - typos in the definition of entropy171 - what are x_i' here?279 - 3 typos415,419 conjunction->conjecture441 - M.Long is not an author of this article (P.Long is the editor)