Clustering Signed Networks with the Geometric Mean of Laplacians
Signed networks allow to model positive and negative relationships. We analyze existing extensions of spectral clustering to signed networks. It turns out that existing approaches do not recover the ground truth clustering in several situations where either the positive or the negative network structures contain no noise. Our analysis shows that these problems arise as existing approaches take some form of arithmetic mean of the Laplacians of the positive and negative part. As a solution we propose to use the geometric mean of the Laplacians of positive and negative part and show that it outperforms the existing approaches. While the geometric mean of matrices is computationally expensive, we show that eigenvectors of the geometric mean can be computed efficiently, leading to a numerical scheme for sparse matrices which is of independent interest.

This paper presents an argument for using the geometric mean of the normalized Laplacian matrix for each of the positive and negative normalized laplacian matrices  associated with a signed graph. The authors present some evidence for this idea in terms of a stochastic  block model.  This is a small but nice observation. They show that the eigenvectors of simple block-models (e.g. matrices (pI + q*ones) and variations for  the signed matrices). ) So the analysis is very simple for the highly idealized case. (There is no study of the empirical convergence to this distribution as  found in Abbe et al.; https://arxiv.org/pdf/1405.3267.pdf)  When this is used for spectral clustering, we need the eigenvectors of the geometric metean.  They also show how the eigenvectors of the geometric mean might be computed more rapidly than forming the geometric mean and computing it's eigenvectors. This is based on a particular application of the extended Krylov subspace method that can rapidly compute various functions of matrices. This contribution is less clear (see below) due to some existing work on the area whose relationship the authors should clarify.   The empirical results for real world datasets are fairly standard finding of "my new method is better". There  are various parameters (e.g. # of classes) that could  be varied (e.g. # of classes) and little sensitivity study, but this is not uncommon. The only troubling finding is on the Wikipedia signed network. The authors claim to have clustering structure based on a highly misleading plot, the dreaded "dot plot" which is arguably the weakest  evidence possible. This section needs far more evidence to support its supposed conclusions.    Questions for authors ---------------------  1. Could you clarify the relationship with other work on computing the geometric mean * a vector, e.g.  * Krylov iterative methods for the geometric mean of two matrices times a vector Jacopo Castellini (too new to have been included as a ref.)  * Computing the weighted geometric mean of two large-scale matrices and its inverse times a vector. http://eprints.ma.man.ac.uk/2474/01/covered/MIMS_ep2016_29.pdf  (It seems this is related to [8], which you already cite.)  In particular, what is the contribution of the Krylov algorithm in this paper vs. [8].  2. Is there additional evidence for clustering in the  wiki-sign network? Figs. 4(a) and 4(b) are not at all convincing. (Especially in light of multiple  negative findings in [15] an and [4].)  Suggestions for future work ---------------------------  Given the results on the stochastic block model are largely expressed with the same subspaces (e.g. the eigenvectors are the same for the pos. and neg, it might be nice to study a more straightforward quantity such as A*B or B*A (or their sum) as a baseline measure to quantify the significance of your approach. This would also suggest a simple extension to your theorem about the blockmodel case for a broader class of matrix functions.   Presentation remarks -------------------- * Harmless typos throughout that do not impede understanding * Theorems 1 and 2 are fairly simple and don't seem like they need to be highlighted. (And can probably be cited from existing work.)
 This is a small but nice contribution. It seems like much of the theory could be generalized to a broader class of matrix functions (e.g. many things with the product of the Laplacians would seem to meet many of the criteria, so there is nothing special about the geometric mean). This could enable and allow better computational methods that are far more straightforward.


Split LBI: An Iterative Regularization Path with Structural Sparsity
An iterative regularization path with structural sparsity is proposed in this paper based on variable splitting and the Linearized Bregman Iteration, hence called Split LBI. Despite its simplicity, Split LBI outperforms the popular generalized Lasso in both theory and experiments. A theory of path consistency is presented that equipped with a proper early stopping, Split LBI may achieve model selection consistency under a family of Irrepresentable Conditions which can be weaker than the necessary and sufficient condition for generalized Lasso. Furthermore, some 2 error bounds are also given at the minimax optimal rates. The utility and benefit of the algorithm are illustrated by applications on both traditional image denoising and a novel example on partial order ranking.

This paper is interested in recovering a (non-sparse?) signal. from noisy obervations. The used loss-function ressembles the lasso but the authors clearly state the differences. An optimisation algorithm is proposed. The work provides many usual theoretical guarantees.
Quite like the paper, funny application (not a really important scientifically motivated question though) towards the end. More exploiting the image denoising to compare to state-of-the-art might be more useful in our community. Lots of theoretical results. Clear which should be noted, it is not always the case at all in related works! A companion empirical study would be welcome.  Small questions:  - l13: explain in more details the meaning/interpretation (e.g. related to practical application) of $ m $ and $ p $. Can see noise in $ \beta^{\star} $. A noisy version of $ \gamma^{\star} $? Still, it is neither observed nor measured.  - l25: meaning of ADMM?  - l37: 'structural' sparsity, please explain.  - l41-44: rephrase please  - l50-51: relationship to Champion et al. JSPI 2014?  - l54 bias-free Oracle: lasso or LBI?  - l66: $ (D_{1}\beta)_{p} = 0 $ ??  - l72: AUC of ROC? What about Precision vs Recall? Depends on sparsity in recovered structure...  - Fig. 1 and corresponding text: include that the larger the $ \nu $, the smaller the penalisation to help with the interpretation. (e.g. l165/166)  - l107: is the restriction to $ L $ a big one? In which setting? Can be assessed on data?  - l109: suprised you don't need the dependence of the Hessian on $ \beta $ and $ \gamma $ since gradient with respect to these parameters are its components. Can you clarify please?  - eqn (2.5) definition of $ A $ is a repeat of line 128  - l134 typo: irrEpresentable; l135: which proveS...  - eqn (2.8) and ones after l152: what do these limits represent? Interpretation??  - end of 2.2 rephrase correctly to mention next section of what follows  - Theorem 2 condition on $ \kappa $ (l189 and 190) are these conditions always compatible??  - l195: comment on rate  - l196: what happens to the last term on the rhs of the equation? (idem in Theorem 3)  - l212: how do you know $ r' $ is small? Hopefully not because of $ n $?!  - l220: 'reset' ?  - l226: evolves: increases??  - l227: can't see picked up random noise on Fig. 2...  - Fig. 2 left: magnify!!


A Probabilistic Framework for Deep Learning
We develop a probabilistic framework for deep learning based on the Deep Rendering Mixture Model (DRMM), a new generative probabilistic model that explicitly capture variations in data due to latent task nuisance variables. We demonstrate that max-sum inference in the DRMM yields an algorithm that exactly reproduces the operations in deep convolutional neural networks (DCNs), providing a first principles derivation. Our framework provides new insights into the successes and shortcomings of DCNs as well as a principled route to their improvement. DRMM training via the Expectation-Maximization (EM) algorithm is a powerful alternative to DCN back-propagation, and initial training results are promising. Classification based on the DRMM and other variants outperforms DCNs in supervised digit classification, training 2-3⇥ faster while achieving similar accuracy. Moreover, the DRMM is applicable to semi-supervised and unsupervised learning tasks, achieving results that are state-of-the-art in several categories on the MNIST benchmark and comparable to state of the art on the CIFAR10 benchmark.

The authors present a generative model, called a Rendering Model which explicitly reasons about class and nuisance variation. More formally, their model generates pixels by patch and assumes a (gaussian) noisy image pixel, and an image template that is a function of the object class and a nuisance variable. The model also makes use of a switching variable that allows the model to deal with two objects overlapping (occlusion or transparency).  They demonstrate that inference in the Rendering Model is (nearly) equivalent to a one-layer conv net which uses a batch-norm (in the beginning, rather than at the end), a fully connected or convolutional layer, a max pooling layer and a relu layer.  They combine multiple Rendering Models to create a Deep Rendering Model (DRM). The DRM is a graphical model whose layers are modified rendering models resulting in a series of affine transformations that render finer and finer details of the object as the model progresses. They argue that a relaxed form of the DRM produces a convolutional neural network.  The DRM can be trained via EM which they argue provides faster convergence than standard gradient descent.   Given their model and the drawn equivalences to convolutional neural networks, they argue that the DRM provides several insights into convolutional networks strengths and weaknesses. Firstly, since DRM inference is equivalent to max-sum message passing, a conv net is effectively doing the same thing (making a number of assumptions about its structure). Secondly, they suggest that convnets poor performance on transparent objects is due to the ambiguity of the label being assigned to the transparency. They relate this weakness to the constraint in the DRM which enforces only a single object being rendered per pixel. Thirdly, they argue that like DRMs, convnets represent appearance models for classes in a factorized-mixture form over various levels of abstraction.   They validate their method on MNIST and experiment with comparing their supervised model with a simple convnet as well as demonstrating decent results for unsupervised and semi-supervised variations of their model. They illustrate decent performances for both and very good performance using the supervised DRM.
The paper presents a very intriguing model and make an ambitious argument about its equivalence with convolutional networks. If anything, the paper suffers slightly from an abundance of content which was very difficult (I imagine) to squeeze into the allotted space. The model is sure to be of interest to anybody working with conv nets with an interest in understanding how and why they work so well and may also generate additional future work. That being said, there were a number of questionable elements of the work which are illustrated below.  Novelty: To my knowledge, the formulation of the RM and DRM are themselves novel and the derivation (supplementary material) of the equivalence is quite interesting. This is probably the strongest part of the paper.  Technical Correctness: There is a relatively large assumption, or approximation is Proposation A.1 (a). This is not equivalent, but rather an approximation. I appreciate that the authors explain that this is in a noise-free limit of the GRM, but whether or not this is true in theory makes ALL the difference in practice. Any practical conclusions made by the paper are suspect if the assumptions made by the work are not true in practice...  Nit: the nuisance variables are 'g' in the paper but 'h' in the supplementary material. This is quite confusing.  Proposition A.1 (b) might be true for a single layer model where it has 3 channels as inputs but it will cease to be true in higher levels of the model.    There is a general claim being made between the equivalencies of the DRM and convnets. However, there is a very particular case of convents being compared to, and that is a convnet composed of (only) batch norm layers followed by conv layers, followed by max pooling layers, followed by relus. Oddly this doesn't actually match three of the more popular conv nets (AlexNet, VGG and GoogleNet) which do not use this configuration exactly (AlexNet uses local contrast normalization, VGG uses multiple convs in a row before pooling, GoogleNet uses Inception Modules which are combinations of various branches which are concatenated.    How is the image divided into patches? Stride 1 overlapping patches? How are boundary patches used?  The order of where batch-norm goes is slightly off. Most models perform batch norm AFTER the operation. Its true that in a deep model, this distinction is not meaningful (e.g. BN just goes between layers) but it certainly matters in the first and last layer.   144: overall pose g^L: this is an interpretation and needs to be validated experimentally.   Evaluation: I really wish any other dataset other than MNIST was used. This dataset has been hacked to bits and the margins in which various models operate in terms of gains or losses of performance are within a handful of samples. Consequently, I genuinely am not sure what to make of the results. That being said, their claim that the model achieves better accuracy (in the abstract) is not correct either (See DropConnect for example) and a 1-layer conv net used to compare to is a bit of a straw-man.   For a model motivated by the problem of explicitly modeling nuisance variables, it would be much better to use a dataset whose nuisance variables are clearer. What are the nuisance variables for MNIST?  Additionally, while the authors do provide a few qualitative results of the generative model (in the supplementary materials), they make a large and ambitious claim about WHAT the model is doing (providing finer and finer details of the image over each layer), but they make no attempt to validate this interpretation. Indeed, even if the DRM and DCN are equivalent in some way, they seem to have traded one problem (why do convents work so well) for another (what is each layer in the DRM really doing). For example, line 167, directly argues about the coarse-to-fine interpretation, but this needs to be validated.     Overall, the paper is quite interesting and worth discussion but there seem to be a large number of assumptions made which aren't true and the evaluation could be way better. I'm on the fence regarding acceptance.


Mixed Linear Regression with Multiple Components
In this paper, we study the mixed linear regression (MLR) problem, where the goal is to recover multiple underlying linear models from their unlabeled linear measurements. We propose a non-convex objective function which we show is locally strongly convex in the neighborhood of the ground truth. We use a tensor method for initialization so that the initial models are in the local strong convexity region. We then employ general convex optimization algorithms to minimize the objective function. To the best of our knowledge, our approach provides first exact recovery guarantees for the MLR problem with K 2 components. Moreover, our method has near-optimal computational complexity e O(Nd) as well as near-optimal sample complexity e O(d) for constant K. Furthermore, we show that our nonconvex formulation can be extended to solving the subspace clustering problem as well. In particular, when initialized within a small constant distance to the true subspaces, our method converges to the global optima (and recovers true subspaces) in time linear in the number of points. Furthermore, our empirical results indicate that even with random initialization, our approach converges to the global optima in linear time, providing speed-up of up to two orders of magnitude.

The paper under review studies the mixed linear regression problem, where the goal is to recover multiple underlying linear models from their unlabeled linear measurements.   The paper proposes a non-convex objective function, which, as shown in the paper, is close to being locally strongly convex in the neighborhood of the ground truth, and use a tensor method for initialization so that the initial modes actually fall into this strongly convex regime.   Specifically, the paper assumes pairs (y_i,x_i) to be generated as  y_i = < w_{z_i},x_i > ,   where z_i is an multinomial distributed index, w_k is the latent parameter of the k-th mixture component, and x_i is Gaussian.   Theorem 1 guarantees that provided the parameters w_k are independent of the samples and lie in the neighborhood of the solution, the Hessian is positive semi-definite (PSD) at the w_k with high probability. This does not imply, however, that the function is strongly convex around the true solution, as the statement is only for the Hessian at a set of given w_k (i.e., at one point, a set of w_k corresponds to one point in the parameter space).   Theorem 2 proves a similar statement than Theorem 2, this time, however, the w_k are not assumed to be independent of the samples. The cost for this stronger statement is that the w_k need to be significantly closer to the w_k^ast.  Again, the statement is for a given set of w_k, and does not hold for all w_k in a ball with high probability.   From those statements, the paper concludes that certain convex algorithms such as gradient descent converge to the true solution.  For this conclusion to be true, more has to be shown; I don't see how from  point wise strongly convexity, convergence follows (why don't we need strong convexity for all points in a ball w.h.p.?).   In section 3.3, the paper proposes a tensor method to initialize the parameters.  This initialization is interesting in that it provides an initialization that falls within the radius of point wise strong convexity of Theorem 1.  Additionally, it has complexity O(KdN + K^3 N poly(K) ). Thus, the complexity is small if K is small; however, for moderate to large K, the complexity is large.   In Section 4, the paper consider extensions to the subspace clustering problem.   Finally, the paper provides simulation results where the data points y_i are corrupted by additive noise.
Overall I think this paper presents an interesting approach, well suited for NIPS. Below are some comments and and also concerns.   1. My strongest concern is on whether Theorem 1 and 2 indeed guarantee convergence of algorithms like gradient descent. Specifically, Theorem 1 and 2 show that with high probability, for a given point {w_k}_k sufficiently close to {w_k^ast}_k, the Hessian is positive semi definite (PSD)---as the paper puts it ``strongly convex at a point''. However the definition of a parameter converging algorithm (Def. 1) and the standard convergence proofs of gradient descent require the function to be strongly convex in a region, not only at a point.  Note that Theorem 1 and 2 do not guarantee that the Hessian is PSD at all points simultaneously.  Maybe Theorem 1 and 2 can be used together with an epsilon-net argument and appropriate union bounds to show that the function is strongly convex in a certain region around the optimum. However this is not done in the current version of the paper.  It is therefore not clear to me whether Corollary 1 and the conclusion in Section 3.1 hold.  Could the paper please address this point by providing a formal proof of Corollary 1 (i.e., of the statement of Theorem 1 implying convergence of say gradient descent)?    2. The proofs are not very formal. Specifically, Appendix A states several lemmas guaranteeing that certain events hold with high probability, and, e.g., the proof of Theorem 1 simply assumes some of those events throughout the proof. However, the proof does not explain why the assumptions of the lemmata hold, and does not put together the events via say the union bound. Additionally, in the derivations the paper does not explain where exactly the assumptions come in.  This makes it difficult and time consuming to verify the correctness of the proofs.   3. Overall the paper is well written; the proofs however miss proper punctuation throughout.   4. Theorem 3 makes some abstract assumptions on the original w_k^ast (Assumption 1). Could the paper give some intuition for which constellations of w_k those assumptions will be satisfied, and for which they fail? E.g., by discussing examples.   5. Theorem 5 on subspace clustering assumes that the subspaces are already initialized in a way so they are very, very close to the true solution (Eq. (9)) (exponentially small in K!). That seems to be a very stringent condition, as it is not clear how one finds such an initialization.   6. Subspace clustering: It is very interesting that no conditions on the affinity of the subspaces are needed. Is the intuition for this result, that the paper does not consider actual clustering, but subspace recovery? E.g., if two subspaces are exactly the same, this would not be an issue for subspace recovery, as considered in the paper, but would make the clustering problem ill posed as a given point could be assigned to one or the other subspace.   7. The numerical results show that the algorithm does very well in terms of running time, which is great. However, the results are for K=3. For larger K, one would expect the results to be different, specifically, the other algorithms would, for large enough K, perform better in terms of computation time than the proposed algorithm PSC. It would be interesting to show simulation for larger values of K, to see for which K PSC is faster, and for which it is not. This would also illustrate that PSC is not uniformly better.


Online Convex Optimization with Unconstrained Domains and Losses
We propose an online convex optimization algorithm (RESCALEDEXP) that achieves optimal regret in the unconstrained setting without prior knowledge of any bounds on the loss functions. We prove a lower bound showing an exponential separation between the regret of existing algorithms that require a known bound on the loss functions and any algorithm that does not require such knowledge. RESCALEDEXP matches this lower bound asymptotically in the number of iterations. RESCALEDEXP is naturally hyperparameter-free and we demonstrate empirically that it matches prior optimization algorithms that require hyperparameter optimization.

The paper studies online convex optimization over a Hilbert space and without any assumption on the loss functions (besides convexity). This is a practically important optimization problem, as many machine learning problems can be reduced to it (via online to batch conversion).  The paper does a nice job explaining the history of this problem. This paper is the first that lifts the assumption to know an a priori bound on the norms of the gradients of the loss functions and achieves an almost linear dependency on the norm of the competitor. (Previous papers either had suboptimal regret or assumed bounds on either the domain or the norm of the gradients.)  The authors prove a lower bound on regret of any algorithm (Theorem 2.1). The lower bound shows that regret can be exponential in the worst case if the norms of the gradients grow too rapidly.  The main result of the paper is an algorithm that roughly matches the lower bound. The algorithm is FTRL with regularizer (x+1)*log(x+1)-x and a clever learning rate schedule. Similar regularizers were considered before, however, the learning rate schedule is novel and very technical.  The paper resolves a COLT 2016 open problem (Francesco Orabona and David Pal "Open Problem: Parameter-Free and Scale-Free Online Algorithms") by proving a lower bound. At the same it opens a host of new open questions, as it is clear that the proposed algorithm is not the final answer to the unconstrained online convex optimization problem -- but that's great news and most welcome! (E.g. doubling method in algorithm can be hopefully removed. Or one could imagine algorithms and regret bounds that are better under certain assumptions on the growth of the norms of the gradients.)  As far as I can judge, the results are correct. I checked the lower bound in great detail. I checked the proof of the main theorem.  However, I didn't check the lemmas in the appendix.  -----  Suggestions for improvement:  1) Please add reference to COLT 2016 open problem by Francesco Orabona and David Pal, "Open Problem: Parameter-Free and Scale-Free Online Algorithms". It directly addresses the problem in the paper. You might get $100 award :)  2) The ratio |g_t|/L(t) is not defined for t=1. This is happens since L(1) is not defined. L(1) can be defined either as zero or minus infinity, but in the either case the ratio |g_t|/L(t) is not defined. More generally, the ratio will not be defined if first few g_t's are zero. Please explain what is meant. The largest confusion is in Theorem 2.1 where if the reader understands the ratio |g_1|/L(1) as plus infinity,  the theorem does not hold. (Algorithm 1 is fine.)  3) In proof of Theorem 2.1. Please use gradients g_t instead of losses l_t(w) so that it matches statement of the Theorem (which uses only g_t's). I.e. please replace l_t(w)= -w with g_t = -1 and l_t(w)=2Tw with g_t=2T.  4) Also, in the proof of Theorem 2.1, between lines 83 and 84. The third inequality is really an equality.  5) Line 107: In the inequality (M_t + |g|_{1:t}^2 >= ...), M_t should be replaced by M_{t+1}. (Or shift indices on g_t's by one.)  6) Algorithm 1: A very minor issue arises when first few g_t's are zero. There are some divisions by zero in the algorithm.   7) Theorem 3.2. Please don't write g_T <= L_{max} >= L. I understand what was meant, but other readers might not. In particular, it is not clear if g_T < L < L_{max} is allowed. Simply, write g_T <= L_{max} and L_{max} >= L.   8) Please number the theorems simply as 1,2,3. Also, I would call Theorem 3.2 a lemma, since it's just technical result.  9)  Though, experiments on neural networks are fashionable these days, the algorithm would shine on regression with absolute loss much better.
This is a good and honest paper on an important practical problem (stated as an COLT 2016 open problem).  Research in this paper is a very big step towards a clean practical and optimal algorithm that will remove the need to "tune" learning rates for online convex unconstrained optimization problems.


An Ensemble Diversity Approach to Supervised Binary Hashing
Binary hashing is a well-known approach for fast approximate nearest-neighbor search in information retrieval. Much work has focused on affinity-based objective functions involving the hash functions or binary codes. These objective functions encode neighborhood information between data points and are often inspired by manifold learning algorithms. They ensure that the hash functions differ from each other through constraints or penalty terms that encourage codes to be orthogonal or dissimilar across bits, but this couples the binary variables and complicates the already difficult optimization. We propose a much simpler approach: we train each hash function (or bit) independently from each other, but introduce diversity among them using techniques from classifier ensembles. Surprisingly, we find that not only is this faster and trivially parallelizable, but it also improves over the more complex, coupled objective function, and achieves state-of-the-art precision and recall in experiments with image retrieval.
Information retrieval tasks such as searching for a query image or document in a database are essentially a nearest-neighbor search [33]. When the dimensionality of the query and the size of the database is large, approximate search is necessary. We focus on binary hashing [17], where the query and database are mapped onto low-dimensional binary vectors, where the search is performed. This has two speedups: computing Hamming distances (with hardware support) is much faster than computing distances between high-dimensional floating-point vectors; and the entire database becomes much smaller, so it may reside in fast memory rather than disk (for example, a database of 1 billion real vectors of dimension 500 takes 2 TB in floating point but 8 GB as 64-bit codes).
Constructing hash functions that do well in retrieval measures such as precision and recall is usually done by optimizing an affinity-based objective function that relates Hamming distances to supervised neighborhood information in a training set. Many such objective functions have the form of a sum of pairwise terms that indicate whether the training points x n and x m are neighbors:
Here, X = (x 1 , . . . , x N ) is the dataset of high-dimensional feature vectors (e.g., SIFT features of an image), h: R D → {−1, +1} b are b binary hash functions and z = h(x) is the b-bit code vector for input x ∈ R D , min h means minimizing over the parameters of the hash function h (e.g. over the weights of a linear SVM), and L(·) is a loss function that compares the codes for two images (often through their Hamming distance z n − z m ) with the ground-truth value y nm that measures the affinity in the original space between the two images x n and x m (distance, similarity or other measure of neighborhood). The sum is often restricted to a subset of image pairs (n, m) (for example, within the k nearest neighbors of each other in the original space), to keep the runtime low. The output of the algorithm is the hash function h and the binary codes Z = (z 1 , . . . , z N ) for the training points, where z n = h(x n ) for n = 1, . . . , N . Examples of these objective functions are Supervised Hashing with Kernels (KSH) [28]
(1) where for KSH y nm is 1 if x n , x m are similar and −1 if they are dissimilar; for BRE y nm = 1 2 x n − x m 2 (where the dataset X is normalized so the Euclidean distances are in [0, 1]); and for the Laplacian loss y nm > 0 if x n , x m are similar and < 0 if they are dissimilar ("positive" and "negative" neighbors). Other examples of these objectives include models developed for dimension reduction, be they spectral such as Locally Linear Embedding [32] or Anchor Graphs [27], or nonlinear such as the Elastic Embedding [7] or t-SNE; as well as objectives designed specifically for binary hashing, such as Semi-supervised sequential Projection Learning Hashing (SPLH) [34]. They all can produce good hash functions. We will focus on the Laplacian loss in this paper.
In designing these objective functions, one needs to eliminate two types of trivial solutions. 1) In the Laplacian loss, mapping all points to the same code, i.e., z 1 = · · · = z N , is the global optimum of the positive neighbors term (this also arises if the codes z n are real-valued, as in Laplacian eigenmaps). This can be avoided by having negative neighbors. 2) Having all hash functions (all b bits of each vector) being identical to each other, i.e., z n1 = · · · = z nb for each n = 1, . . . , N . This can be avoided by introducing constraints, penalty terms or other mathematical devices that couple the b bits. For example, in the Laplacian loss (1) we can encourage codes to be orthogonal through a constraint Z T Z = N I [35] or a penalty term Z T Z − N I 2 (with a hyperparameter that controls the weight of the penalty) [14], although this generates dense matrices of N × N . In the KSH or BRE (1), squaring the dot product or Hamming distance between the codes couples the b bits.
An important downside of these approaches is the difficulty of their optimization. This is due to the fact that the objective function is nonsmooth (implicitly discrete) because of the binary output of the hash function. There is a large number of such binary variables (bN ), a larger number of pairwise interactions (O(N 2 ), less if using sparse neighborhoods) and the variables are coupled by the said constraints or penalty terms. The optimization is approximated in different ways. Most papers ignore the binary nature of the Z codes and optimize over them as real values, then binarize them by truncation (possibly with an optimal rotation; [16]), and finally fit a classifier (e.g. linear SVM) to each of the b bits separately. For example, for the Laplacian loss with constraints this involves solving an eigenproblem on Z as in Laplacian eigenmaps [2,35,36], or approximated using landmarks [27]. This is fast, but relaxing the codes in the optimization is generally far from optimal. Some recent papers try to respect the binary nature of the codes during their optimization, using techniques such as alternating optimization, min-cut and GraphCut [4,14,26] or others [25], and then fit the classifiers, or use alternating optimization directly on the hash function parameters [28]. Even more recently, one can optimize jointly over the binary codes and hash functions [8,14,31]. Most of these approaches are slow and limited to small datasets (a few thousand points) because of the quadratic number of pairwise terms in the objective.
We propose a different, much simpler approach. Rather than coupling the b hash functions into a single objective function, we train each hash function independently from each other and using a single-bit objective function of the same form. We show that we can avoid trivial solutions by injecting diversity into each hash function's training using techniques inspired from classifier ensemble learning. Section 1 discusses relevant ideas from the ensemble learning literature, section 2 describes our independent Laplacian hashing algorithm, section 3 gives evidence with image retrieval datasets that this simple approach indeed works very well, and section 4 further discusses the connection between hashing and ensembles.

Binary hashing can improve information retrieval efficiency from both time and space perspectives. Most existing methods learn hashing codes and function by a composite joint optimization, which makes the optimization a hard problem. Inspired by the diversity requirement in classifier ensemble, this paper proposes to learn hash function for each hashing code independently. Three types of diversity strategies are used, namely on different initializations, different training sets and different feature subsets. The new approach benefits from its computational efficiency and nesting property. Diverse experiments validate the effectiveness of the proposed ILH strategy.
Inspired by diversity requirement to train a good classifier in an ensemble scenario, this paper proposes to train binary hashing codes independently. This Independent Laplacian Hashing (ILH) approach optimizes in a two-step way: first learns hash codes and then trains the hashing function. Three types of diversity are proposed, and from the experiment results the one learned with different training sets can achieve better performance.  This paper investigates ILH approach from different perspectives, and explains the results in detail. Some discussions about orthogonal codes are interesting.  Here are some suggestions/questions about the experiments: 1. Statement in line 186-188 is not very clear. 2. In most experiments, only precision is reported. For example, in Fig.1, its better to show the change of recall at the same time. 3. In Fig.1, different methods are plotted in different graphs. Ploting different approaches with the same b value in a single graph may be better to find their differences. 4. Since ILH method is based on the random initialization, training subset and feature subset, different trials will get various results. How to control experiments w.r.t. this randomness is important. Will different random trails all output good last results or the results are based on a specific randomize strategy? 5. Performance of ILHf relies on the property of features. It discards useful features when training on CIFAR dataset, will it perform better when there are irrelevant or redundant features? 6. Time comparison on different datasets (about different versions of ILH and other compared methods) may be necessary to show the property of the proposed method.


Statistical Inference for Cluster Trees
A cluster tree provides a highly-interpretable summary of a density function by representing the hierarchy of its high-density clusters. It is estimated using the empirical tree, which is the cluster tree constructed from a density estimator. This paper addresses the basic question of quantifying our uncertainty by assessing the statistical significance of topological features of an empirical cluster tree. We first study a variety of metrics that can be used to compare different trees, analyze their properties and assess their suitability for inference. We then propose methods to construct and summarize confidence sets for the unknown true cluster tree. We introduce a partial ordering on cluster trees which we use to prune some of the statistically insignificant features of the empirical tree, yielding interpretable and parsimonious cluster trees. Finally, we illustrate the proposed methods on a variety of synthetic examples and furthermore demonstrate their utility in the analysis of a Graft-versus-Host Disease (GvHD) data set.

The article discusses an inference problem for cluster trees arising from density estimation and proposes a method of constructing confidence sets by a bootstrap. In addition, to aid visualisation, additional pruning method is explored. The authors approach the problem by topological consideration of comparing trees, and propose an $\ell_\infty$ metric-based pruning. The main contribution seems to develop the framework for its theoretical justification, though some numerical examples are also included.   Overall, the problem is interesting and important, and the careful consideration of the theory is impressive, the solution provided is reasonable and practical.
1. Although I appreciate its importance of theoretical development, the numerical example does not seems to demonstrate well the potential of the proposed method. In particular, since the main objective is the confidence set, I wonder if there is a way to show the richness of the confidence sets so that the readers better appreciate the visualisation by pruning.   2. Related to the first point, I originally thought that the partial ordering was considered for comparing different trees, but the way the partial ordering is defined seems to require the main split should occur at exactly the same location (Fig.2), which would never happen for two different estimates, although the variation of the branching points would be small enough to define an approximation. So the partial ordering is well defined for pruning of the given estimate, but it wouldn't help for comparing the various trees within the confidence sets. Or do I miss something here?   3. The definition of the metrics in Section 3 doesn't necessarily assume that $p$ and $q$ are the density functions representing the same underlying population (like KDE estimates from two different realisations of the population), but any functions that are defined on the same domain so that they don't have to be close at all. Is it true?   4. Similarly, for $p_{inf}$ and the definition of $a$, if we consider density functions, unless they have compact support, $p_{inf}$ and $a$ would be zero. Then in Lemma 1, "more generally" means whether it has compact support or not?  5. Considering the distance with respect to the truth, intuitively as KDE has bias proportional to the second derivatives, the main error would occur at peaks and valleys, which are directly linked to the merging height. Then the equivalence of $d_\infty$ and $d_M$ is expected. However, this doesn't seem the case when we compare two different functions, the relative difference between these functions don't have to be amplified at those landmarks, yet I feel that the same principle should be relevant. Could you give some insights into the source of my confusion?  6. In figure 4, the second example (mickey mouse data), I would expect that heights would be different for the main part and two small clusters as the probabilities would be different. Why is it not the case?


Phased Exploration with Greedy Exploitation in Stochastic Combinatorial Partial Monitoring Games
Partial monitoring games are repeated games where the learner receives feedback that might be different from adversary's move or even the reward gained by the learner. Recently, a general model of combinatorial partial monitoring (CPM) games was proposed [1], where the learner's action space can be exponentially large and adversary samples its moves from a bounded, continuous space, according to a fixed distribution. The paper gave a confidence bound based algorithm (GCB) that achieves O(T 2/3 log T ) distribution independent and O(log T ) distribution dependent regret bounds. The implementation of their algorithm depends on two separate offline oracles and the distribution dependent regret additionally requires existence of a unique optimal action for the learner. Adopting their CPM model, our first contribution is a Phased Exploration with Greedy Exploitation (PEGE) algorithmic framework for the problem. Different algorithms within the framework achieve O(T 2/3 √ log T ) distribution independent and O(log 2 T ) distribution dependent regret respectively. Crucially, our framework needs only the simpler "argmax" oracle from GCB and the distribution dependent regret does not require existence of a unique optimal action. Our second contribution is another algorithm, PEGE2, which combines gap estimation with a PEGE algorithm, to achieve an O(log T ) regret bound, matching the GCB guarantee but removing the dependence on size of the learner's action space. However, like GCB, PEGE2 requires access to both offline oracles and the existence of a unique optimal action. Finally, we discuss how our algorithm can be efficiently applied to a CPM problem of practical interest: namely, online ranking with feedback at the top.

The paper considers partial monitoring in the stochastic combinatorial setting with linear feedback mechanism, the version of the original problem that was introduced by Lin et al. in [1], and improves on some of the results in [1]. In particular, the paper proposes two algorithms, PEGE and PEGE2, for globally observable problems, both of which avoids some (but not all!) of the limitations of algorithm GCB of [1], including uniqueness constraint of the optimum, having access to oracles that solve (potentially) hard optimization tasks, and having a sample complexity that depends on the size of the action space.
The advantages of the PEGE algorithms, compared to GCB is that - they do not require an oracle that returns the second best solution of an optimization problem - they do not require the uniqueness of the optimal solution - their regret do not depend on the size of the action set. The disadvantages of the PEGE algorithms, compared to GCB are that - the claimed distribution dependent bound requires setting parameter h so that it satisfies some distribution-dependent conditions, which is not clear how to do efficiently if the parameters of the problem are not known - the distribution dependent bound has worse dependence on the time horizon T, and has an extremely bad (exponential) dependence on h. - to guarantee the claimed distribution dependent bounds requires different parameter setting than to guarantee the claimed distribution independent bounds.  The issue with parameter h is a serious one, and thus the paper proposes a solution to it in form of another algorithm, PEGE2. However, the obtained algorithm does not provide a satisfactory solution to this issue, and suffers from other shortcomings too, as discussed below. Advantages, compared to GCB: - its regret does not depend on the size of the action set Disadvantages, compared to GCB: - the distribution dependent bound is not guaranteed in every case (see Equation (7)). It is not clear what kind of distribution dependent bound can be guaranteed in general.  Finally, none of the proposed algorithms can avoid solving linear optimization problems in every step.  In view of this, the results in the paper should be considered as interesting additions to our knowledge of the combinatorial partial monitoring problem, but it is not clear whether they take us any closer to an actual, effective solution of the problem. In particular, the obtained distribution dependent bounds seem particularly week. The strongest part, however, seems to be that the regret bounds do not depend on the size of the action set any more. It would be nice to highlight the main idea behind this in the main part of the paper.   Additional remarks - The paper could benefit from adding a formal (algorithm-like) presentation of the protocol. - Please make it explicit that sigma in line 4 of Algorithm 1 is the globally observable set. - The remark in lines 176-180 is confusing. In what sense is the bound in [6] less explicit? Additionally, there should be an "is" between "in" and "not" in line 177. - Why do you introduce \delta, when it is always set to 1/T? - When adapting the combinatorial partial monitoring games to online ranking, how do the obtained results compare to existing methods? - "note -ve sign" in line 304 seems to be a typo. - Citation 4 in line 317 does not have all the authors names.


Provable Efficient Online Matrix Completion via Non-convex Stochastic Gradient Descent
Matrix completion, where we wish to recover a low rank matrix by observing a few entries from it, is a widely studied problem in both theory and practice with wide applications. Most of the provable algorithms so far on this problem have been restricted to the offline setting where they provide an estimate of the unknown matrix using all observations simultaneously. However, in many applications, the online version, where we observe one entry at a time and dynamically update our estimate, is more appealing. While existing algorithms are efficient for the offline setting, they could be highly inefficient for the online setting. In this paper, we propose the first provable, efficient online algorithm for matrix completion. Our algorithm starts from an initial estimate of the matrix and then performs non-convex stochastic gradient descent (SGD). After every observation, it performs a fast update involving only one row of two tall matrices, giving near linear total runtime. Our algorithm can be naturally used in the offline setting as well, where it gives competitive sample complexity and runtime to state of the art algorithms. Our proofs introduce a general framework to show that SGD updates tend to stay away from saddle surfaces and could be of broader interests to other non-convex problems.

The paper studies a stochastic gradient descent algorithm for the problem of matrix completion. With a good initialization, it is guaranteed to converge to the true matrix with a geometric rate under standard assumptions. The algorithm is simple and easy to implement. The authors also make efforts to sketch the high-level idea for the proof.
The main contribution in this work is showing that in the context of matrix completion, when equipped with a good initialization, the sequence of the solutions produced by a widely used (but without theoretical guarantee) SGD algorithm converges linearly to the true matrix. The paper reads well and most of theoretical result looks sound. But I have some concerns as follows.  - To guarantee a good initial solution, it (Theorem 3.1) requires a sample complexity "O(mu * d * k^2 * log(d))", which is of the same order as the batch solvers. Is it possible to access fewer samples, e.g., "O(mu * d * k * log(d))" while still ensuring global convergence?  - The vanishing probability of success with respect to iteration counter is strange (it does not conform the intuition). Theorem 3.1 says "for any fixed T >= 1, with probability at least 1 - T/(d^10), we have linear convergence". That means, if we run the algorithm (i.e., repeatedly reveal the samples) for a longer time, we have a LOWER confidence to obtain the true matrix. For example, picking "T = 0.9 * d^10", then we are just guaranteed to obtain convergent solutions with probability at least "10%". If we pick "T" much larger, say "T = O(d^20)", it is not clear whether the algorithm succeeds. However, in practice, one should expect that if the algorithm continues to update the solution, asymptotically, the optimum is obtained.  ---------------------------- Updates:  I slightly increase my score on "technical quality" from "2" to "3", given the considerable amount of work involved. I believe the techniques here are interesting to the community, and many people will pick it up. Yet, it would be grateful if the authors could present a more careful proof technique to get rid of the dependency of "T", or comment on that "T" is always very small.  A minor comment: It appears that another parallel work [A] also studies alternating minimization in online setting. It would be grateful if the authors could comment on the difference.  [A] Fast Algorithms for Robust PCA via Gradient Descent, arXiv:1605.07784


Fundamental Limits of Budget-Fidelity Trade-off in Label Crowdsourcing
Digital crowdsourcing (CS) is a modern approach to perform certain large projects using small contributions of a large crowd. In CS, a taskmaster typically breaks down the project into small batches of tasks and assigns them to so-called workers with imperfect skill levels. The crowdsourcer then collects and analyzes the results for inference and serving the purpose of the project. In this work, the CS problem, as a human-in-the-loop computation problem, is modeled and analyzed in an information theoretic rate-distortion framework. The purpose is to identify the ultimate fidelity that one can achieve by any form of query from the crowd and any decoding (inference) algorithm with a given budget. The results are established by a joint source channel (de)coding scheme, which represent the query scheme and inference, over parallel noisy channels, which model workers with imperfect skill levels. We also present and analyze a query scheme dubbed k-ary incidence coding and study optimized query pricing in this setting.

The paper considers a crowdsourcing set up where a taskmaster divides a big task (say an inference problem) into small tasks. These small tasks are then presented to crowd or workers. A crowdsourcer then collects the information provided by these workers (after completion of their assigned tasks) and process this information to conclude the inference task. This setup allows for the worker to have varying skill levels as well as to be unreliable in reporting their information to the crowdsourcer. The objective of the whole system is to ensure certain level of fidelity or 'goodness' of the final outcome at the crowdsourcer, even in the presence of some wrong information presented by the participating workers. This requires the taskmaster to carefully design small tasks (or queries) to be allocated to the workers. The paper defines the total number of queries presented to the workers as the budget of the crowdsourcing system.  The paper studies a fundamental trade-off between the budget (number of queries presented to the workers by the taskmaster) and fidelity (quality of the inference task performed at the crowdsourcer). This trade-off is obtained by mapping the crowdsourcing system described above to the problem of joint source channel coding in information theory literature. The taskmaster is treated as an transmitter and the crowdsourcer is mapped to a receiver/decoder. The unreliable links from the workers to the crowdsourcer constitute a communication channel. Given this mapping a lower bound on the budget (no. of queries) for a desirable fidelity level follows from the standard results in information theory literature. This bound is independent of the schemes used by the taskmaster to design the queries and the algorithm used at the crowdsourcer to process the information collected from the workers. The authors then consider a specific scheme at the taskmaster, called k-ary incidence coding and analyze its performance. The authors then compare the performance of this encoding scheme with the obtained information theoretic lower bound for a specific channel model.
The reviewer finds the parallel drawn between the rate-distortion framework and the crowdsourcing system with unreliable workers quite interesting. This work may invite other researcher to study the crowdsourcing framework using information theoretic tools. However, the reviewer is not sure if the results presented in this paper model the real crowdsourcing systems in a very realistic way. One reason behind this could be that the paper wants the crowdsourcing problem to exactly map to the point-to-point communication problem so that the existing results from the information theory literature can be directly applied with the minimal efforts.  It would be great if the authors can comment on the following issues.   1) It seems that the reliability and skill level of the workers is merged together and modeled as a single channel. The paper assumes that the transmitter (taskmaster) allocates same no. of queries to all the workers irrespective of their skill level. This may not be true in the real systems where the workers may carry some rating with them which represents their skill level.  2) How realistic are the channel models to take the unreliability of the workers into account. What about the workers which can have some knowledge of the task at hand and try to degrade the fidelity in an adversarial manner.  The explanation of Numerical results is not very clear. The labeling of various curves in Fig.~2 can be improved. What does $k=1$ case correspond to. The worker directly gives you the correct label of the presented query as you can not present a pair and ask if they share the same label in this case?  In page 6, line 246 the authors mention that they assume the knowledge of $P(B(X))$. The reviewer believe that this should have been stated much earlier in the paper.   In page 4, line 135:  Figure 1.a --> Figure 1.b In page 4, line 135:  Figure 1.b --> Figure 1.c  The authors repeatedly use 'Theorem', 'Lemma' and 'Corollary' even when these terms are not accompanied by a specific theorem or lemma number. The reviewer believes that 'theorem', 'lemma' and 'corollary' would be more appropriate in such cases.


Greedy Feature Construction
We present an effective method for supervised feature construction. The main goal of the approach is to construct a feature representation for which a set of linear hypotheses is of sufficient capacity -large enough to contain a satisfactory solution to the considered problem and small enough to allow good generalization from a small number of training examples. We achieve this goal with a greedy procedure that constructs features by empirically fitting squared error residuals. The proposed constructive procedure is consistent and can output a rich set of features. The effectiveness of the approach is evaluated empirically by fitting a linear ridge regression model in the constructed feature space and our empirical results indicate a superior performance of our approach over competing methods.

This paper presents an effective method for feature construction by exploiting the scalability of existing algorithms for training linear models while overcoming their low capacity on input features. The authors first embed the data into a space with a high capacity set of linear hypotheses, and then use a greedy approach to construct features by fitting residuals.  Overall, I think the contribution of this paper is significant. As noted by the authors, linear models are frequently used for learning on large scale data sets. Yet, the set of linear hypotheses defined on input features is usually of low capacity, thus leading to inaccurate descriptions of target concepts. By embedding the input data into a high capacity set of linear hypotheses and then constructing features in a greedy manner, the proposed method can construct a high capacity feature space. The experimental results also demonstrate the effectiveness of the proposed approach.
This paper presents an effective method for feature construction by exploiting the scalability of existing algorithms for training linear models while overcoming their low capacity on input features. The authors first embed the data into a space with a high capacity set of linear hypotheses, and then use a greedy approach to construct features by fitting residuals.  Overall, I think the contribution of this paper is significant. As noted by the authors, linear models are frequently used for learning on large scale data sets. Yet, the set of linear hypotheses defined on input features is usually of low capacity, thus leading to inaccurate descriptions of target concepts. By embedding the input data into a high capacity set of linear hypotheses and then constructing features in a greedy manner, the proposed method can construct a high capacity feature space. The experimental results also demonstrate the effectiveness of the proposed approach.   Despite of its significance, I think the paper can be improved in the following aspects.  First, the flow of this paper should be improved. Although English grammar is ok, this paper is not very easy to follow. I understand that the authors need to meet the page limits, but it would be better if the authors can include several sentences in the introduction part to summarize the proposed method, which serve as a clue for the following paragraphs.  Second, the effectiveness of the proposed method has been clearly demonstrated by the experiments. Is it possible for the authors to evaluate the classification accuracy for the proposed approach?    Therefore, my opinion for this paper is borderline.


Brains on Beats
We developed task-optimized deep neural networks (DNNs) that achieved state-ofthe-art performance in different evaluation scenarios for automatic music tagging. These DNNs were subsequently used to probe the neural representations of music. Representational similarity analysis revealed the existence of a representational gradient across the superior temporal gyrus (STG). Anterior STG was shown to be more sensitive to low-level stimulus features encoded in shallow DNN layers whereas posterior STG was shown to be more sensitive to high-level stimulus features encoded in deep DNN layers.

This paper performs a RSA similarity analysis assessment of the similarity of trained DNN to human fMRI traces, using music samples as stimuli.  This type of approach has been highly successful in the visual domain to improve on the understanding of the visual hierarchy, finding great correlations between DNN trained on ImageNet and human/monkey IT.  The authors replicate this using music datasets (MagnaTagaTune to train the DNN, Studyforrest for the RSA).  Their DNN models trained on MagnaTagaTune obtain SOTA classification accuracy. They then find that DNN layers representations similarity seems to go from anterior to posterior along the STG, providing some hints about the salient features represented in STG.
This paper has 2 main contributions: 1) Improve the SOTA on DNN for music tag classification. 2) Look for correlations with the human STG through a RSA + Searchlight procedure.  The treatment of 1) is good and well explained. In Section 2.3, it would be more helpful to spell out directly the number of filters, along with sizes and strides per layers, instead of referring to AlexNet and providing the set of rules to adapt them...  A small figure would be more space efficient. All the latest tricks and techniques were used, there were no flawed in my opinion.  Compared to the previous state of the art in music classification using DNN, it seems like the main difference is a drastic change in filter sizes and using the latest optimizers. If I parsed the methods properly, they use conv filter sizes {121, 25, 9, 9, 9} (strides similarly adapted), compared to {8, 8} in [32]. This is quite a drastic improvement, which might explain the SOTA classification performance. Still, great accomplishment!  The authors do not comment on the low performance for "no vocals" types of labels, it would be interesting to address it (my guess would be that this is a "negation" label which has huge support and variance and is hard to classify from 6sec clips).  Considering contribution 2), it's a good replication of the work performed in the visual domain by several other groups (cited appropriately). The techniques used are sound and SOTA (RSA + Searchlight), but I do not have the knowledge to assess the specificities of the fMRI analysis. The baseline models used for comparison are sound and a welcome addition.  It would have been helpful to provide the RSA matrices of the fMRI data itself, in order to visually compare to Figure 3, as is usually done in the visual DNN-RSA literature (if I missed it in the published papers I apologize, but couldn't find it in [23]). Given the strong ambient/symphonic vs rest subdivision, regressing that effect out would have been interesting (similar to [13] for animate vs inanimate).  I'm not sure how to interpret the pretty low mean representational similarity in Figure 5A when going posterior along the STG. This doesn't make me want to trust the significance of the (layer 6 ; posterior STG) correlation much. Similarly, the drop in similarity as the layer index increases in Figure 4 is a bit contrary to what people have found when looking at IT correlations... It would seem that these results only show that early layers processing are similar to STG, but not later ones. The paper is a bit lacking in interpretations and discussions on that aspect.  Given what is known about Wernicke's area, I would have liked to see style-conditioned similarities: did music genres with vocals representations significantly differ between hemispheres?  Overall, it seems like part 2) is less discussed and lacks a bit of interpretation and context to be really useful for NIPS-attendees. But overall I think it is a strong paper which will find its audience.


f -GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Generative neural samplers are probabilistic models that implement sampling using feedforward neural networks: they take a random input vector and produce a sample from a probability distribution defined by the network weights. These models are expressive and allow efficient computation of samples and derivatives, but cannot be used for computing likelihoods or for marginalization. The generativeadversarial training method allows to train such models through the use of an auxiliary discriminative neural network. We show that the generative-adversarial approach is a special case of an existing more general variational divergence estimation approach. We show that any f -divergence can be used for training generative neural samplers. We discuss the benefits of various choices of divergence functions on training complexity and the quality of the obtained generative models.

This paper shows how the GAN framework can be extended to train the model with many different divergences, rather than just the Jensen-Shannon Divergence.
I think this paper should be accepted.  The authors should feel free to take a few sentences from what I say here and put it in the paper: Currently, many researchers working on generative models are interested in the differences between GANs and VAEs. A general consensus, prior to the appearance of this paper on ArXiv, was that VAEs were good at assigning high likelihood to data points, and GANs were good at generating realistic samples, and that this difference is due to the differing cost functions used. Because this paper shows that GANs behave similarly whether they are trained with the KL divergence cost or the cost proposed in [10], that changes the interpretation of the situation. It now seems that GAN samples are high quality due to something about their training procedure or the way the model family is specified, not because of the cost function used to train them. So this paper provides important evidence that is relevant to settling an important scientific question.  It's also possible that the specific divergences provided here might find practical application.   Here are a few things that should be improved, but that I don't think are dealbreakers: - There's one big technical problem here, which is that the bound goes the wrong way. Usually for variational methods, we want to minimize an upper bound on the cost, but here we are minimizing a lower bound. That means that in the general case, the algorithm may exhibit strong non-convergence behavior. This problem is discussed a little bit in this ICLR workshop paper: https://arxiv.org/pdf/1412.6515v4.pdf We also don't really know how much we are improving the true cost, because we don't have any idea how tight the bound is. The algorithm seems to be at least somewhat sane in practice, so it isn't necessarily broken, but it's worth discussing in the paper that the bound doesn't automatically give us all of the theoretical guarantees it gives in the usual variational learning setting. - The same ICLR workshop paper ( https://arxiv.org/pdf/1412.6515v4.pdf ) also shows how to modify GANs to minimize the KL divergence, but in a different way than this paper. It's worth discussing this in the related work. This work shows how to construct a bound on the KL. The workshop paper shows how to make a stochastic approximation to the gradient of the KL that would be unbiased if the discriminator were optimal, but in practice is subject to overfitting and underfitting in the discriminator. The technique in the workshop paper requires departing from the minimax game formulation and instead uses an independent cost for each player. - The original GAN paper [9] was accompanied by an open source code release that included the algorithm presented in sec 3.1, and several follow-up projects based on [9] have used that same procedure, so the algorithm itself is not an original contribution of this paper. However, the proof that it converges is. - Likewise, the practice of monitoring true positive rates / true negative rates is attributed to Larsen and Sønderby, but this was actually done in the original code release accompanying [9]. -  Typo on line 35: "networks" instead of "network" - The review of f-divergences might explain whether there are any desirable properties of the family of f-divergences. This could help motivate the paper a bit more. As the reader, I have the general idea that for an underpowered model, some divergences result in choosing a subset of modes to capture, while others result in averaging modes together, but I don't know exactly what the benefit of having a wide family of divergences is. - Eq 4: I didn't understand why it is necessary to use Jensen's inequality to pull the sup out of the integral. In the case of discrete x, the sup could be pulled outside of the sum with strict equality, as long as the family of functions for T(x) is unrestricted. Is the use of Jensen's inequality just necessary because issues that are specific to continuous x, like restrictions to function families that are integrable / handling cases where the supremum is not actually attained so it's not possible for T(x) to take on a value that results in the supremum? (This is a minor point because I agree that the restriction of the family of T(x) requires an inequality) - In the discussion of Eq 4, it's worth mentioning that in practice, part of the reason we'll have an inequality is that T(x) will be imperfectly optimized. - Numerous issues with Table 2: First, there's one issue with the evaluation itself: The table on the right seems to show that the variational bound reaches nearly its optimal value, but because we don't see (estimates of) the true divergence, we don't have any idea how tight that bound is.  The rest of the issues with table 2 are related to clarity. I spent approximately half an hour reading it, feeling like I was taking an IQ test. These should be easy to fix, given a specification of what confused me:  1) I don't understand what "objective values to the true distribution" means. 2)  I wasn't sure what the F values in the table in the left were. Are these:   a) The theoretical minimum of F   b) The value of F found by your training algorithm   c) The value of F attained when D is at its theoretical minimal?  I think the answer is (a), based on issue #4  3) Are the values in the table on the right the lower bounds or the actual divergence values? I think they are the lower bounds because some of them go lower than the optimal D_f value shown in the table on the left. I would suggest clearly indicating them as bounds by calling them \hat{KL} rather than KL, \hat{JS} rather than JS, etc.  4)  In the table on the right, some values are lower than their minimum value reported on the left. For example, training and testing with KL-rev yields .2414, but in the table on the left its optimal value is reported as .2415. The same thing also happens with JS, with a larger discrepancy. Initially I thought this was an error in the table, but then I realized it is possible I misinterpreted what was being reported for F in the table on the left.   Full disclosure: I've seen this paper on ArXiv, before I was assigned to review it. I didn't know the authors previously, and don't think that knowledge of who wrote it has influenced my ratings.


Learning Influence Functions from Incomplete Observations
We study the problem of learning influence functions under incomplete observations of node activations. Incomplete observations are a major concern as most (online and real-world) social networks are not fully observable. We establish both proper and improper PAC learnability of influence functions under randomly missing observations. Proper PAC learnability under the Discrete-Time Linear Threshold (DLT) and Discrete-Time Independent Cascade (DIC) models is established by reducing incomplete observations to complete observations in a modified graph. Our improper PAC learnability result applies for the DLT and DIC models as well as the Continuous-Time Independent Cascade (CIC) model. It is based on a parametrization in terms of reachability features, and also gives rise to an efficient and practical heuristic. Experiments on synthetic and real-world datasets demonstrate the ability of our method to compensate even for a fairly large fraction of missing observations.
1 r. Random observation loss naturally occurs when crawling data from social media, where rate restrictions are likely to affect all observations equally.
We establish both proper and improper PAC learnability of influence functions under incomplete observations for two popular diffusion models: the Discrete-Time Independent Cascade (DIC) and Discrete-Time Linear Threshold (DLT) models. In fact, randomly missing observations do not even significantly increase the required sample complexity. The result is proved by interpreting the incomplete observations as complete observations in a transformed graph, The PAC learnability result implies good sample complexity bounds for the DIC and DLT models. However, the PAC learnability result does not lead to an efficient algorithm, as it involves marginalizing a large number of hidden variables (one for each node not observed to be active).
Towards designing more practical algorithms and obtaining learnability under a broader class of diffusion models, we pursue improper learning approaches. Concretely, we use the parameterization of Du et al. [3] in terms of reachability basis functions, and optimize a modified loss function suggested by Natarajan et al. [17] to address incomplete observations. We prove that the algorithm ensures improper PAC learning for the DIC, DLT and Continuous-Time Independent Cascade (CIC) models. Experimental results on synthetic cascades generated from these diffusion models and real-world cascades in the MemeTracker dataset demonstrate the effectiveness of our approach. Our algorithm achieves nearly a 20% reduction in estimation error compared to the best baseline methods on the MemeTracker dataset.
Several recent works also aim to address the issue of missing observations in social network analysis, but with different emphases. For example, Chierichetti et al. [2] and Sadikov et al. [21] mainly focus on recovering the size of a diffusion process, while our task is to learn the influence functions from several incomplete cascades. Myers et al. [15] mainly aim to model unobserved external influence in diffusion. Duong et al. [6] examine learning diffusion models with missing links from complete observations, while we learn influence functions from incomplete cascades with missing activations. Most related to our work are papers by Wu et al. [23] and simultaneous work by Lokhov [13]. Both study the problem of network inference under incomplete observations. Lokhov proposes a dynamic message passing approach to marginalize all the missing activations, in order to infer diffusion model parameters using maximum likelihood estimation, while Wu et al. develop an EM algorithm. Notice that the goal of learning the model parameters differs from our goal of learning the influence functions directly. Both [13] and [23] provide empirical evaluation, but do not provide theoretical guarantees.

The authors consider the problem of PAC-learning the influence function in cascade models under a incomplete observation model. For each observed cascade, only the initial seeds and random subset of final active nodes are observed---specifically, each active node is observed as being active independently with probability 1-mu. The specific learning goal is to estimate the marginal probabilities of each node being activated given any source set S.  The paper considers both proper PAC learning (the estimated influence function corresponds to an actual diffusion model) and improper PAC learning (the estimated influence function is parameterized differently). For proper learning, they extend results of [14] to the case of incomplete observations. The main idea is to reduce (by a fairly straightforward reduction) the problem of learning with incomplete cascades to the problem of learning with complete observations in a modified graph (and with modified training cascades). They show that influence functions are PAC-learnable with a sample complexity that increases by a factor of 1/(1-mu)^4 over that of learning with complete cascades.   However, proper PAC learning is computationally intractable, so they instead consider improper learning.  For improper learning, they extend the results of [2] for "model-free" learning of influence functions to the case of incomplete observations. In particular, the influence function is estimated by approximating the reverse reachable sets of each node v as a mixture of sets sampled from some distribution q that approximates the true one. The results are parameterized in terms of how good an approximation q is. The sample complexity increases by a factor of 1/(1-mu)^2 compared with learning with complete cascades. They also improve the analysis of [2] to show a better dependence (logarithmic instead of polynomial) on the parameter C that quantifies how good an approximation q is.   Experiments compare with a number of different baselines to show that their model-free method performs much better with an increasing fraction of missing observations.
In general, the paper is extremely well written and technically sound.  In terms of significance and impact, I appreciate the general line of work of directly estimating influence functions from pairs of seed and activation sets. This seems a lot more practically applicable than related problems of estimating true parameters or recovering the edges of an unobserved graph because it can be applied directly in a predictive setting.   The specific high-level problem tackled by this paper---of estimating influence functions when some fraction of observations are missing---is relevant but not obviously a problem of huge impact. However, the problem is addressed very nicely. Two previous results are extended to the new observation model (refs [2,14]), the latter using another relatively recent result about learning with noisy labels (ref [15]). Each of the two major results in this paper required careful adaptation (and not just reuse) of the previous results.   The experiments were thorough and showed improvements compared with a number of baselines.  A limitation of the paper are the assumptions that: (1) the missing probability mu is known and, (2) observations are missing at random. While this is a reasonable place to start, the paper would be stronger if the authors acknowledge and discuss these limitations (e.g. lines 39-41 argue in favor of the assumption but do not acknowledge limitations).  I also suggest that the tradeoffs in Section 4 be clarified. It is unclear leading up to Theorem 4 where the distribution q for generating features comes from, and whether the final choice (lines 264-265: "approximate \beta_T^* with the product of marginal distributions") carries any theoretical guarantee. The footnote suggests that C may be exponential in the number of nodes, but Theorem 4 asserts that on the order of C^2 features are required for the theoretical guarantee. How big is C in practice? Is it known? A better discussion of these issues is warranted.  Minor comments:  I believe these two references are also relevant. [1] estimates diffusion parameters with missing data. [2] recovers the edges of a graph from activation-only cascades.  [1] Parameter Learning for Latent Network Diffusion. Wu, Kumar, Sheldon, Zilberstein. IJCAI 2013.   [2] Learning form contagion (without timestamps). Amin, Heidari, Kearns, ICML 2014.   In Section 2.1, clarify whether G and {w_{uv}} are known or unknown.  In Theorems 1 and 2, it is somewhat confusing that the dependence of the sample complexity on delta is suppressed by the \tilde{O} notation. From the previous discussion, I expected to see delta in these expressions.  The assertion in Lines 180-181 seems redundant with the previous equation. Isn't this case (non-seed nodes) already included in the statement of Eq. (1)?  Line 195: I suggest saying the "computing the objective function" and not "writing down the objective function" is hard.  For the experiments, one other baseline to consider is a naive adjustment to the predictions of the parametric models (CIC, DIC, etc.) that multiplies the predicted influence by 1/(1-mu). This would correct for missingness in the prediction phase but not the training phase. Was this already considered?


Optimal Cluster Recovery in the Labeled Stochastic Block Model
We consider the problem of community detection or clustering in the labeled Stochastic Block Model (LSBM) with a finite number K of clusters of sizes linearly growing with the global population of items n. Every pair of items is labeled independently at random, and label appears with probability p(i, j, ) between two items in clusters indexed by i and j, respectively. The objective is to reconstruct the clusters from the observation of these random labels. Clustering under the SBM and their extensions has attracted much attention recently. Most existing work aimed at characterizing the set of parameters such that it is possible to infer clusters either positively correlated with the true clusters, or with a vanishing proportion of misclassified items, or exactly matching the true clusters. We find the set of parameters such that there exists a clustering algorithm with at most s misclassified items in average under the general LSBM and for any s = o(n), which solves one open problem raised in [2]. We further develop an algorithm, based on simple spectral methods, that achieves this fundamental performance limit within O(npolylog(n)) computations and without the a-priori knowledge of the model parameters.

This paper explores the fundamental recovery limits of community recovery in a generalized version of the stochastic block model, where each community has size linear in n.  The measurements between each pair of vertices can be one of L labels, which allows to model the SBM, the planted clique problem, and many others using a unified model.  The focus is exact recovery or strong recovery (for which one expects the portion of misclassified nodes to be asymptotically zero).   The theoretical results are quite strong, which generalize many of the prior works along this direction.  The model is also of a lot of interest to multiple communities, so the contribution is timely and important.  I think this is worth being accepted to NIPS.    Despite the strong theory,  I am not completely satisfied by the way this paper is written.  There are many key assumptions / metrics that have not been given enough interpretation,  so it is hard for non-experts to understand what these results / conditions really mean.  The main algorithms are not described in the main text, and there are no simulations carried out (so it's hard to see how much practical relevance the theory bears).  I hope that the authors can consider my comments when revising their paper.
 1.  The key conditions (A1) and (A2) need more interpretation. How critical are these assumptions?  Are they coming from computational considerations or statistical considerations? Will the fundamental information limits change without these two assumptions?    2.  The authors start by introducing "labels" in the SBMs but defer all motivations / applications to much later.  I think the paper would read better if the authors at least describe some concrete applications right after (or before) describing the mathematical setup. This would be particularly beneficial for those who are non-experts in this field.   3.  Some interpretation of the divergence metric D_{L+} is missing, as this is the most important metric that dictates the recovery limits.  This metric is not easy to parse especially for those who have not read [2,3].  It might be nice to give some simple examples to interpret it.  4.  The achievability part (i.e. Theorem 2) operates upon another assumption (A3), but I do think this additional assumption needs further interpretation as well.    5.  I understand that Algorithms 1 and 2 are deferred to the supplemental materials due to space limitation,  but I think these are very crucial for the paper and deserve a place in the main text (at least I think presenting the procedures is more helpful than elaborating the proof techniques in such a short conference paper).    6.  There is no simulation carried out in this paper,  so it is hard to see how well the derived information limits match practice (e.g. how large does n need to be in order for the practical phase transition to match the theoretical prediction).


Assortment Optimization Under the Mallows model
We consider the assortment optimization problem when customer preferences follow a mixture of Mallows distributions. The assortment optimization problem focuses on determining the revenue/profit maximizing subset of products from a large universe of products; it is an important decision that is commonly faced by retailers in determining what to offer their customers. There are two key challenges: (a) the Mallows distribution lacks a closed-form expression (and requires summing an exponential number of terms) to compute the choice probability and, hence, the expected revenue/profit per customer; and (b) finding the best subset may require an exhaustive search. Our key contributions are an efficiently computable closed-form expression for the choice probability under the Mallows model and a compact mixed integer linear program (MIP) formulation for the assortment problem.

This paper studies an assortment optimization problem under the Mallow distribution model. It first offers a closed form expression for computing the choice probability of an item among a certain assortment that relies on discrete convolution. The authors argue that this result implies that the probability can be obtained efficiently using a Fourier transform method. On the other hand, they reject this closed form expression in their attempt to solve the assortment optimization where they rely on a recursive computation method to formulate a mixed integer linear programming problem formulation. With this formulation, they show that they have the capacity of solving problems with 20 items.
In my opinion, the authors do not provide convincing enough arguments that computing choice probabilities is as hard as they let it appear. The main reference that is stated relates to the difficulty of « computing the probability of a general parital order » which is not what is achieved by the methods that are proposed here. Instead, based on the repeated insertion procedure that is described in this paper (and obtained from Doignon et al. 2004) it appears that this probability can be efficiently computed using Monte-Carlo simulations (in fact, such simulations could even be used to construct a sample average approximation of problem (2)). I agree with the authors that the two methods that they propose are more efficient but have difficulty estimating the size of the contribution based on the text. Perhaps some information about sample average statistical convergence or some numerical evaluation of the methods would provide such an argument.  Regarding the second contribution, while I found the derivation of the MILP cleverly done. It fells short of convincing the reader that the model as real practical appeal. Indeed, in practice one would suspect that assortment planning have to handle numbers of items that are 10 to 100 times bigger than what the authors were able to achieve. One would also suspect that solutions to the assortment problem must be made almost instantaneously as users are visiting a web site and information are gathered about them.  Overall, I felt that there is a disconnect between section 3 and the rest of the paper that distracts the message that is made. Moreover, I believe that the MILP formulation has value but further efforts should be deployed in order to improve numerical performances if the goal is to have a practical impact with it. I would also encourage the author to identify a journal or conference that is more strongly oriented towards operations research or operations management.  Finally, here are some minor things that could be corrected : - Page 2, line 65 : « to to efficiently » should be « to efficiently » - Page 4, line 161 : « choosing a_k \in a_{i_l,j_l} » you should also define « l » here - Page 6, line 211 : « at position s » should sat « and at position s » - Page 7, line 228 : I don’t see how you can assume that a_1 \in S here given that a_1 is the preferred item in U. Please explain how the algorithm needs to change when it is not the case. Do we simply discard all items from U that are preferred to all items in S. - Page 8, line 280 : By « add cutting planes », I assume you mean « add valid inequalities » so that the LP relaxation is tighter. Or do you propose employing a decomposition scheme : e.g. master problem only holds the x_i’s while the slave problem takes care of the z’s and y’s


Local Similarity-Aware Deep Feature Embedding
Existing deep embedding methods in vision tasks are capable of learning a compact Euclidean space from images, where Euclidean distances correspond to a similarity metric. To make learning more effective and efficient, hard sample mining is usually employed, with samples identified through computing the Euclidean feature distance. However, the global Euclidean distance cannot faithfully characterize the true feature similarity in a complex visual feature space, where the intraclass distance in a high-density region may be larger than the interclass distance in low-density regions. In this paper, we introduce a Position-Dependent Deep Metric (PDDM) unit, which is capable of learning a similarity metric adaptive to local feature structure. The metric can be used to select genuinely hard samples in a local neighborhood to guide the deep embedding learning in an online and robust manner. The new layer is appealing in that it is pluggable to any convolutional networks and is trained end-to-end. Our local similarity-aware feature embedding not only demonstrates faster convergence and boosted performance on two complex image retrieval datasets, its large margin nature also leads to superior generalization results under the large and open set scenarios of transfer learning and zero-shot learning on ImageNet 2010 and ImageNet-10K datasets.

The authors propose to capture local feature similarity metric using PDDM, to tackle the issues brought by global distance metrics. They also introduce the large-margin double-header hinge loss to jointly optimize the similarity metric learning. They show good performance on image retrieval and transfer learning tasks to demonstrate the efficacy of their method.
The paper is well written with clear logic and well-designed figures, and shows promising results in both image retrieval and transfer learning tasks. The authors demonstrate clearly the effectiveness of combining distance metric with local similarity achieved by PDDM mechanism. The contribution of PDDM mechanism shows a novel local similarity metric which combines feature difference and absolute position information from [35], and presents surprising fast training convergence and better performance. The contribution of two double-header hinge losses in Eq (4) which combine local similarity metric and distance metric on the hard quadruplet has similar idea with [27], but uses less samples and has low computational cost and faster convergence.  Questions for authors: 1) For better understand and self-inclusive of the paper, there should be more formalized description of the comparative methods (i.e. contrastive embedding, triplet embedding and lifted structured embedding) to emphasize the improvement and advantage of the proposed method besides the comparison in Figure 3. 2) The authors should give more analysis or demonstration to explain the key reason of the effectiveness brought by the concatenation of feature difference vector and feature mean vector in the PDDM unit, which could be beneficial and instructive to readers. 3) The authors mention that the whole network, which contains 4 identical CNNs and PDDM units, is trained end-to-end. However, the hard quadruplet mining during a forward pass also needs 2 CNNs and PDDM units. The authors should add more information on the particular forward pass for easy understanding of the end-to-end architecture. 4) I want to confirm that you choose ONLY ONE hard quadruplet ($\hat{i}$, $\hat{j}$, $\hat{k}$, $\hat{l}$) for training per mini-batch (64 samples). If so, you have very few samples to fine-tune the CNN part and train the fully-connected layers of PDDM unit per epoch. I wonder if the samples are enough for training the network parameters on CUB-200-2011 dataset (5,864 training images). Please include more details about this.  There are a few typos which need to be amended: 1) Line 47: learn an local-adaptive -> learn a local-adaptive 2) Line 103: into an feature space -> into a feature space 3) Eq (2). $u_1$ and $v_1$ should be $u'$ and $v'$


Adaptive Skills Adaptive Partitions (ASAP)
We introduce the Adaptive Skills, Adaptive Partitions (ASAP) framework that (1) learns skills (i.e., temporally extended actions or options) as well as (2) where to apply them. We believe that both (1) and (2) are necessary for a truly general skill learning framework, which is a key building block needed to scale up to lifelong learning agents. The ASAP framework can also solve related new tasks simply by adapting where it applies its existing learned skills. We prove that ASAP converges to a local optimum under natural conditions. Finally, our experimental results, which include a RoboCup domain, demonstrate the ability of ASAP to learn where to reuse skills as well as solve multiple tasks with considerably less experience than solving each task from scratch.

This paper addresses the problem of learning temporally extended actions ("skills") as well as when to apply them in reinforcement learning. The main idea is to smoothly parameterize every component of the skill architecture (the state partition, the skill policies, and the assignment of partitions to skills) and to directly adapt those parameters using policy gradient. Experimental results show that the ASAP algorithm can effectively adapt its arbitrary initial skills to suit its environment.
***After Author Response***  I would only add that I hope the authors will take pains to clarify issues surrounding the number of partitions and policies if this paper is accepted. Several of the reviewers brought this up, and since the reviewers are at least somewhat representative of the paper's audience, it should be carefully addressed in the paper. I understand the authors to be saying that the algorithm is capable of making some partitions vanishingly small, and is capable of making some options vanishingly unlikely to be executed. In that sense, yes, ASAP can effectively learn how many partitions/policies are needed. However, I think the authors are being a little bit disingenuous about this point. For instance, ASAP can't create *more* partitions if they turn out to be necessary, which is I think what people mean when they say something like "how do you pick the correct number?". Also, even if ASAP might eventually find a structure that effectively uses only a few policies, on the way to get there (and actually even once it is there) it has to maintain and adapt the parameters for exponentially many policies. So that would suggest that it is not safe to just throw a whole bunch of partitions at it and let it sort it out. It seems that one does need to carefully consider how many partitions to start with -- not too few, not too many. So one of two things is happening. If, as I suspect, this is a genuine limitation of this approach, it's important that the authors straightforwardly and clearly acknowledge it as a limitation; the results are interesting anyway, and it points the way to remaining open questions. If basically all of the reviewers have misunderstood something and this is not a practical limitation, then there must be a clarity issue in the paper which must be addressed.  ---Technical Quality---  As far as I can tell the paper is technically sound. The derivations of the gradients are wholly contained within the supplemental material, and I did not pore over every detail, but they seemed correct and reasonable. The experiments are well-constructed and do effectively support the main claims.  I will say that the claim that ASAP re-uses skills seemed a bit weaker than the others. In "Three Rooms" the system apparently learned to apply the same skill in two different partitions, which is cool, but in the "Flipped Two Rooms" domain, the system apparently kept the hyperplane while re-learning the same two skills (just in the opposite regions), which is less cool. I think it's fair to say that the experiments demonstrate that skills *can* be re-used, but they also seem to show that this is highly dependent on the parameterization and search procedure. In other words, this framework permits skill re-use, but does nothing in particular to encourage/facilitate it.   ---Novelty---  The most similar work I am aware of is from Bacon and Precup (which the paper cites). They similarly parameterize the option policies, option termination conditions, and overall policy so all components can be adapted via policy gradient. This paper deals directly with continuous state spaces (defining state partitions using hyperplanes), which Bacon and Precup did not. This would not seem like sufficient novelty on its own, however the Bacon and Precup paper appeared in a workshop proceedings, not in an archival publication. Other than Bacon & Precup, as far as I am aware this approach of directly the entire skill architecture via PG is novel (and interesting).  ---Impact---  I believe this paper has the potential for substantial impact. Certainly the problems of learning and applying skills is an important one. This paper's approach of folding all of the relevant architectural decisions into one learning process is intriguing and shows demonstrated promise. This particular algorithm definitely has weaknesses; the exponental number of skill policies to learn is probably chief among them. However, the formulation of skill learning as purely a highly structured parameterization of the policy is a core idea that I can easily see being further explored in follow-up work. Furthermore, the experiments show some emergent desirable properties of this relatively straightforward approach (e.g. skill re-use, learning that fewer partitions are necessary) that immediately suggest follow-up research into how to more systematically encourage these behaviors.  ---Clarity---  I found the paper to be very well written. I thought the approach was well-motivated and clearly described, as were the presented results.  I felt that leaving the derivations for the supplementary appendix was sensible, since these are mostly algebra, and the derivation techniques applied are not really critical to the contribution. Even though some of the experimental results are left to the appendix, I felt that the authors did a good job of describing the results, and selected appropriate figures that could be left out without too much harm to the reader's flow. I will say that I personally don't like the format of making the supplementary materials simply a longer version of the paper. As a reader, it makes it harder to flip to the appendix to see what I want to see. Also, it is confusing that the supplementary version still refers to the supplementary materials!  A minor question: Why is Masson & Konidaris (2015) in Table 1? It is nowhere discussed in the paper, and, while I can see the relevance, it does not seem to be directly addressing the issue of skill learning in the same way as the others.  ---Summary of Revew---  This is a novel approach to skill learning that is intruiging in its directness and that shows significant promise. While there are some drawbacks to the specific algorithm presented here, the core idea is flexible and seems likely to inspire further investigation.