Bayesian Optimization with Gradients
Bayesian optimization has been successful at global optimization of expensiveto-evaluate multimodal objective functions. However, unlike most optimization methods, Bayesian optimization typically does not use derivative information. In this paper we show how Bayesian optimization can exploit derivative information to find good solutions with fewer objective function evaluations. In particular, we develop a novel Bayesian optimization algorithm, the derivative-enabled knowledgegradient (d-KG), which is one-step Bayes-optimal, asymptotically consistent, and provides greater one-step value of information than in the derivative-free setting. d-KG accommodates noisy and incomplete derivative information, comes in both sequential and batch forms, and can optionally reduce the computational cost of inference through automatically selected retention of a single directional derivative. We also compute the d-KG acquisition function and its gradient using a novel fast discretization-free technique. We show d-KG provides state-of-the-art performance compared to a wide range of optimization procedures with and without gradients, on benchmarks including logistic regression, deep learning, kernel learning, and k-nearest neighbors.

Major:
To me this paper is an important one as it presents substantial contributions to Bayesian Optimization that are at the same time elegant and practically efficient. While the paper is well written, it could be formally improved in places; below are
a few comments questions that might be useful toward such improvements. Besides this, I was a bit puzzled by the fact that Theorem 1 is not very precisely stated; also, given the fact that it is proven in appendix (which is understandable given
the space limitations) and with a proof that requires substantial time to proofread formally (as it is almost a second paper in itself, with a specific policy formalism/set-up), I had to adapt my confidence score accordingly. Also, I was wondering if the consistency proof could work in the case of noiseless observations, where it does not make sense to replicate evaluations at the same point(s)?
Nevertheless, from the methodological contributions and their practical relevance, I am conviced that the presented approach should be spread to BO/ML researchers and that NIPS is an excellent platform to do so.
Minor:
* In the abstract: "most optimization methods" is a bit strong. Derivative-free optimization methods should not be forgotten.
Yet, gradients are indeed essential in (differentiable) local optimization methods.
* In the abstract: syntax issue in "We show d-KG provides".
* Introduction: from "By contrast", see comment above regarding *local* optimization.
* Introduction: "relative value" does not sound very clear to me. Maybe speak of "variations" or so?
* Introduction: "we explore the what, when, and why". In a way yes, but the main emphasis is on d-KG, and even if generalities
and other approaches are tackled, it is probably a bit strong to claim/imply that *Bayesian Optimization with Gradients* is
*explored* in a paper subjected to such space restrictions.
* Introduction: from "And even" to "finite differences"; in such case, why not use this credit to do more criterion-driven
evaluations? Are there good reasons to believe that two neigbouring points approximating a derivative will be better than
two (more) distant well-chosen points? [NB: could there exist a connection to "twin points", in Gaussian kernel settings?]
* Section 3.1: "to find argmin"; so, the set [in case of several minimizers]?
* Section 3.1: "linear operator" calls for additional regularity condition(s). By the way, it would be nice to ensure
differentiability of the GP via the chosen mu and K.
* Section 3.1: the notation used in equation (3.2) seem to imply that there cannot be multiple evaluations at the same x.
Notations in terms of y_i (and similar for the gradient), corresponding to points x_i, could accomodate that.
* Section 3.1: the paragraph from line 121 to 126 sounds a bit confusing to me (I found the writing rather cryptic there)
* Section 3.2: "expected loss"; one would expect a translation by the actual min there (so that loss=0 for successful optim)
* Section 3.2: in equation (3.3), conditioning on "z" is abusive. By the way, why this "z" notation (and z^(i)) for "x" points?
* Algorithm 2, lines 2: a) =argmax abusive and b) is theta on the unit sphere? restricted to canonical directions?
* Section 3.3: in the expression of the d-KG factor, the term \hat{\sigma}^{(n)} appears to be related to (a generalization of)
kriging update formulas for batch-sequential data assimilation. Maybe it could make sense to connect one result ro the other?
* Section 3.3: the approach with the enveloppe theorem is really nice. However, how are the x^*(W) estimated? Detail needed...
* Section 3.3: "K samples"; from which distribution? + Be careful, the letter K was already used before, for the covariance.
* Section 3.3, line 191: "conditioning" does not seem right here...conditional? conditionally?
* Section 3.4: in Prop.1, you mean for all z^{(1:q)}? Globally the propositions lack mathematical precision...
* Section 3.4: ...in particular, what are the assumptions in Theorem 1? E.g., Is f assumed to be drawm from the running GP?
* Section 4: notation Sigma is used for the covariance kernel while K was used before.
* Section 4: about "the number of function evaluations"; do the figures account for the fact that "d-approaches" do several f
evaluations at each iteration?
* Section 5: about "low dimensional"; not necessarily...See REMBO and also recent attempts to combine BO and kernel (sparse) decompositions.


Fast, Sample-Efficient Algorithms for Structured Phase Retrieval
We consider the problem of recovering a signal x * ∈ R n , from magnitude-only measurements, y i = | a i , x * | for i = {1, 2, . . . , m}. Also known as the phase retrieval problem, it is a fundamental challenge in nano-, bio-and astronomical imaging systems, and speech processing. The problem is ill-posed, and therefore additional assumptions on the signal and/or the measurements are necessary. In this paper, we first study the case where the underlying signal x * is s-sparse. We develop a novel recovery algorithm that we call Compressive Phase Retrieval with Alternating Minimization, or CoPRAM. Our algorithm is simple and can be obtained via a natural combination of the classical alternating minimization approach for phase retrieval, with the CoSaMP algorithm for sparse recovery. Despite its simplicity, we prove that our algorithm achieves a sample complexity of O s 2 log n with Gaussian samples, which matches the best known existing results. It also demonstrates linear convergence in theory and practice and requires no extra tuning parameters other than the signal sparsity level s. We then consider the case where the underlying signal x * arises from structured sparsity models. We specifically examine the case of block-sparse signals with uniform block size of b and block sparsity k = s/b. For this problem, we design a recovery algorithm that we call Block CoPRAM that further reduces the sample complexity to O (ks log n). For sufficiently large block lengths of b = Θ(s), this bound equates to O (s log n). To our knowledge, this constitutes the first end-toend linearly convergent family of algorithms for phase retrieval where the Gaussian sample complexity has a sub-quadratic dependence on the sparsity level of the signal.

The authors' rebuttal contains some points which should be explicitly included in a revised version if the paper is accepted.
-----------
The authors study the problem of compressive (or sparse) phase retrieval, in
which a sparse signal x\in\R^n is to be recovered from measurements abs(Ax),
where A is a design (measurement) matrix, and abs() takes the entry-wise
absolute value. The authors propose an iterative algorithm that applies CoSaMP
in each iteration, and prove convergence to the correct, unknown sparse vector x
with high probability, assuming that the number of measurements is at least
C*s*log(n/s), where s is the sparsity of x and n is the ambient dimension.
The paper is well written and clear, and the result is novel as far as I can
tell. However, the problem under study is not adequately motivated. The authors
only consider real-valued signals throughout the paper, yet bring motivation
from real phase retrieval problems in science in which the signals are, without
exception, complex. The authors aim to continue the work in refs. [21,22] which
also considered only real-valued signals. Looking at those papers, I could find
no satisfying motivation to study the problem as formulated.
The algorithm proposed is compared with those proposed in refs. [21,22].
According to Table 1, it enjoys the same sample complexity requirement and the
same running time (up to constants) as SPARTA [22] and TWF [21]. (While the
sample complexity requirement is proved in the paper - Theorem 4.2 - I could not
find a proof of the running time claimed in Table 1). As far as I could see, the only
improvements of the proposed algorithm over prior art is the empirical
performance (Section 6).
In addition, the authors study a closely related problem, where the signal is
assumed to be block sparse, and develop a version of the algorithm adapted to
this problem.
As for the sparse (non-block) version, in my opinion the limited simulation
study offered by the authors is not sufficient to establish that the proposed
algorithm improves in any way over those of [21,22]. Without a careful empirical
study, or alternatively a formal results showing that the proposed algorithm
improves over state of the art, the merit of the proposed algorithm is not
sufficiently established.
As for the block version, it seems that here the algorithm, which is
specifically tailored to the block version, improves over state of the art.
However, in my opinion the problem of recovery of *real* block-sparse signals
from absolute value of linear measurements is not sufficiently motivated.
Other comments:
The authors assume a Gaussian measurement matrix. The main results (Theorem 4.1,
Theorem 4.2, Theorem 5.2) must explicitly specify this assumption. As written
now, the reader may incorrectly infer that the Theorem makes no assumptions on
the measurement matrix.


Countering Feedback Delays in Multi-Agent Learning
We consider a model of game-theoretic learning based on online mirror descent (OMD) with asynchronous and delayed feedback information. Instead of focusing on specific games, we consider a broad class of continuous games defined by the general equilibrium stability notion, which we call λ-variational stability. Our first contribution is that, in this class of games, the actual sequence of play induced by OMD-based learning converges to Nash equilibria provided that the feedback delays faced by the players are synchronous and bounded. Subsequently, to tackle fully decentralized, asynchronous environments with (possibly) unbounded delays between actions and feedback, we propose a variant of OMD which we call delayed mirror descent (DMD), and which relies on the repeated leveraging of past information. With this modification, the algorithm converges to Nash equilibria with no feedback synchronicity assumptions and even when the delays grow superlinearly relative to the horizon of play.

If we accept that distributed learning is interesting, then this article presents a nice treatment of distributed mirror descent in which feedback may be asynchronous and delayed. Indeed, we are presented with a provably convergent learning algorithm for continuous action sets (in classes of games) even when individual players' feedback are received with differing levels of delay; further more the regret at time T is controlled as a function of the total delay to time T.  This is a strong result, achieved by using a suite of very current proof techniques - lambda-Fenchel couplings serving as primula-dual Bregman divergences and associated tools.
I have some concerns, but overall I think this is a good paper.
- (very minor) In the first para of Section 2.2, "following learning scheme" actually refers to Algorithm 1, over the page.
- Lemma 3.2. If the concept of variational stability implies that all Nash equilibria of a game are in a closed and convex set, to me this is a major restriction on the class of games for which the result is relevant. Two example classes are given in the supplementary material, but I think the main paper should be more upfront about what kinds of games we expect the results to hold in.
- At the bottom of page 5, we are presented with an assumption (buried in the middle of a paragraph) which is in some sense the converse to Lemma 4.2. While the paper claims the assumption is very weak, I would still prefer it to be made more explicitly, and some more efforts made to explain why it's weak, and what might make it fail
- In Algorithm 2, the division by |G^t_i| is very natural. Why is something similar not done in Algorithm 1?
- (Minor) Why do we talk about "Last iterate convergence"?  This is a term I'm unfamiliar with. I'm more used to "convergence of intended play" or "convergence of actual play".
- References 15 and 16 are, I think, repeats?
- You should probably be referencing recent works e.g. [Mertikopoulos] and [Bervoets, S., Bravo, M., and Faure, M]


Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure
Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions. Unfortunately, these techniques are unable to deal with stochastic perturbations of input data, induced for example by data augmentation. In such cases, the objective is no longer a finite sum, and the main candidate for optimization is the stochastic gradient descent method (SGD). In this paper, we introduce a variance reduction approach for these settings when the objective is composite and strongly convex. The convergence rate outperforms SGD with a typically much smaller constant factor, which depends on the variance of gradient estimates only due to perturbations on a single example.

Summary of the paper
====================
The paper considers the setting of finite-sum problems where each individual function undergoes a random pertuerbation. The authors devise an adapation of the finite-sums algorithm MISO\Finito, called S-MISO, for this setting, and provide a convergence analysis based on the noise (and the analytic) parameters of the model, along with supporting experiments on various datasets from different domains.
Evaluation
==========
The algorithm devised by the authors seems to effectively exploit the extra structure offered by this setting (in comparison to the generic stochastic optimization setting) - both theoretically where S-MISO is proven to improve upon SGD in a factor dependent on the overall variance and the average individual variance, and empirically, where S-MISO is shown to outperform SGD (designed for generic stochastic problems) and N-SAGA (designed for related settings). This setting subsumes important applications where data augmentation\perturbation have been proven successful (particularly, image and text classification), rendering it meaningful and interesting.
On the flip side, the convergence analysis is somewhat loosely stated (e.g., the use of epsilon bar) and there are highly non-trivial user-specified parameters which have to be determined and carefully tuned. Moreover, although the expected performance on multi-layer network are considered in the body of the paper, they are not addressed appropriately in Section 4 where supporting experiments are presented (only 2-layers networks) - I find this very disappointing as this one of the main promises of the paper. Lastly, the paper is relatively easy-to-follow.
Comments
========
L8 - As convergence rates are usually stated using O-notation, it is not utterly clear how to inrepret 'smaller constant factor'.
L17 - What does 'simple' stand for? proximal-friendly?
L21 - Can you provide pointers\cross-reference to support 'fundamental'? Also, consider rephrasing this sentence.
L47 - Consider repharsing sentence.
L65 - Table 2: Data radius is not seemed to be defined anywhere in the text.
L66 - How does concentrating on a quantity that depends only on the minimizer affect the asymptotic properties of the stated convergence analysis?
L69 - 'note that\nabla' is missing a space.
L77 - In what sense is this rate 'unimprovable'?
L78 - This pararaph seems somewhat out of place. Maybe put under 'realted work'?
L90 - This paragrpah too seems somewhat out of place (seems to be realted to section 4).
L116+L121 -  the wording 'when f_i is an expectation' is somewhat confusing.
L183+L185 - The use of epsilon bar is not conventional and makes it hard to parse this statement. Please consider restating the convergence analysis in terms of the pre-defined accuracy level (alterntaively, consider providing more explenatation of the upper bound parameters).
L192 - Can you elaborate more on concrete choices for gamma?
L198 - 'as large as allowed' in terms of EQ 9?
L200 - Figure 1: can you elaborate more on 'satisfies the thoery'?
L208 - some expremients seem to be 400-epochs long.
L226 - Here and after, numbers are missing thousands separator.
L261 - Comparing could be more fair if a non-uniform version of SGD would be used.
L271 - Line seems out of place.


Efficient Approximation Algorithms for String Kernel Based Sequence Classification
Sequence classification algorithms, such as SVM, require a definition of distance (similarity) measure between two sequences. A commonly used notion of similarity is the number of matches between k-mers (k-length subsequences) in the two sequences. Extending this definition, by considering two k-mers to match if their distance is at most m, yields better classification performance. This, however, makes the problem computationally much more complex. Known algorithms to compute this similarity have computational complexity that render them applicable only for small values of k and m. In this work, we develop novel techniques to efficiently and accurately estimate the pairwise similarity score, which enables us to use much larger values of k and m, and get higher predictive accuracy. This opens up a broad avenue of applying this classification approach to audio, images, and text sequences. Our algorithm achieves excellent approximation performance with theoretical guarantees. In the process we solve an open combinatorial problem, which was posed as a major hindrance to the scalability of existing solutions. We give analytical bounds on quality and runtime of our algorithm and report its empirical performance on real world biological and music sequences datasets.
acids) and predicting proteins fold (functional three dimensional structure) are essential tasks in bioinformatics. Sequence classification algorithms have been applied to both of these problems with great success [3,10,13,18,19,20,25]. Music data, a real valued signal when discretized using vector quantization of MFCC features is another flavor of sequential data [26]. Sequence classification has been used for recognizing genres of music sequences with no annotation and identifying artists from albums [12,13,14]. Text documents can also be considered as sequences of words from a language lexicon. Categorizing texts into classes based on their topics is another application domain of sequence classification [11,15].
While general purpose classification methods may be applicable to sequence classification, huge lengths of sequences, large alphabet sizes, and large scale datasets prove to be rather challenging for such techniques. Furthermore, we cannot directly apply classification algorithms devised for vectors in metric spaces because in almost all practical scenarios sequences have varying lengths unless some mapping is done beforehand. In one of the more successful approaches, the variable-length sequences are represented as fixed dimensional feature vectors. A feature vector typically is the spectra (counts) of all k-length substrings (k-mers) present exactly [18] or inexactly (with up to m mismatches) [19] within a sequence. A kernel function is then defined that takes as input a pair of feature vectors and returns a real-valued similarity score between the pair (typically inner-product of the respective spectra's). The matrix of pairwise similarity scores (the kernel matrix) thus computed is used as input to a standard support vector machine (SVM) [5,27] classifier resulting in excellent classification performance in many applications [19]. In this setting k (the length of substrings used as bases of feature map) and m (the mismatch parameter) are independent variables directly related to classification accuracy and time complexity of the algorithm. It has been established that using larger values of k and m improve classification performance [11,13]. On the other hand, the runtime of kernel computation by the efficient trie-based algorithm [19,24] is O(k m+1 |Σ| m (|X| + |Y |)) for two sequences X and Y over alphabet Σ.
Computation of mismatch kernel between two sequences X and Y reduces to the following two problems. i) Given two k-mers α and β that are at Hamming distance d from each other, determine the size of intersection of m-mismatch neighborhoods of α and β (k-mers that are at distance at most m from both of them). ii) For 0 ≤ d ≤ min{2m, k} determine the number of pairs of k-mers (α, β) ∈ X × Y such that Hamming distance between α and β is d. In the best known algorithm [13] the former problem is addressed by precomputing the intersection size in constant time for m ≤ 2 only. While a sorting and enumeration based technique is proposed for the latter problem that has computational complexity O(2 k (|X| + |Y |), which makes it applicable for moderately large values of k (of course limited to m ≤ 2 only).
In this paper, we completely resolve the combinatorial problem (problem i) for all values of m. We prove a closed form expression for the size of intersection of m-mismatch neighborhoods that lets us precompute these values in O(m 3 ) time (independent of |Σ|, k, lengths and number of sequences). For the latter problem we devise an efficient approximation scheme inspired by the theory of locality sensitive hashing to accurately estimate the number of k-mer pairs between the two sequences that are at distance d. Combining the above two we design a polynomial time approximation algorithm for kernel computation. We provide probabilistic guarantees on the quality of our algorithm and analytical bounds on its runtime. Furthermore, we test our algorithm on several real world datasets with large values of k and m to demonstrate that we achieve excellent predictive performance. Note that string kernel based sequence classification was previously not feasible for this range of parameters.

The authors describe an approximation algorithm for k-mer (with mismatches) based string kernels.
The contribution is centered around a closed form expression of the intersection size of mismatching neighbourhoods.
The algorithm is evaluated in the context of sequence classification using SVM.
I think this is a great paper: clear motivation, good introduction, clear contribution, theoretical back-up, nice experimental results.
I have a few concerns regarding presentation and structuring, as well as doubts on relevance on the presented theory.
Presentation.
I think Section 3 is too technical. It contains a lot of notation, and a lot of clutter that actually hinder understanding the main ideas. On the other hand, intuition on crucial ideas is missing, so I suggest to move a few details into Appendix and rather elaborate high-level ideas.
-A table with all relevant quantities (and explanations) would make it much easier for a non-string-kernel person to follow establishing Theorem 3.3. An intuitive description of the theorem (and proof idea) would also help.
-Algorithm 1 is not really referenced from the main test. It first appears in Thm 3.10. It is also very sparsely annotated/explained. For example, the parameter meanings are not explained. Subroutines also need annotation. I think further deserves to be in the center of Section 3's text, and the text should describe the high level ideas.
-It should be pointed out more that Thm 3.3 is the open combinatoric problem mentioned in the abstract.
-In the interest of readability, I think the authors should comment more on what it is that their algorithm approximates, what the parameters that control the approximation quality are, and what would be possible failure cases.
-Theorem 3.13 should stay in the main text as this concentration inequality is the main theoretical message. All results establishing the Chebbyshev inequality (Thm 3.11, Lemma 3.12, Lemma 3.14 requirements should go to the appendix. Rather, the implications of the Theorem 3.13 should be elaborated on more. I appreciate that the authors note that these bounds are extremely loose.
Theory.
-Theorem 3.13 is a good first step to understanding the approximation quality (and indeed consistency) of the algorithm. It is, however, not useful in the sense that we do not care about the kernel approximation itself, but we care about the generalization error in the downstream algorithm. Kernel method theory has seen a substantial amount of work in that respect (e.g. for Nystrom, Fourier Feature regression/classification). This should be acknowledged, or even better: established. A simple approach would be perturbation analysis using the established kernel approximation error, better would be  directly controlling the error of the estimated decision boundary of the SVM.
Experiments.
-Runtime should be checked for various m
-Where does the approximation fail? An instructive synthetic example would help understanding when the algorithm is appropriate and when it is not.
-The authors mention that their algorithm allows for previously impossible settings of (k,m). In Table 3, however, there is only a single case where they demonstrate an improved performance as opposed to the exact algorithm (and the improvement is marginal). Either the authors need to exemplify the statement that their algorithm allows to solve previously unsolved problems (or allow for better accuracy), or they should remove it.
Minor.
-Figure 1 has no axis units
-Typo line 95 "theset"
-Typo line 204 "Generating these kernels days;"


Real Time Image Saliency for Black Box Classifiers
In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier. We train a masking model to manipulate the scores of the classifier by masking salient parts of the input image. Our model generalises well to unseen images and requires a single forward pass to perform saliency detection, therefore suitable for use in real-time systems. We test our approach on CIFAR-10 and ImageNet datasets and show that the produced saliency maps are easily interpretable, sharp, and free of artifacts. We suggest a new metric for saliency and test our method on the ImageNet object localisation task. We achieve results outperforming other weakly supervised methods.

The paper proposes an approach to learn saliency masks. The proposed approach is based on a neural network and can process multiple images per second (i.e. it is fast).
To me the paper is borderline, I would not object rejection or acceptance. I really believe in the concept of learning to explain a model and I think the paper has some good ideas.
There are no obvious mistakes but there are clear limitations.
-The saliency is only indirectly measured. Either through weakly supervised localisation or by the proposed saliency metric both these methods have clear limitations and I think these limitations should be discussed in the paper.
The weakly supervised location is not perfect as a measure. If the context in which an object appears is essential to determine its class, the object localisation does not have to correlate with saliency quality. The results on weakly supervised localisation are interesting, but I think there is a big caveat when using them as a quality metric for saliency.
The saliency metric is not perfect because of how it is applied. The estimated salient region is cropped. This crop is then rescaled to the original image size, with the original aspect ratio. This could introduce two artefacts. First, the change of the aspect ratio might impact how well it can be classified. Second in the proposed metric a small salient region is preferred. Since a small region is blown up heavily for re-classification the scale at which the object is now presented to the classifier might not be ideal. (Convnets are generally pretty translation invariant, but the scaling invariance must be learned, and there are  probably limits to this).
What is not discussed here is how much the masking model depends on the architecture for learning the masks? Did the authors at one point experiment with different architectures and how this influenced the result?
Minor comments
Are the results in Table 1 obtained for all classes or only for the correct class?
- Please specify with LRP variant and parameter setting was used for comparison. They have an epsilon, alpha-beta, and more variants with parameters.
*** Post rebuttal edit ***
The fact that  how well the saliency metric works depends on the quality and the scale invariance of the classifier is strongly limiting the applicability of the proposed method. It can only be applied to networks having this invariance.
This has important consequences:
-  The method cannot be used for models during the training phase, nor for models that do not exhibit this invariance.
- This limit the applicability to other domains (e.g. spectrogram analysis with CNN's).
- The method is not generally applicable to black-box classifiers as claimed in the title.
Furthermore, the response hints at a strong dependence on the masking network.
- As a result, it is not clear to me whether we are visualizing the saliency of the U-network or the masking network.
If these effects are properly discussed in the paper I think it is balanced enough for publication. If not it should not be published


Principles of Riemannian Geometry in Neural Networks
This study deals with neural networks in the sense of geometric transformations acting on the coordinate representation of the underlying data manifold which the data is sampled from. It forms part of an attempt to construct a formalized general theory of neural networks in the setting of Riemannian geometry. From this perspective, the following theoretical results are developed and proven for feedforward networks. First it is shown that residual neural networks arenite di erence approximations to dynamical systems of rst order di erential equations, as opposed to ordinary networks that are static. This implies that the network is learning systems of di erential equations governing the coordinate transformations that represent the data. Second it is shown that a closed form solution of the metric tensor on the underlying data manifold can be found by backpropagating the coordinate representations learned by the neural network itself. This is formulated in a formal abstract sense as a sequence of Lie group actions on the metric bre space in the principal and associated bundles on the data manifold. Toy experiments were run to con rm parts of the proposed theory, as well as to provide intuitions as to how neural networks operate on data.

The paper develops a mathematical framework for working with neural network representations in the context of finite differences and differential geometry.
In this framework, data points going though layers have fixed coordinates but space is smoothly curved with each layer.
The paper presents a very interesting framework for working with neural network representations, especially in the case of residual networks.
Unfortunately, taking the limit as the number of layers goes to infinity does not make practical application very easy and somewhat limits the impact of this paper.
The paper is not always completely clear. Since the goal of the paper is to present a minority perspective, clarity should be paramount.
experiments are a bit disapoiting. Their goal is not always very explicit. What the experiments do exactly is not very clear (despite their obvious simplicity). The experiments which show how a simple neural networks disentangle data points feel well known and their relation to the current paper feels a bit tenuous.
Line 62: The . in tensors (…) ensure consistency in the order of the superscrips and subscripts.
=> Not very clear. I assume the indices with a dot at the end are to be on the left of indices with a dot at the beginning (so that the dots would sort of align).
line 75, Eq 1:
Taking the limit as L -> /infty seems to pose a problem for practical applications. Wouldn’t an equation of the form bellow make sense (interpolation between layers):
x^a(x + \delta l) = x^a(l) + f^a(x^b(l); l) \delta l
instead of
x^a(x + 1 ) = x^a(l) + f^a(x^b(l); l) \delta l
6.1 & Figure1: The Harmonic oscillator
Section 6.1 and the corresponding figures are unclear.
=> The given differential equation does not indicate what the parameters are and what the coordinates are. This is a bit confusing since \xi is a common notation for coordinates in differential geometry. We are told that the problem is two dimensional so I assume that x is the two dimensional variable.
=> Figure 1 is confusing. What does the finite differencing plot show ? what do the x and y axis represent ? What about metric transformations ? What about the scalar metric values ?
The fact that the particle stands still is understandable if in the context of a network but confusing w.r.t. the given differential equation for x.
=> a, b = 1,2 is not clear in this context. Does this mean that a = 1 and b = 2 ? can the state space representation be written in the given way only if a, b = 1,2 ?
Figure 2: Every plot has the legend “layer 11”. Typo ?
section 6.3 is a bit superfluous.


Linearly constrained Gaussian processes
We consider a modification of the covariance function in Gaussian processes to correctly account for known linear operator constraints. By modeling the target function as a transformation of an underlying function, the constraints are explicitly incorporated in the model such that they are guaranteed to be fulfilled by any sample drawn or prediction made. We also propose a constructive procedure for designing the transformation operator and illustrate the result on both simulated and real-data examples.

Summary of the Paper:
This paper describes a mechanism to incorporate linear operator constraints in the framework of Gaussian process regression. For this, the mean function and the covariance function of the Gaussian processes are changed. The aim of this transformation is to guarantee that samples from the GP posterior distribution satisfy the constraints indicated. These constraints are typically in the form of partial derivatives, although any linear operator can be considered in practice, e.g., integration too. Traditional methods incorporated these constraints by introducing additional observations. This has the limitation that is more expensive and restricted to the observations made. The framework proposed is evaluated in a synthetic problem and in a real problem, showing benefits with respect to the data augmentation strategy.
Detailed comments:
Quality:
I think the quality of the paper is good in general. It is a very well written paper. Furthermore, all the key points are carefully described. It also has a strong related work section. The weakest point is however, the section on experiments in which only a synthetic dataset and a real dataset is considered.
Clarity:
The clarity of the paper is high.
Originality:
As far as I know the paper seems original. There are some related methods in the literature. However, they simply augment the observed data with virtual instances that have the goal of guaranteeing the constraints imposed.
Significance:
I think the problem addressed by the authors is relevant and important for the machine learning community. However, I have the feeling that the authors have not success in noting this. The examples used by the authors are a bit simple. For example they only consider a single real example and only the linear operator of derivatives. I have the feeling that this paper may have potential applications in probabilistic numeric methods, in which often a GP is used.
Summing up, I think that this is a good paper. However, the weak experimental section questions its significance. Furthermore, I find difficult to find practical applications within the machine learning community. The authors should have made a bigger effort on showing this. I would consider it hence a borderline paper.


Multi-Armed Bandits with Metric Movement Costs
We consider the non-stochastic Multi-Armed Bandit problem in a setting where there is a fixed and known metric on the action space that determines a cost for switching between any pair of actions. The loss of the online learner has two components: the first is the usual loss of the selected actions, and the second is an additional loss due to switching between actions. Our main contribution gives a tight characterization of the expected minimax regret in this setting, in terms of a complexity measure C of the underlying metric which depends on its covering numbers. In finite metric spaces with k actions, we give an efficient algorithm that achieves regret of the form O(max{C 1/3 T 2/3 , √ kT }), and show that this is the best possible. Our regret bound generalizes previous known regret bounds for some special cases: (i) the unit-switching cost regret Θ(max{k 1/3 T 2/3 , √ kT }) where C = Θ(k), and (ii) the interval metric with regret Θ(max{T 2/3 , √ kT }) where C = Θ(1). For infinite metrics spaces with Lipschitz loss functions, we derive a tight regret bound of Θ(T d+1 d+2 ) where d ≥ 1 is the Minkowski dimension of the space, which is known to be tight even when there are no switching costs.

The authors consider the setting of Multi-Armed Bandits with movement costs.
Basically, the set of arms is endowed with a metric and the player pays a price when she changes her action depending on the distance with the latest played action.
The main contribution of the paper is to generalize previous work to general metrics. They prove matching (up to log factors) upper and lower bounds for the problem and adapt the Slowly-Moving Bandit Algorithm to general metrics (it was previously designed for intervals only).
To my opinion the paper is pretty well written and pleasant to read. The literature is well cited. The analysis seem to be quite similar to the previous analysis of the SMB algorithm. The analysis and the algorithm are based on the same idea of hierarchical trees and the main ingredient of the proof seems to deal with non-binary trees. Though, I think it is a nice contribution.
Some remarks:
- your notion of complexity only deals with parametric spaces of arms. Do you think you can get results for non-parametric spaces, like Lipschitz functions?
- what is the complexity of the algorithm for instance if the space of arms is [0,1]^d? I.e. to build the correct tree and to run the algorithm
- your hierarchical tree remind me the chaining technique. Though the later cannot be used for bandits, I am wondering if there is any connexion.
- maybe some figures would help to understand the algorithm
- is it possible to calibrate eta and the depth of the tree in T?
- do you think it may be possible to learn the metric online?
Typos:
- l171: a dot is missing
- l258: "actions K. and"
- Algo1: a space is missing; add eta > 0 to the input;
- Eq (3): some formating issue
- l422: I did not understand the explanation on condition (2)c, don't we have $k > = C$ by definition of C?
- 476: tilde are missing


Inductive Representation Learning on Large Graphs
Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.

The authors introduce GraphSAGE, an inductive learning representation learning method for graph-structured data.  Unlike previous transductive methods, GraphSAGE is able to generalize the representation to previously unseen nodes.  The representation is learned through a recursive process that samples from a node's neighborhood and aggregates the results.  GraphSAGE outperforms other popular embedding techniques at three node classification tasks.
Quality:
The quality of the paper is very high.  The framework has several attractive qualities: a representation that is invariant with respect to node index, strong performance at three inductive node classification tasks, and fast training and inference in practice.  The authors include code that they intend to release to the public, which is likely to increase the impact of the work.
Clarity:
The paper is very well-written, well-organized, and enjoyable to read.
Originality:
The idea is incremental but creative and useful.
Significance:
This work is likely to be impactful on the NIPS community due to the strong results as well as the fast and publicly available implementation.  The work builds on recent developments to offer a tangible improvement to the community.
Overall impression: A high-quality paper that is likely to be of interest to the NIPS community and impactful in the field.  Clear accept.


Spectrally-normalized margin bounds for neural networks
This paper presents a margin-based multiclass generalization bound for neural networks that scales with their margin-normalized spectral complexity: their Lipschitz constant, meaning the product of the spectral norms of the weight matrices, times a certain correction factor. This bound is empirically investigated for a standard AlexNet network trained with SGD on the mnist and cifar10 datasets, with both original and random labels; the bound, the Lipschitz constants, and the excess risks are all in direct correlation, suggesting both that SGD selects predictors whose complexity scales with the difficulty of the learning task, and secondly that the presented bound is sensitive to this complexity.

This paper proposes analysing the margin distributions of deep neural network to analyse their generalization properties. In particular they propose a normalized margin estimator, which is shown to reflect generalization properties of certain neural networks much better than the naive margin computation.
Your notion of margin is different than what I'm used to. (I usually hear margin defined as the minimum distance in input space to a point of a different class, as SVMs are commonly depicted). Maybe just take a few words to stress this?
Why does the x axis on some (if not all?) your figures not start at 0? Shouldn't the margin always be positive? Unless you're not taking y to be the argmax of F(x) but instead the true class, in which case it would be negative for a missclassification. Maybe mention this.
What is the data distribution in your plots? Train, validation, or test? I general your plots should have clearer axes or captions.
In Theorem 1.1 it's not clear to me what B and W are (without the appendix), maybe just briefly mention what they are.
CIFAR and MNIST are acronyms, and should be capitalized. Ditto for SGD.
There are some typos throughout, and I feel like the writing is slightly "casual" and could be improved.
Your section on adversarial examples is somewhat puzzling to me. The margins, let's say M1, that you describe initially are in Y-space, they're the distance between the prediction f(x)_y and the next bess guess max_i!=y f(x)_i. This is different than the input space margin, let's say M2, which is something like `min_u ||u-x|| s.t. f(u) \neq f(x)`. As such when you say "The margin is nothing more than the distance an input must traverse before its label is flipped", you're clearly referring to M2, while it would seem your contribution revolves around M1 concept of margin.
While intuitively M1 and M2 are probably related, unless I'm missing something about your work relating them, claiming "low margin points are more susceptible to adversarial noise than high margin points" is a much stronger statement than you may realise.
Overall I quite like this paper, both in terms of the questions it is trying to answer, and in the way in does.
On the other hand, the presentation could be better, and I'm still not convinced that the margin in output space has too much value in the way we currently train neural networks. We know for example that the output probabilities of neural networks are not predictive of confidence (even to an extent for Bayesian neural networks). I think more work relating the input margin with the output margin might be more revealing.
I've given myself a 2/5 confidence because I am not super familiar with the math used here, so I may be failing to get deeper implications. Hopefully I'll have time to go through this soon.


On the Power of Truncated SVD for General High-rank Matrix Estimation Problems
We show that given an estimate � A that is close to a general high-rank positive semidefinite (PSD) matrix A in spectral norm (i.e., � � A−A� 2 ≤ δ), the simple truncated Singular Value Decomposition of � A produces a multiplicative approximation of A in Frobenius norm. This observation leads to many interesting results on general high-rank matrix estimation problems:
1. High-rank matrix completion: we show that it is possible to recover a general high-rank matrix A up to (1 + ε) relative error in Frobenius norm from partial observations, with sample complexity independent of the spectral gap of A. 2. High-rank matrix denoising: we design an algorithm that recovers a matrix A with error in Frobenius norm from its noise-perturbed observations, without assuming A is exactly low-rank. 3. Low-dimensional approximation of high-dimensional covariance: given N i.i.d. samples of dimension n from N n (0, A), we show that it is possible to approximate the covariance matrix A with relative error in Frobenius norm with N ≈ n, improving over classical covariance estimation results which requires N ≈ n 2 .

This paper shows that given an estimate that is close to the target high-rank PSD matrix A in spectral norm a simple truncated SVD gives an estimate of A that is also close to A in Frobenius norm as well. This leads to a number of applications: high-rank matrix completion, high-rank matrix denoising and low-dimensional estimation of high-dimensional covariance. The paper also gives a bound that depends on the sigma_k - sigma_{k + 1} gap (Theorem 2.2).
Overall, I think this is a pretty strong paper that gives a nice set of results. Specific discussion of some of the strengths and weaknesses is below.
Strengths:
-- Clean set of results, good presentation and clear comparison with the previous work.
-- The algorithm is very simple and is very practical.
Weaknesses:
-- The constant in multiplicative error depends on k and delta
-- Results for high-rank matrix completion are conditional on the matrix satisfying a mu_0-spikiness condition which I didn’t find very natural
-- No experimental results are given
Technical comments:
-- In theorem 2.1 you need a bound on delta in terms of \epsilon and \sigma_{k + 1}. How do you ensure such a bound without knowing sigma_{k + 1}?


Preventing Gradient Explosions in Gated Recurrent Units
A gated recurrent unit (GRU) is a successful recurrent neural network architecture for time-series data. The GRU is typically trained using a gradient-based method, which is subject to the exploding gradient problem in which the gradient increases significantly. This problem is caused by an abrupt change in the dynamics of the GRU due to a small variation in the parameters. In this paper, we find a condition under which the dynamics of the GRU changes drastically and propose a learning method to address the exploding gradient problem. Our method constrains the dynamics of the GRU so that it does not drastically change. We evaluated our method in experiments on language modeling and polyphonic music modeling. Our experiments showed that our method can prevent the exploding gradient problem and improve modeling accuracy.

Summary
The authors propose a method for optimizing GRU networks which aims to prevent exploding gradients. They motivate the method by showing that a constraint on the spectral norm of the state-to-state matrix keeps the dynamics of the network stable near the fixed point 0. The method is evaluated on language modelling and a music prediction task and leads to stable training in comparison to weight clipping.
Technical quality
The motivation of the method is well developed and it is nice that the method is evaluated on two different real-world datasets. I also like the plots that show the relation between the spectral radius of the linear approximation of the system and the norm of the gradient. However, one important issue I have with the evaluation is that the learning rate is not controlled for in the experiments. Unfortunately, this makes it hard to draw strong conclusions from the results. The starting learning rates seem to be quite high and this may very well be the reason why the clipping-based optimization performs so poorly. Experiments with a range of different starting learning rates could give more insights into the importance of the method in general. Of course it would still be a strength of the method that it is less sensitive to the choice of learning rate but if simply lowering the learning rate a bit solves most of the issues the clipping has, it lessens the impact of the method.
Clarity
Despite the large number of equations, the argumentation is easy to follow. Except for some grammar mistakes, the paper is well-written in general.
Novelty
While recurrent neural networks have been quite thoroughly analysed in similar ways, this is to my knowledge the first work that does this for GRUs. I also don't know about any methods that combine SGD training with SVD to clip certain singular values.
Significance
While the analysis is specific to GRUs, I don't see why it couldn't inspire similar analysis of other popular RNN architectures. GRUs are also quite popular and for that reason the proposed method could have practical value for people. That said, it is hard to predict whether some new architecture will show up and take over or not and whether this method will be significant in the long run.
pros:
The method is well-motivated and the derivation can be useful by itself.
Very insightful plot of the relation between the gradient norm and spectral radius.
GRUs are very popular at the moment so if the method works well it may find widespread use.
cons:
Only works for GRUs
The experiments could be the result of the particular choice of starting learning rate


Label Efficient Learning of Transferable Representations across Domains and Tasks
We propose a framework that learns a representation transferable across different domains and tasks in a label efficient manner. Our approach battles domain shift with a domain adversarial loss, and generalizes the embedding to novel task using a metric learning-based approach. Our model is simultaneously optimized on labeled source data and unlabeled or sparsely labeled data in the target domain. Our method shows compelling results on novel classes within a new domain even when only a few labeled examples per class are available, outperforming the prevalent fine-tuning approach. In addition, we demonstrate the effectiveness of our framework on the transfer learning task from image object recognition to video action recognition.

The works proposed a new framework to learn a representation that can transfer across domains as well as different tasks. There are two main novelties of this work: 1) a multi-layer domain adversarial loss vs existing work which learns a layer-by-layer embedding (generator) and discriminator; 2) a semantic transfer loss defined in equation (4) to address the cases when the source and target domain have different tasks (classes). Experimental results demonstrated that the multi-layer domain adversarial loss improves domain alignment as well as stability of adversarial training. Table 1 suggests that semantic transfer offers a significant boost to the performance of the proposed method.
The paper is mostly well-written and easy to follow. The notations are not very consistent. Section 3.3 could be improved. The experimental results showed significant improvement over existing methods with the two modification introduced. There are three components introduced in the semantic transfer section. It is not clear from the experimental results that which one is contributing to the improvement mostly. It is not clear why the authors refer to the loss in equation (5) as a metric-based loss.
In section 3.3, it seems like the similarity \phi(,) should be defined over the learned embedding instead of the original raw input? Please clarify the notation in section 3.3 to be consistent with section 3.2 if it is the case.
In equation (4) and (6), do the authors use all the labeled examples or only the prototype examples to compute the cross-entropy? The former case seems prohibitively expensive, especially when the number of samples in the source domain is big.  If it's the later case, how were the prototype examples chosen?
Minor question:
1) what is the \gamma parameter shown in table 1 ?


Scalable Lévy Process Priors for Spectral Kernel Learning
Gaussian processes are rich distributions over functions, with generalization properties determined by a kernel function. When used for long-range extrapolation, predictions are particularly sensitive to the choice of kernel parameters. It is therefore critical to account for kernel uncertainty in our predictive distributions. We propose a distribution over kernels formed by modelling a spectral mixture density with a Lévy process. The resulting distribution has support for all stationary covariances-including the popular RBF, periodic, and Matérn kernelscombined with inductive biases which enable automatic and data efficient learning, long-range extrapolation, and state of the art predictive performance. The proposed model also presents an approach to spectral regularization, as the Lévy process introduces a sparsity-inducing prior over mixture components, allowing automatic selection over model order and pruning of extraneous components. We exploit the algebraic structure of the proposed process for O(n) training and O(1) predictions. We perform extrapolations having reasonable uncertainty estimates on several benchmarks, show that the proposed model can recover flexible ground truth covariances and that it is robust to errors in initialization.

This paper models the spectrum of a Gaussian process kernel with a location-scale mixture of Lalpacians, using the LARX method of [1] for a non-parametric prior over the coefficients. It is claimed that the levy prior promotes sparse solutions. A reverse jump mcmc method is proposed, following [1]. Inference is accelerated using a SKI approximation from [2], and results are presented on the Airline dataset and also a dataset from natural sounds.
The paper is nicely written and the idea is interesting and novel, but the evaluation of the method is extremely weak. In particular, there are no quantitive comparisons, nor any mention of other kernel learning approaches (e.g. compositional kernels [3]) or other spectral approaches to GP regression (e.g. sparse spectrum GPs [5])
The first experiment presented is actually not the approach of the paper, but instead the method of [1]. It is unclear how this provides evidence for the claims of the novel contributions of the paper. The second experiment shows (qualitatively) that the model can fit a sample from its prior better than a different model. It is hardly surprising that the method of [5] fares less well on this task as Gaussian model over the spectrum is misspecified. The third experiment provides a comparison with the method of [5], but from the graph it is extremely unclear which of the two method is better, and no results on held-out data are presented. The final experiment serves only as a demonstration of scalability as the original data from the in-filled section is not shown, and the only evaluation given is that ‘the reconstucted signal has no discernable artifacts when played back as sound’. Presumably the sound file will be included in the final version, but even so, it is not easy to assess the significance of such evidence. Do other more simple approaches achieve this property? (Incidentally, the quoted sentence appears not to have been spell-checked - fortunately it seems to be the only one!)
Beyond the experiments, I feel the paper should be improved by distinguishing its key contributions. It appears that there are two: the use of laplacian basis functions to model the fourier spectrum (as opposed to Gaussian in [5]), and the use of the levy prior. Is maximum likelihood over the laplacian features as effective as [5]? It should be made clear whether this is a modelling implication or chosen for convenience. Secondly, it should be explained in what circumstances a ML approach leads to overfitting, requiring the use of the levy prior.
Finally, I feel that the use of ‘scalable’ in the title is a little misleading. The kernel interpolation approach is an approximation (the quality of which is not discussed), and does not extend readily to high dimensions.
In summary, to improve this paper I suggest the results section is made more quantitive and emphasis on scalability is dropped and replaced by a focus on exactly why such priors are useful over a simpler ML approach.
[1] Wolpert, R.L., Clyde, M.A., and Tu, C. Stochastic expansions using continuous dictionaries: Levy adaptive regression kernels. The Annals of Statistics, 39(4):1916–1962, 2011.
[2] Wilson, Andrew Gordon and Nickisch, Hannes. Kernel interpolation for scalable structured Gaussian processes (KISS-GP). International Conference on Machine Learning (ICML), 2015.
[3] Duvenaud, David, et al. "Structure discovery in nonparametric regression through compositional kernel search." arXiv preprint arXiv:1302.4922 (2013).
[4] Quinonero-Candela, Joaquin, Carl Edward Rasmussen, and Annbal R. Figueiras-Vidal. "Sparse spectrum Gaussian process regression." Journal of Machine Learning Research 11.Jun (2010): 1865-1881.
[5] Wilson, Andrew, and Ryan Adams. "Gaussian process kernels for pattern discovery and extrapolation." Proceedings of the 30th International Conference on Machine Learning (ICML-13). 2013.
APA


Improving the Expected Improvement Algorithm
The expected improvement (EI) algorithm is a popular strategy for information collection in optimization under uncertainty. The algorithm is widely known to be too greedy, but nevertheless enjoys wide use due to its simplicity and ability to handle uncertainty and noise in a coherent decision theoretic framework. To provide rigorous insight into EI, we study its properties in a simple setting of Bayesian optimization where the domain consists of a finite grid of points. This is the so-called best-arm identification problem, where the goal is to allocate measurement effort wisely to confidently identify the best arm using a small number of measurements. In this framework, one can show formally that EI is far from optimal. To overcome this shortcoming, we introduce a simple modification of the expected improvement algorithm. Surprisingly, this simple change results in an algorithm that is asymptotically optimal for Gaussian best-arm identification problems, and provably outperforms standard EI by an order of magnitude.

The authors propose a new best arm identification algorithm called TTEI that is an extension of "expected improvement" (EI), a known heuristic from the Bayesian optimization community. The motivation for the extension is to match the asymptotic optimalities (rate of posterior convergence & expected stopping time), which are important properties of an algorithm. The authors analyze the asymptotic property of TTEI, show its (near)-optimality, and present its superior empirical performance in synthetic datasets.
Overall, well-written paper on an interesting problem. I highly rate the novelty of the algorithm. On the other hand, the asymptotic guarantee often does not capture the reality well in my opinion (described later, though I think this isn't a reason for rejection). I think the experiment part is the weakest in that they do not describe how and why their algorithm work better. Also, I would like to see how much TTEI behaves differently from TTTS since they share many asymptotic properties. This was not captured in the experiments.
Question on the rate of posterior convergence: The authors claim at line 97 that the analysis takes place in a frequentist setting. That is true, but isn't the analysis on posterior convergence essentially on the posterior probability "computed by the algorithm"? Sure, we can talk about a quantity that is computed by the algorithm, but if one considers a frequentist setting then what is the meaning of Theorem 2 and how it is solving the problem in the frequentist setting? That is, shouldn't one also talk about what it means to have (1-\alpha_{n,1}) approach to 0 in the frequentist setting?
One weakness would be that the guarantee is limited to Gaussian (unlike other papers like the lil' ucb paper that applies to sub-Gaussian RVs), so it does not apply to bernoulli case that is very popular in practice. I wonder if one can relax the random variables to be sub-Gaussian whose scale parameter is bounded below a known constant. What is the authors' opinion on this? I wonder if there are critical steps in the proof that works for Guassian but not for generic sub-Gaussian.
Any moderate-confidence regime results? Recent analysis on the top arm identification (such as Chen and Li On the Optimal Sample Complexity for Best Arm Identification, 2016) tells us that there are some important terms that does not depend on \log(1/\dt), so there might be some practically relevant terms the asymptotic analysis is missing. I appreciate the asymptotic optimality, but what happens in practice is that no one wants to use \delta that is too small and in most cases \log(1/\delta) can be treated as constants in the set of problems we care about. I also wonder whether this fact can affect the practical performance.
Still, I like to emphasize that I don't think the lack of non-asymptotic analysis should be the reason for rejection in any ways as such analysis is often hard and I believe people should go beyond that to explore new algorithms.
Could you provide a more detailed comparison between Russo'16? Are there any difference in the asymptotic optimality? Any different sampling behavior that one can observe in practice?
Below L194, why do we need |mu_n,i - \mu_i| \le \epsilon there? If so, what is the key mathematical reason for that? (or, is it for convenience of mathematical derivation?) As far as intuition goes, it seems like if N is large and the sampling proportion is controlled, then one should be able to control |mu_n,i - \mu_i|. Let me know what the authors think.
In experiments, why do we use "max_i \alpha_{n,i} \ge c" as the stopping criterion? I thought the good stopping criterion is the one from Garivier & Kaufmann. How is this stopping rule used for the experiments justified in the frequentist setting? Also, I think the authors must have compared the actual probability of misidentification, especially as a graph in P(misid.) vs the # of samples, which measures 'anytime' performance that I think is practically relevant. Could the authors comment on this?
Also, I was expecting the reason why we see RSO works worse than TTEI.
L274: f => of
L219: allocation => allocate
L260: guarantees -> guarantee
L294: what is the definition of the bold-faced numbers in Table 2? Does that coloring involves some statistical significancy test?


Fitting Low-Rank Tensors in Constant Time
In this paper, we develop an algorithm that approximates the residual error of Tucker decomposition, one of the most popular tensor decomposition methods, with a provable guarantee. Given an order-K tensor X ∈ R N1×···×N K , our algorithm randomly samples a constant number s of indices for each mode and creates a "mini" tensorX ∈ R s×···×s , whose elements are given by the intersection of the sampled indices on X. Then, we show that the residual error of the Tucker decomposition ofX is sufficiently close to that of X with high probability. This result implies that we can figure out how much we can fit a low-rank tensor to X in constant time, regardless of the size of X. This is useful for guessing the favorable rank of Tucker decomposition. Finally, we demonstrate how the sampling method works quickly and accurately using multiple real datasets.

Motivated by the problem of selecting an appropriate Tucker rank for low-rank tensor approximation, the authors provide an algorithm for estimating the RMSE of a tensor at a given Tucker rank. The algorithm is simple: randomly sample a subset of the indices in each mode, restrict to the resulting tensor, and compute the RMSE for the given Tucker rank of this smaller tensor.
Their theory argues that the number of samples needed to get a good additive error approximation to the RMSE at a given Tucker rank depends on intrinsic parameters of the problem rather than the dimensions of the tensor: in particular, on the order of the tensor, the largest magnitude entry in the tensor (its spikiness), the spikiness of the Tucker core tensor of the original tensor, and the spikiness of the Tucker core tensor of the sampled tensor.
The experimental results are impressive, and show that not only does this algorithm provide good estimates of the RMSE at given Tucker ranks (compared to the HOOI baseline), it also preserves the ordering of the Tucker ranks by the RMSEs this achieve. The latter point is crucial for model selection.
This work is interesting and worth publishing in NIPS because it tackles the problem of rank selection for low-rank tensor approximation in a novel way, provides theoretical guarantees that seem reasonable, and the resulting algorithm is both fast and does a good job of rank selection. The main drawback of this paper is that the theory is presented in a non-intuitive manner that makes it hard to understand the connection that the authors draw between tensors and dikernels and how it is useful in establishing the main result (I did not read the supplementary material).
- I would like to see the authors address the dependence on the spikiness of the Tucker core tensor of the subsampled tensor: is there a reason this should be here, or can it be removed? can this spikiness be bounded in terms of properties of the original tensor, or can it be arbitrarily poorly behaved?
- It would be useful to state your theorem 3.4 for a matrix first, so the readers can use their experience in that domain to understand the quantities involved in your bounds
- The argument in line 162 that Lemma 3.3 implies the following equation is not clear to me, since it seems to ignore the effect of the measure-preserving bijection. I did not check the supplementary material to see if this argument is made clearer, but if so, please state this in the paper.
Another point:
- I would like to see bold and italicization in Tables 2--4 to indicate the best and second best performing methods, as is standard


On Blackbox Backpropagation and Jacobian Sensing
From a small number of calls to a given "blackbox" on random input perturbations, we show how to efficiently recover its unknown Jacobian, or estimate the left action of its Jacobian on a given vector. Our methods are based on a novel combination of compressed sensing and graph coloring techniques, and provably exploit structural prior knowledge about the Jacobian such as sparsity and symmetry while being noise robust. We demonstrate efficient backpropagation through noisy blackbox layers in a deep neural net, improved data-efficiency in the task of linearizing the dynamics of a rigid body system, and the generic ability to handle a rich class of input-output dependency structures in Jacobian estimation problems.

The paper focuses on automatic differentiation of multiple-variable vector-valued functioned in the context of training model parameters from input data. A standard approach in a number of popular environments rely on a propagation of the errors in the evaluation of the model parameters so far using backpropagation of the estimation error as computed of the (local) gradient of the loss function. In this paper, the emphasis is on settings where some of the operators of the model are exogenous blackboxes for which the gradient cannot be computed explicitly and one resorts to finite differencing of the function of interest. Such approach can be prohibitively expensive if the Jacobian does not have some special structure that can be exploited.

The strategy pursued in this paper consists in exploiting the relationship between graph colouring and Jacobian estimation. In a nutshell, the mapping encodes the outputs that jointly influence the model. The consequence of this is a known result stating that the chromatic number of the associated graph provides a bound on the number of calls to a function to compute its Jacobian, in the noiseless setting. The paper goes one step beyond this established fact and proposes an algorithms “rainbow” that defines the perturbation directions where the blackbox needs to be evaluated in order to compute the associated Jacobian . The crux of the algorithm is to encode the properties of the Jacobian (e.g. sparsity, symmetry) using a graph colouring procedure, arising from the aforementioned graph mapping, into a simple linear programme.

The authors theoretically evalute the performance of this algorithm in section 3.4.  Theorem 3.3. upperbounds the number of blackbox needed to evaluate its Jacobian in terms of the graph encoding input output dependencies as well as its intrinsic degrees of freedom among other things. In Section 4, the proposed algorithm is evaluated in following settings: two synthetic input-output dependency structures (random and power-law), a CNN, and a planning and control setting.

The paper is clearly written and properly spells out the relevant technical details of the proposed algorithm as well as its performance.
The main weakness of the paper resides in section 3.3, focusing on the LP approach to define the perturbation direction needed to evaluate the Jacobian. There are also two minor points that would improve the legibility of the paper. First, the paragraph after theorem 3.3 (page 7 l.257-l .260) should provide further intuition about the bounds on the calls to the function needed by the algorithm to estimate its Jacobian. Then, the experiment section 4 would benefit from providing a rationale behind the choice of the scenarios explored and what is illustrated about the algorithm in each of them.


Good Semi-supervised Learning That Requires a Bad GAN
Semi-supervised learning methods based on generative adversarial networks (GANs) obtained strong empirical results, but it is not clear 1) how the discriminator benefits from joint training with a generator, and 2) why good semi-supervised classification performance and a good generator cannot be obtained at the same time. Theoretically we show that given the discriminator objective, good semisupervised learning indeed requires a bad generator, and propose the definition of a preferred generator. Empirically, we derive a novel formulation based on our analysis that substantially improves over feature matching GANs, obtaining state-of-the-art results on multiple benchmark datasets 2 .

After reading the rebuttal I changed my score to 7.  Overall it is an interesting paper with an interesting idea. Although the theoretical contributions are emphasized I find the empirical findings more appealing. The theory presented in the paper is not convincing (input versus feature, convexity etc).
I think the link to classical semi-supervised learning and the cluster assumption should be emphasized, and the * low density assumption on the boundary* as explained in this paper :
Semi-Supervised Classification by Low Density Separation
Olivier Chapelle, Alexander Zien
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.76.5826&rep=rep1&type=pdf

I am changing my review to 7, and I hope that the authors will put their contribution in the context of known work in semi-supervised learning , that the boundary of separation  should lie in the low density regions . This will put the paper better in context.
-----
This paper provides an analysis of how GAN helps in  semi supervised learning (in the “K+1” setting of [15]). The paper outlines some assumptions under which it is shown that a complement generator is needed to improve the accuracy of the supervised task at hand. Meaning that the generator needs to  target low densities areas in the input space.
Using this idea an algorithm is given that combines that feature matching criterium , with a density estimation (using a pixel CNN ) under which the generator targets low density areas of the fixed estimated model (for a given threshold of log proba,).
Other entropy regularizers are added to encourage diversity in the generator. Positive empirical results are reported.

Understanding GANs in the semi supervised setting and improving  it  is  an important problem, however the paper has many caveats:
- while the analysis is done in the feature space under a lot of assumptions, the method proposed is in the input space , which gives a big mismatch between the analysis and the proposed method . Convexity that is used all over the proofs is not any more valid.
-  the KL expression (line 238 ) is wrong:  the term assumed to be constant is not constant. It is equal to ‘-log(C) P(x sim p_g, p(x)<=epsilon)’, this term should be optimized as well.  Some other way to come up at the objective presented need to be developed.
Maybe just motivating the minimization of the cross entropy term, and adding a cross entropy regularization?
- From the analysis and the  2D example  with uniform sampling off the manifold, the paper seems to suggest that the generator should supply samples outside the manifold, in a sense it reinforces the boundaries of the classifier by providing only negative samples. The truth is in between: the generator should not provides too strong samples (very realistic, easy to classify as a class ) , nor too weak samples that   are easily labeled as fake. it should be portion on the manifolds to reinforce the positive , and portions outside  to reinforce the negative.
A more realistic setup may be probabilistic where the assumption are assumed to hold with probability 1- delta as off the manifold, and delta on the manifold, although corollary 1 is not proven and hard to understand how it could hold , it seems not realistic to me .
Balancing this delta seem to be crucial and seems inline with the analysis of https://arxiv.org/pdf/1705.08850.pdf

- Ablation study:  Performance of the method  in section 6 for many values of epsilon values would illustrate the discussion above, and should be reported, also a justification of the 10 quantile should be provided.
-Overall using a density estimation (Pixel CNN) and entropic regularizer with another estimator seems a bit adhoc and not satisfying, wonder if the authors have other thoughts to avoid those estimators while encouraging a form of  'low density sampling’ while using only implicit modeling. or  hybrid implicit/ prescribed modeling while maintaining an end to end training?
Minor comments:
- the loss is L_supervised + L_unsupervised have you experimented with balancing the two terms with a penalty term lambda, how does this  balancing interact with the threshold epsilon?
- In lines '162 and 163'  This builds a connection with ... [2,25] which leverage "the low density boundary assumption". I don't understand this comment , can you elaborate more on what this assumption in laplacian regularization for instance?
- line 178 typo : 'the a  bounded'
- corollary one : the proof is not provided. I don't understand how those claims are true uniformly over all X, and what does it means |G| goes to infinity, you may need  stronger assumptions here....


Introspective Classification with Convolutional Nets
We propose introspective convolutional networks (ICN) that emphasize the importance of having convolutional neural networks empowered with generative capabilities. We employ a reclassification-by-synthesis algorithm to perform training using a formulation stemmed from the Bayes theory. Our ICN tries to iteratively: (1) synthesize pseudo-negative samples; and (2) enhance itself by improving the classification. The single CNN classifier learned is at the same time generative -being able to directly synthesize new samples within its own discriminative model. We conduct experiments on benchmark datasets including MNIST, CIFAR-10, and SVHN using state-of-the-art CNN architectures, and observe improved classification results.

The paper proposes a technique to improve the test accuracy of a discriminative model, by synthesizing additional negative input examples during the training process of the model. The negative example generation process has a Bayesian motivation, and is realized by "optimizing" for images (starting from random Gaussian noise) to maximize the probability of a given class label, a la DeepDream or Neural Artistic Style. These generated examples are added to the training set, and training is halted based on performance on a validation set. Experiments demonstrate that this procedure yields (very modest) improvements in test accuracy, and additionally provides some robustness against adversarial examples.
The core idea is quite elegant, with an intuitive picture of using the "hard" negatives generated by the network to tighten the decision boundaries around the positive examples. The authors provide some theoretical justification showing that the distribution of these negatives examples converges to the distribution of positive examples.
While I find the approach intriguing, there are a few shortcomings. The improvement in classification accuracy demonstrated in the experiments is relatively small for the datasets shown, although the MNIST demonstration with varying sample sizes is very encouraging. A more complex dataset (e.g. ImageNet) would be very interesting to see, but potentially quite expensive to train... I'm unable to determine how many pseudo-negative samples are generated in each round (it says "e.g. 200" at one point; is it 200?), which affects the practicality of the approach (how much slower is training?).
While the Bayesian motivation is nice, it's not clear to me exactly what it means to sample from the distribution of negative examples using SGD. I'm not actively familiar with the references listed tying SGD to MCMC; if there is in fact a meaningful interpretation of the proposed sampling procedure that justifies the vague (but repeated) use of the term "fair sample", it would be nice to include a more explicit summary in the text.
Finally, it would have been nice to see comparisons against DCGAN for CIFAR10 and SVHN as well, as well as a simple baseline of label smoothing (instead of using one-hot target labels, use a mixture of e.g. .9 * one-hot + .1 * uniform). Label smoothing has been reported (e.g., in https://arxiv.org/pdf/1512.00567.pdf) to yield comparably sized improvements, essentially by encouraging the model to be less confident. One explanation for the success of the ICN approach could simply be that by training on negative samples that are close to the positive samples, the model is forced to be less confident overall, reducing overfitting. Comparing against label smoothing could help tease out whether that explanation is sufficient (and if so, label smoothing is substantially simpler and cheaper, if a bit of a hack).
Despite the above criticisms, I think the paper is well written and could be a valuable contribution; I recommend acceptance.


Stochastic and Adversarial Online Learning without Hyperparameters
Most online optimization algorithms focus on one of two things: performing well in adversarial settings by adapting to unknown data parameters (such as Lipschitz constants), typically achieving O( √ T ) regret, or performing well in stochastic settings where they can leverage some structure in the losses (such as strong convexity), typically achieving O(log(T )) regret. Algorithms that focus on the former problem hitherto achieved O( √ T ) in the stochastic setting rather than O(log(T )). Here we introduce an online optimization algorithm that achieves O(log 4 (T )) regret in a wide class of stochastic settings while gracefully degrading to the optimal O( √ T ) regret in adversarial settings (up to logarithmic factors). Our algorithm does not require any prior knowledge about the data or tuning of parameters to achieve superior performance.

This paper introduces a parameter-free algorithm for online learning that attains non-trivial guarantees in both the stochastic and adversarial settings simultaneously. Previous work in this area includes the MetaGrad algorithm, which was not parameter-free and required knowledge of L_max, but simultaneously attained O(B L_max \sqrt{T}) regret in the adversarial setting and O(dBL_max log(T)) in the stochastic or strongly setting. The key insight by the authors is to interpret an upper bound on the regret as an expectation over the learner's output vectors (scaled by the norm of the gradient at each round), to rewrite this expression in terms of a bias and variance term, and then to shift the regularization component of an existing parameter-free algorithm by a running estimate of the previously mentioned expectation. This results in an algorithm with O(\sqrt{T} \log(T)^2 L_max B) regret. When the loss functions satisfy a certain curvature property called \alpha-acute convexity, the algorithm is able to attain a regret in O(\log(T)^4 L_max B/\alpha).
This paper introduces a clever method for an interesting area of online learning, that of parameter-free algorithms that work well in multiple environments. The manuscript is well-written and the authors provide the reader with an abundance of both insight into the problem and intuition for their solution. The main concern I have for the paper is the practical utility of their algorithm. The algorithm's adversarial guarantee includes an extra log(T)^2 factor in addition to the usual terms, and, as the authors admit, the stochastic guarantee in the stochastic setting may only be compelling for extraordinarily large values of T. Still, this paper is at the very least an interesting theoretical contribution.
Some technical questions:
1) Line 107: Can the authors explain how they arrived at the 2nd inequality?
2) Line 184: Should the last expression be \frac{\mu}{2} \|w - w^*\|^2?
3) Line 187: Can the authors explain how they arrived at the 2nd inequality (i.e. .... \geq \frac{\mur}{2}\|w\|)?
4) As the authors mention, for strongly convex functions, \alpha = \mu/(2 L_max). Thus, the stochastic bound will become O(\log(T)^4B/\mu). However, the standard online convex optimization guarantee for strongly convex loss functions is O(log(T)L^2/\mu). Apart from the additional log(T) terms, can the authors discuss the difference in dependence on B vs L^2?
Post author response: I've read the author response and choose to maintain my score.


Label Distribution Learning Forests
Label distribution learning (LDL) is a general learning framework, which assigns to an instance a distribution over a set of labels rather than a single label or multiple labels. Current LDL methods have either restricted assumptions on the expression form of the label distribution or limitations in representation learning, e.g., to learn deep features in an end-to-end manner. This paper presents label distribution learning forests (LDLFs) -a novel label distribution learning algorithm based on differentiable decision trees, which have several advantages: 1) Decision trees have the potential to model any general form of label distributions by a mixture of leaf node predictions. 2) The learning of differentiable decision trees can be combined with representation learning. We define a distribution-based loss function for a forest, enabling all the trees to be learned jointly, and show that an update function for leaf node predictions, which guarantees a strict decrease of the loss function, can be derived by variational bounding. The effectiveness of the proposed LDLFs is verified on several LDL tasks and a computer vision application, showing significant improvements to the state-of-the-art LDL methods.

The authors describe a method for label distribution learning based on differentiable decision trees. The authors use differentiable sigmoid units to estimate a label distribution using leaf nodes of trees. Learning in split nodes is done via backprop. whereas for leaf nodes the authors propose a methodology based on variational bounding. The authors compare their work with relevant methods on learning label distributions and show the competitiveness of their method.
I think this is a good paper, providing a sound methodology for learning LD. The main weakness of this paper is that it is somewhat difficult for the reader to differentiate the actual contribution of this paper with respecto to reference [20]. I would like to see an answer to this in the rebuttal phase.
detailed comments:
typo: The best parameters ARE determined by:
It is not clear to me whether the way split-nodes are learned is a contribution of this paper, or if it is described in [20], please clarify
Do you have any plans of releasing the code of the LDF method with caffe?
The numbers of trees and deep of trees is quite small, the authors make an effort to analyze the performance of their method under different parameter settings, but, the number of trees and depth is still small, what are author thoughts in much larger ensembles and tree depths? Have the authors performed experiments evaluating this setting? Please elaborate on this
For the experimental results from Section 4.1, are the results from Table 1 comparable to each other? How do you guarantee this? Please provide more details
For the problem approached in Section 4.2, can the authors please report the state of the art performance? (e.g., not only based on label distribution learning)
What is the complexity of the model?, also, can the authors report runtime?


Early stopping for kernel boosting algorithms: A general analysis with localized complexities
Early stopping of iterative algorithms is a widely-used form of regularization in statistics, commonly used in conjunction with boosting and related gradienttype algorithms. Although consistency results have been established in some settings, such estimators are less well-understood than their analogues based on penalized regularization. In this paper, for a relatively broad class of loss functions and boosting algorithms (including L 2 -boost, LogitBoost and AdaBoost, among others), we exhibit a direct connection between the performance of a stopped iterate and the localized Gaussian complexity of the associated function class. This connection allows us to show that local fixed point analysis of Gaussian or Rademacher complexities, now standard in the analysis of penalized estimators, can be used to derive optimal stopping rules. We derive such stopping rules in detail for various kernel classes, and illustrate the correspondence of our theory with practice for Sobolev kernel classes.

The paper establishes minimax optimal rates of estimation for early stopping of kernel boosting, where the stopping index is expressed in terms of localized Rademacher/Gaussian complexities.
Overall, this paper is well written and closes a gap in the literature: while lower bounds in terms of localized complexity measures are known, a similar result for the choice of the nb. of iterations (and therefore upper bounds) was unknown.
Some comments:
***Remark after Th. 1 about random design setting should be more explained (at least for the not initiated reader); applies (11b) to squared population L^2-norm with f^T or \bar f^T ?
***l. 188: I think Yang et al. [30] state a lower bound involving the critical radius delta_n defined as the smallest positive solution such that R(delta)<=delta^2/sigma ,
with R(delta) given in (6) (rather than the worst-case constant sup delta^2_n; or are these two numbers equivalent ?).
***l.223/224: could you provide a reference?
***a possible weak spot: numerical experiments are provided only for the error involving f^T in distinction to Th. 1. The authors remark in l.282/283 that a rigorous proof is missing
(on the other hand, based on this simulations one can indeed strongly suspect that the same result holds for f^T as well)
A few very minor remarks about typesetting:
*** l. 155 ff. --> the function class {\mathcal E} and also delta depend on n, which could be indicated for better understanding; same for the effective noise level sigma for least squares
*** l.158: "Gaussian complexity ... at scale sigma"-->should be "delta"


Identifying Outlier Arms in Multi-Armed Bandit *
We study a novel problem lying at the intersection of two areas: multi-armed bandit and outlier detection. Multi-armed bandit is a useful tool to model the process of incrementally collecting data for multiple objects in a decision space. Outlier detection is a powerful method to narrow down the attention to a few objects after the data for them are collected. However, no one has studied how to detect outlier objects while incrementally collecting data for them, which is necessary when data collection is expensive. We formalize this problem as identifying outlier arms in a multi-armed bandit. We propose two sampling strategies with theoretical guarantee, and analyze their sampling efficiency. Our experimental results on both synthetic and real data show that our solution saves 70-99% of data collection cost from baseline while having nearly perfect accuracy.

This work is concerned with a variant of the multi-armed bandit problem, and more specifically a pure exploration, fixed confidence problem. Here, the objective is to identify with probability at least 1-\delta the arms that are over a threshold, using as few samples as possible. The twist, compared to a setting introduced in [1], is that this threshold is not known a priori by the learner, and instead depends on the means of the arms (unknown to the learner) and an outlier parameter k (known to the learner). With respect to the existing literature, this game can be seen as a pure exploration bandit problem with a double exploration trade-off: first, pulling arms in a balanced way as to estimate each mean to improve the estimate of the (unknown) threshold \hat \theta and second, pulling arms whose means are close to \hat \theta in order to determine whether they belong to the set of outliers. If \theta were known to the learner, it is well understood that one would require roughly 1/(y_i - \theta)^2 samples to determine to which set arm i belongs. To solve this problem, two algorithms are introduced. The first one is a simple uniform exploration algorithm, that comes down to a test (making use of confidence intervals on the estimated threshold and means) as to discover which arms are outliers with high probability. The second algorithm is a distorted version of this algorithm, which overpulls (with respect to the uniform sampling) arms that are still active (i.e. we have not yet identified in which set they belong). The analysis shows that careful tuning of this distortion parameter \rho leads to improved performances over the uniform sampling algorithm. The tuning of this parameter with respect to the complexity measures H_1 (worst case complexity) and H_2 (which is similar to the usual complexity measure in pure exploration problems) is discussed. Extensive experiments are conducted to validate the theoretical results, on both a synthetic and real dataset.
I believe there is a typo in the definition of \sigma_y (there should be a sum). Some references are missing see e.g. [1] and [2,3]. In [2,3] the objective is to estimate as well as possible all the means. On the other hand [1] introduced the thresholding bandit problem. Though the lower-bound construction therein does not apply trivially (it is harder to construct problems for the lower-bound given all the interdependencies between the means in this setting), in spirit the complexity terms seem very related, as one should not expect to have a complexity for this problem lower than H_2 (as defined in this work). It would be interesting to see an even more adaptive strategy to tackle this problem, maybe with a different definition for the (problem dependent) threshold if this one proves too difficult for natural adaptivity. I don’t understand the 99% claim in the abstract, you should tone it down or explain it better in the experiments section.
Regarding the experiments, I believe that \delta should be set to a much lower value (0.01) and correspondingly they should be repeated at least 1000 times (even if \delta is not changed, experiments should be averaged over many more than 10 runs). I believe that an interesting baseline could be one where the threshold is given a priori to the learner, and it can simply successively accept or reject arms with confidence parameter \delta/n (to account for the union bound). This strategy should be the hardest baseline (which we don’t expect to beat, but it shows the possible improvement).  Some further: they could be repeated with different values of \rho, and the \rho parameter used here should be discussed. It would be interesting to see how the “worst-case” \rho parameter as described in (6) performs against a perfectly tuned \rho, on different problems. The synthetic experiments could be run instead on carefully crafted instances that correspond to different extreme problems.
Overall, I believe this new problem is well motivated, and the two algorithms introduced are natural strategies to tackle this problem which should serve as strong baselines for future work. The paper is written in a very clear and clean fashion, and it is easy to follow. One weakness is that the algorithms and techniques used are rather basic as the problem itself is hard to tackle. Moreover, a fixed budget transposition of the setting seems like challenging problem to solve in the future.
[1] Locatelli, Andrea, Maurilio Gutzeit, and Alexandra Carpentier. "An optimal algorithm for the thresholding bandit problem." Proceedings of the 33rd International Conference on International Conference on Machine Learning-Volume 48. JMLR. org, 2016.
[2] Carpentier, Alexandra, A Lazaric, M Ghavamzadeh, R Munos, Peter Auer
 "Upper-confidence-bound algorithms for active learning in multi-armed bandits." International Conference on Algorithmic Learning Theory. Springer, Berlin, Heidelberg, 2011.
[3] Antos, A., Grover, V., Szepesvári, C.: Active learning in heteroscedastic noise. Theoretical Computer Science 411, 2712–2728 (2010)


Unsupervised Transformation Learning via Convex Relaxations
Our goal is to extract meaningful transformations from raw images, such as varying the thickness of lines in handwriting or the lighting in a portrait. We propose an unsupervised approach to learn such transformations by attempting to reconstruct an image from a linear combination of transformations of its nearest neighbors. On handwritten digits and celebrity portraits, we show that even with linear transformations, our method generates visually high-quality modified images. Moreover, since our method is semiparametric and does not model the data distribution, the learned transformations extrapolate off the training data and can be applied to new types of images.

The paper proposes an algorithm for unsupervised learning of transformations based on modeling nearest neighbor pairs as linear combination of transforms. The technique only models the transformations, and not the full data distribution and so can (in principle) be applied to other data sets (for eg, learning from MNIST, but applying to characters from other languages). While this problem/objective function is non-convex, they provide a convex relation by approximating the true transformation matrix as a linear combination of rank-1 matrices derived from sampling the data (and they show that this is a good approximation to the true transform matrix).  They show that the technique recovers known transforms such as stroke thickness/rotation in addition to new transforms (blur, loop size).
Overall, I found the ideas in the paper interesting, and the paper well written.