Neurally-Guided Procedural Models: Amortized Inference for Procedural Graphics Programs using Neural Networks
Probabilistic inference algorithms such as Sequential Monte Carlo (SMC) provide powerful tools for constraining procedural models in computer graphics, but they require many samples to produce desirable results. In this paper, we show how to create procedural models which learn how to satisfy constraints. We augment procedural models with neural networks which control how the model makes random choices based on the output it has generated thus far. We call such models neurally-guided procedural models. As a pre-computation, we train these models to maximize the likelihood of example outputs generated via SMC. They are then used as efficient SMC importance samplers, generating high-quality results with very few samples. We evaluate our method on L-system-like models with imagebased constraints. Given a desired quality threshold, neurally-guided models can generate satisfactory results up to 10x faster than unguided models.

The authors consider the task of generating samples from constrained procedural models for computer graphics, similar to that considered in the stochastically ordered SMC paper by Ritchie et. al. (as cited by authors). They present an inference procedure based on sequential Monte Carlo, similar to recent approaches proposed by Gu et al. (as cited) and Paige & Wood (see below for ref), in which data-driven importance proposals are learned by gradient descent on an importance sampling estimate of the evidence upper bound (i.e. an expectation-propagation). The main contribution is the proposed neural net architecture for the data-driven proposals, which is specific to procedural graphics problems. This architecture that shows good performance in the experiments under consideration.
I think this paper is clearly written and makes some reasonable contributions, but could do with an editing pass to frame the approach a bit better and relate it to recent contributions that similarly seek to amortize inference in sequential models by training neural net proposals.  While the authors are the first (to my knowledge) to train NN proposals for SMC in a procedural graphics / probabilistic programming setting, their approach is of course very closely related the one developed by Gu and colleagues [1], which the authors cite and the one proposed by Paige and Wood [2], which they do not. I am assuming this work was done more or less concurrently and independently, and I in principle don't see a problem for this paper from the point of view of novelty. That said, the paper in its current revision still reads a bit like a graphics paper that pitches using NN proposals for SMC as its core idea. This is unfortunate, in that it would have been nice to see the authors relate their work to that done by others.  When it comes to amortizing inference in sequential problems by training neural nets, the key question (in my opinion) is how the design of the neural net should relate to the structure of the generative model. References [1] and [2] both present different different takes on this problem. I think the problem / model class under consideration by the authors has three some interesting properties, which I think they could discuss more explicitly  1. A partially generated image can be compared against the target image at any time. This means that the likelihood at any resampling step is actually a pretty good heuristic for the likelihood of the completed image. This is very different the setting in, say, state space models, where you evaluate a model against a previously unseen data point at every resampling step.   2. As far as I can tell, the models under consideration don't have any global parameters. This greatly simplifies the problem. For example, if you want to do sequential inference in a Gaussian mixture model in which you explicitly sample the model parameters, then you would need to design a neural net that predicts the mean  and covariance for each cluster given a dataset, which is a highly problem-specific task.  3. Because the authors are considering images, they can extract features in a coarse-to-fine like manner by averaging over pixels. This seems like a great thing to be able to do.   4. Another interesting thing about this setting (which the authors do discuss explicitly) is that the evaluation cost of the neural nets is not negligible. One therefore has to consider not only the amortized cost of training, but also the cost of prediction.   In terms of impact, I think the specificity of the problem class considered by the authors is both a good thing and a bad thing. On the one hand you might argue that there is little here that generalizes beyond procedural graphics tasks. On the other I think the paper serves as a nice case study.   On balance I am inclined to say this paper just about meets the threshold for acceptance.   Post Author Feedback edit: While I do agree with the authors that data driven proposals can, in principle, be of great use in vision as inverse graphics approaches, I am not immediately convinced that the particular neural net architecture proposed by the authors is directly applicable to this task.    Minor  > Our insight is that while such an oracle can be difficult or impossible to write by hand, it is possible to learn a program chain_neural that comes close.  - Let's get rid of "Our insight" and cite [1] and [2] here.  > When used for procedural modeling, however, it leads to models whose outputs lack diversity, making them unsuitable for generating visually-varied content.  - In this context it would be helpful to state SMC is an importance sampling method. In importance sampling methods the target density must be absolutely continuous with respect to the proposal density (see any introductory reference, the first that pops to mind is [3]), which in the authors language equates to the requirement that the proposal density is "less compact" than the target density.  - Why do the authors use a Gaussian to define the loss criterion in equation (3)? A priori, something like an exponential would seem more natural.   - In section 4.2, I would like the authors to define the notation ||∇ I_2(p)||=1, since a gradient is technically not defined for a binary image.   - The authors should call out the SI when positing equation 1.   - In the derivation of equation 1 in SI:          xs ∼ P_CM(x) -> xs ∼ P_CM(x | c)  - The authors should probably cite the Wake Sleep algorithm [4] as related work.    References  [1] Gu, S., Ghahramani, Z., and Turner, R. E. (2105) "Neural Adaptive Sequential Monte Carlo". Neural Information Processing Systems   [2] Paige, B., and Wood, F. (2016) Inference Networks for Sequential Monte Carlo in Graphical Models. International Conference for Machine Learning Research.   [3] Geyer, C. J. (2011). Importance Sampling, Simulated Tempering, and Umbrella Sampling. In Handbook of Markov Chain Monte Carlo Methods.  [4] Hinton, G. E., Dayan, P., Frey, B. J., and Neal, R. M. (1995). The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214):1158–1161.


Quantum Perceptron Models
We demonstrate how quantum computation can provide non-trivial improvements in the computational and statistical complexity of the perceptron model. We develop two quantum algorithms for perceptron learning. The first algorithm exploits quantum information processing to determine a separating hyperplane using a number of steps sublinear in the number of data points N , namely O( √ N ). The second algorithm illustrates how the classical mistake bound of O( 1 γ 2 ) can be further improved to O( 1 √ γ ) through quantum means, where γ denotes the margin. Such improvements are achieved through the application of quantum amplitude amplification to the version space interpretation of the perceptron model.

The authors present two ways of training the classical perceptron algorithm on a quantum computer based on Grover's search algorithm.   The key idea behind their algorithms is the mapping of the training procedure to a search for misclassified datapoints or separating hyperplanes (in the version space) from a uniform distribution over the data and a defined number of sampled hyperplanes, respectively. This mapping is explained in detail.  The authors formally show that their algorithms yield a quadratic speedup in the number of necessary training data points and the margin between the two classes, respectively.
In my opinion, this paper deserves the attention of the NIPS audience at least on poster level for sound investigation of an highly interdisciplinary and fundamental question whose answer might eventually have a strong impact on ML.  To the best of my knowledge, the proofs and arguments presented are correct. While an experimental evaluation would of course have been very good to back up the theoretical claims, the absence of a device capable of executing the algorithm excuses the authors from my point of view. As far as quantum algorithms go, the results are hence well presented and analyzed.  It is clear that this paper stands out in terms of originality and novelty (at least for the NIPS audience). While Grover's search is not a new algorithm, this application of it and the corresponding interdisciplinary insight certainly are novel by NIPS standards.  The impact is perhaps the most critical point as, for the reason stated above, this work is of purely theoretical nature and also far away from what most machine learning researches probably work on. However, it certainly does bridge multiple disciplines and should a quantum computer be built in the future, this work will surely have a strong impact and spawn much future work. Even if the immediate impact might not be high, researchers should be interested in important future developments in their field for which this work poses an example.  From my perspective, the authors (apart from some superfluous words that should be deleted and referring to Algorithm 2 in the proof of Lemma 2 when it should be Algorithm 1) try to express their ideas in a language understandable for the NIPS audience and rely on as little previous knowledge about quantum mechanics as possible. The explanations and proofs are also almost exclusively based on purely mathematical arguments, so that they should be understandable. However, some concepts  such as 'superposition' and resulting speedups might have deserved more explanation. The lack of knowledge of basic principle in QM might hinder the understanding of some readers. However, given the page limit I find that the authors did reasonably well in introducing the most important concepts from quantum mechanics.


Stochastic Optimization for Large-scale Optimal Transport
Optimal transport (OT) defines a powerful framework to compare probability distributions in a geometrically faithful way. However, the practical impact of OT is still limited because of its computational burden. We propose a new class of stochastic optimization algorithms to cope with large-scale OT problems. These methods can handle arbitrary distributions (either discrete or continuous) as long as one is able to draw samples from them, which is the typical setup in highdimensional learning problems. This alleviates the need to discretize these densities, while giving access to provably convergent methods that output the correct distance without discretization error. These algorithms rely on two main ideas: (a) the dual OT problem can be re-cast as the maximization of an expectation; (b) the entropic regularization of the primal OT problem yields a smooth dual optimization which can be addressed with algorithms that have a provably faster convergence. We instantiate these ideas in three different setups: (i) when comparing a discrete distribution to another, we show that incremental stochastic optimization schemes can beat Sinkhorn's algorithm, the current state-of-the-art finite dimensional OT solver; (ii) when comparing a discrete distribution to a continuous density, a semidiscrete reformulation of the dual program is amenable to averaged stochastic gradient descent, leading to better performance than approximately solving the problem by discretization ; (iii) when dealing with two continuous densities, we propose a stochastic gradient descent over a reproducing kernel Hilbert space (RKHS). This is currently the only known method to solve this problem, apart from computing OT on finite samples. We backup these claims on a set of discrete, semi-discrete and continuous benchmark problems.

This paper presents methods for the stochastic optimization of large-scale (regularized) optimal transport (OT), which is built upon the dual smoothed representation of entropic regularized OT [Cuturi and Peyré, 2016] (not cited). The paper shows that the dual smoothed objective (appeared in [Cuturi and Peyré, 2016]) can be rewritten differently as a finite separable sum, thus immediately admits stochastic methods by sampling one separate term at a time. As such, the paper explores three different settings of OT, where one or both measures are discrete or continuous, each with one or some illustrative numerical examples. In particular, it proposes to use SAG [16] for the discrete-discrete setting, SGD for the discrete-continuous setting, and RKHS parametrization for the continuous-continuous setting.   In this paper, the convergence complexities are stated inaccurate (see my explanation below), experiments are done very briefly, some important claims are not well justified. The three research problems attempted certainly go beyond an 8-page paper could embrace.   Cuturi, Marco, and Gabriel Peyré. "A smoothed dual approach for variational Wasserstein problems." SIAM Journal on Imaging Sciences 9.1 (2016): 320-343.
(Update!) The authors have responded to most of my questions in the rebuttal. Given I was the only one who gave a negative score, I would like to summarize my take on their responses.   1. They indeed resolves most of my concerns in the rebuttal, which could have been appeared in their original submission. I believe as long as my questions got cleared out, the paper will meet the threshold of technical quality that NIPS asks for. Particularly, I would like to see:    (a) Clarify that semi-dual formulation is not strongly convex. This message is not in the original paper and can be a plausible excuse that they don't specify the iteration complexity.  (b) Show CPU time comparison between SAG and Sinkhorn per iteration.  (c) Discuss the relation between their RKHS OT approach and the famous MMD kernel approach.  (d) Rephrase their contribution section to avoid inaccurate statements.  (e) Add sensitivity study on ɛ   Given the authors have carefully answered most of my questions, I will raise the technical quality score to 3. Other scores are unchanged.   2. Some views that I made in the review still remains unchanged. But I will let others to decide whether they are valid reasons to reject this paper.   (a) The originality is limited because the semi-dual formulation (new) is only one-step further than the smoothed dual formulation (already existed, but not clearly stated in the paper).  (b) For the D-D case, given there is no theoretical justification that SAG is better than Sinkhorn. The empirical justification is quite brief, not so convincing at this point.  (c) My view on "The authors study three problems in a single 8-page paper. Although they share the same optimal transport formulation, the actual approaches do not really hold any tight connections."   ***** As I summarized, this paper has many issues. Four major problems are   (1) The authors study three problems in a single 8-page paper. Although they share the same optimal transport formulation, the actual approaches do not really hold any tight connections. As a result, this paper leaves many questions unanswered for their approach on each problem, which I’d like to list them one by one later. Some questions I listed are very important, and should be answered to justify a valid algorithmic contribution beyond a half-baked idea.   (2) The formulation (Prop 2.1) are not completely new, Eq. (D_\varepsilon) and later Eq. (\bar{D}_\varepsilon)  already exist in [Cuturi and Peyré, 2016]. Thus, the originality is quite limited.   (3) Some claimed contributions of this paper are not properly justified or ambiguous.    (4) Experiments are not very useful for understanding the convergence properties of each algorithm.   As such, this paper could be improved by focusing on one of the three settings, clarifying and justifying its novel contributions,  and empirically demonstrating its superiority or validity more carefully. More detailed comments are as follows.  (a) Discrete-discrete setting. This is the most practical setting I want to focus in this review.   First of all, there is no evidence in the paper that the SAG approach gives a convergent solution for classical optimal transport. No experiments has done in this regard. Therefore, claims in Line 73 “These methods can be used to solve both classical OT problems and their entropic-regularized versions (which enjoy faster convergence properties). ” are not properly justified.   Second, I want to clarify the differences between SAG and SGD. SAG and many other variance reducing variants are quite different from SGD in terms of their iteration complexity. SAG is linearly convergent (or exponentially convergent ) for smooth, strongly convex problems, whose exponential rate depends on the data size N. SGD has much worse convergence rate, but the iteration complexity does NOT depend on the data size. As such, SAG is more closer to full gradient approach, rather than SGD (lucidly explained in [16]).    SAG, like Sinkhorn iterations, enjoys linear convergence for regularized optimal transport [4,6] due to the use of entropic regularization. In order to justify one is faster than another, one has to either show theoretically and empirically one has a better exponential rate than the other (Please see what has been done for evaluating SAG [16, Table 1], where comparison between SAG and full gradient are made.) Because the iteration complexity of SAG depends on the problem’s conditional number (strongly convex constant and Lipschitz constant), which actually depends on the regularizing parameter ɛ. The comparison of complexities becomes even subtle. Is it possible that SAG becomes slower than Sinkhorn iteration for some ɛ ? Therefore, the statements in Line 78 “while still enjoying a convergence rate O(1/k)” is inaccurate. On one hand, the paper implies SAG has very cheap computation complexity per iteration, which is true, on the other hand, it does not specify the iteration complexity accurately (the dependencies upon N, L and ɛ). In fact, SAG is more expensive in terms of data passes than Sinkhorn.   Third, the paper conducts an experiment comparing SAG and Sinkhorn iterations. Three flaws are (1) as mentioned earlier, sensitivity on ɛ is not explored. ɛ is a crucial parameter for making entropic regularization successful.  (2) empirical exponential rates are not compared.  (3) comparisons are made against passes over data, rather than the actual CPU/GPU time. The iteration of the SAG approach requires the computation of functions exp() and log(), which could be far more expensive than the Sinkhorn iteration. Therefore, the claims in Line 204 “SAG can be more than twice faster than Sinkhorn on average  …” is not proper.   Fourth, as mentioned in [Benamou, 2015] section 1.3, Sinkhorn iteration has a maximal possible iteration number for a small ɛ due to some quantities manipulated in Sinkhorn become smaller than machine precision. Does the SAG approach have this similar issue?  Benamou, Jean-David, et al. "Iterative bregman projections for regularized transportation problems." SIAM Journal on Scientific Computing 37.2 (2015): A1111-A1138.  (b) Discrete-Continuous Setting The paper claims in Line 80 “the SGD approach is numerically advantageous over the brute force approach consisting in sampling first …”. But their empirical validation is rather weak. Because (1) the experimental setting in which a huge number of iterations are used, is advantageous to SGD which converges rather slow. More iterations are clearly not useful for a problem with intrinsic discretization error, but one can increase N to reduce error just like the way SGD asks for more iterations. (2) The ground-truth v^\star “is approximated numerically by running SGD with a large enough number of iterations”. How large is large enough? Please specify the actual number. The logarithmic errors plotted in Figure 2 are known quite sensitive to the small changes of v^\star. As such, it makes the comparison even unfair. (3) Because entropic regularized OT is already an approximation to classical OT, I didn’t see much value to have a very accurate approximation to another approximation.   (c) Continuous-Continuous Setting I didn’t see any practical value or motivation in this approach. It does not compute the exact solution for regularized OT, and converges slowly.   In fact, if u and v are chosen from RKHS rather than some family of continuous functions, the dual OT problem is to compute maximum mean discrepancy, an extensively studied problem in the community of kernel methods. With RKHS embeddings, what’s the motivation to further consider the entropic regularization in regularizing the objective of maximum mean discrepancy? Please see for example [Gretton, 2012].  Gretton, Arthur, et al. "A kernel two-sample test." The Journal of Machine Learning Research 13.1 (2012): 723-773.   I try to give some positive words in the end. I think the SAG approach for Discrete-Discrete setting is an important direction and should be empirically investigated more if there is truly any practical value found in it. The complexity subtleties should be clarified and made easy and accessible to reader, so one can appreciate this idea to some degree. Simply put, for a regularized OT setup (N, L, ɛ), what is the iteration complexity? What is the computation time per iteration? How they compare to Sinkhorn?  Given an OT with m,n support sizes, the current approach mainly considers only one of n,m is large. What if both m and n are large? Is it possible to use doubly stochastic primal-dual approaches, e.g. [Yu, 2015] [Zhang, 2014]? Stochastic optimization of OT is an important direction, and I encourage the authors to improve their work, and re-submit with higher solidility in the future.   Yu, Adams Wei, Qihang Lin, and Tianbao Yang. "Doubly Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization with Factorized Data." arXiv preprint arXiv:1508.03390 (2015).  Zhang, Yuchen, and Lin Xiao. "Stochastic primal-dual coordinate method for regularized empirical risk minimization." arXiv preprint arXiv:1409.3257(2014).


Data Poisoning Attacks on Factorization-Based Collaborative Filtering
Recommendation and collaborative filtering systems are important in modern information and e-commerce applications. As these systems are becoming increasingly popular in the industry, their outputs could affect business decision making, introducing incentives for an adversarial party to compromise the availability or integrity of such systems. We introduce a data poisoning attack on collaborative filtering systems. We demonstrate how a powerful attacker with full knowledge of the learner can generate malicious data so as to maximize his/her malicious objectives, while at the same time mimicking normal user behavior to avoid being detected. While the complete knowledge assumption seems extreme, it enables a robust assessment of the vulnerability of collaborative filtering schemes to highly motivated attacks. We present efficient solutions for two popular factorizationbased collaborative filtering algorithms: the alternative minimization formulation and the nuclear norm minimization method. Finally, we test the effectiveness of our proposed algorithms on real-world data and discuss potential defensive strategies.

This paper investigates how attackers can inject malicious data into the training dataset of a recommendation system, in order to either maximise the error of the system, or increase/decrease the popularity of a subset of certain items. Specifically, they design a projective gradient ascent method (PGA) that optimises the attack strategy. They also propose a technique based on stochastic gradient Langevin dynamics (SGLD) optimization that can generate malicious data mimicking normal users. Experimental results show that PGA and SGLD are more effective than strawman uniform attacks.
The paper explores an important topic – adversarial machine learning. While the paper contributes interesting results, it seems slightly lacking in novelty/depth.  In general, the paper is well presented. Both the attack models and strategies are clearly derived and explained. It is indeed important to have this kind of analysis to fully understand the vulnerability of collaborative filtering schemes. New connections are made (to SGLD).  A main concern revolves around novelty/impact. Researchers have achieved numerous attacks on learners now (with most key references cited appropriately in the paper), demonstrating that the vulnerability of learning to poisoning is fundamentally well understood. Attacks tend to be formulated in the same way, as an optimisation on the prediction score function (or learning map; or the composition of the two); combined with some form of gradient descent, leaving just the calculation of (sub)gradients. It is interesting to follow through with this (for a learner of choice) if the attack has some closed form solution that can be defended against, or in some way provides a surprising conclusion – not the case here.  Formulating the different kinds of attacks on learners (availability, integrity, etc.) is not novel as a contribution. The paper should probably cite Barreno et al. (how secure is machine learning circa 2006 or so), or the papers that followed it up by the same authors.  Attacking the iterative fitting process (not just a global solution) appears more novel. However the calculations for the attacks, while very involved and non-trivial are not novel in themselves. For example exploiting KKT conditions (line 165) has been done by e.g. Mei and Zhu (AISTATS 2015) in the context of attacking LDA.  Mimicry attacks are well known e.g. Tan et al. (2002) were one of the earliest (applied to intrusion detection); Wagner & Soto (2002); Wang et al. (2006) the Anagram system. Formulating them as an optimisation is neat. The hybrid attack doesn't seem as well motivated as the pure strategies.  A point missing is a discussion on the feasibility of the attacks. Even if the attacker has complete knowledge of the target system, the overhead of computing the gradient does not seem to be trivial. In addition, the experimental results suggest that when the purpose is to maximise the system error, there is not much difference between PGA/SGLD and uniform attacks, until the percentage of poisoning samples reaches 10% or even 15% (Fig. 1 (a) (b), Fig. 2 (a) (b)). It isn't clear that the overhead (in computation, but more importantly sophistication) outweighs the gain. In practice, is it practical for attackers to insert that much malicious data? This is important, in the context of the next comment: that the uniform attack is a strawman.  Another problem with the experiment is that considering uniform attacks, both ratings and items are randomly chosen. This doesn’t make sense when the purpose is to boost/reduce the popularity of certain specific items – would a realistic baseline not bias sampling towards such items (or perform something like a dictionary attack).  Updating from author feedback: Thankyou for the response. While my position on novelty remains substantively the same, I would agree with the authors that the typical stance in cybersecurity is that worst-case attacks are indeed of interest. For one, "no security through obscurity" (we cannot guarantee robustness by assuming the attacker is unaware of training data) but also, the worst-case bounds the average case. That said, the most powerful attacks are those that demonstrate influence over a target system while leveraging minimal capabilities.


On Explore-Then-Commit Strategies
We study the problem of minimising regret in two-armed bandit problems with Gaussian rewards. Our objective is to use this simple setting to illustrate that strategies based on an exploration phase (up to a stopping time) followed by exploitation are necessarily suboptimal. The results hold regardless of whether or not the difference in means between the two arms is known. Besides the main message, we also refine existing deviation inequalities, which allow us to design fully sequential strategies with finite-time regret guarantees that are (a) asymptotically optimal as the horizon grows and (b) order-optimal in the minimax sense. Furthermore we provide empirical evidence that the theory also holds in practice and discuss extensions to non-gaussian and multiple-armed case.

This paper aims at providing theoretical evidence to support the fact that Explore-Then-Commit strategies are necessarily sub-optimal in terms of regret in a multi-armed bandit setting. In this regard the paper considers a simple two armed bandit problem with Gaussian rewards with different means but same variance. In this setting, the authors show lower bounds for ETC strategies. This shows that ETC strategies are necessarily sub-optimal compared to some strategies that follow fully sequential exploration. The paper also shows that a strategy similar to UCB* is asymptotically optimal and order optimal in the minimax sense at the same time.
1. One interesting part of the paper to me is the strategy in Algorithm 5 that is asymptotically optimal and optimal in the minimax sense. Although the strategy is not very original (a small modification of a strategy suggested by Lai), it is a new observation for a strategy to be asymptotically optimal and minimax optimal. However, it should be noted that it is optimal only in the case of two arms, which is a major drawback. The authors are encouraged to improve on this result.   2. I like the idea of using the hypothesis test result of Wald to come up with the ETC strategy in Algorithm 2 that achieves the lower bound for ETC strategy. This idea is new to the best of my knowledge. However, again it is challenging to extend this to multi-armed case.   3. I feel that Theorem 5 is rather straight forward to see from the results of the optimal BAI algorithm and there is not a lot of novelty in this result.   4. Although presented as the main result of this paper, I do not see very novel ideas in the proof of the lower bounds for these strategies. Hypothesis testing inequalities stemming from a particular inequality in Tsybakov's book " Introduction to Nonparametric Estimation" , has been used before in the bandit literature for proving lower bounds ". In fact the main step in the proof is Garivier's inequality. The parts of the proof that follows is fairly straight forward. Moreover the results have been presented only for the two arm setting.  5. Overall I like the flavor of the results presented in the paper. I like the Algorithm 5 and corresponding proof, as to the best of my knowledge this is original in the literature. The lower bounds for ETC are new results but may seem straight forward. The major drawback of the paper is the simple two arm setting and that there is not much practical significance of the paper. I would encourage the authors to generalize these results in future.   6. The presentation of the paper is very clear.


Improved Variational Inference with Inverse Autoregressive Flow
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables. We propose a new type of normalizing flow, inverse autoregressive flow (IAF), that, in contrast to earlier published flows, scales well to high-dimensional latent spaces. The proposed flow consists of a chain of invertible transformations, where each transformation is based on an autoregressive neural network. In experiments, we show that IAF significantly improves upon diagonal Gaussian approximate posteriors. In addition, we demonstrate that a novel type of variational autoencoder, coupled with IAF, is competitive with neural autoregressive models in terms of attained log-likelihood on natural images, while allowing significantly faster synthesis.

This paper proposes "inverse autoregressive flow", a technique that transforms a simple distribution into a more flexible posterior distribution for variational inference. IAF can be viewed as a simplified form of the normalizing flow (Rezende and Mohamed ,2015), where autoregressive functions are used as nonlinear transformations to make the posterior approximation for variational inference more flexible. The transformation has simple Jacobian determinant so it’s computationally efficient. In this paper, three variants of the IAF have been introduced and experimented: linear IAF, nonlinear IAF through MADE and nonlinear IAF through recurrent neural networks. The model is validated on MNIST and CIFAR-10 database. The proposed VAE+IAF outperforms vanilla VAE with simple diagonal Gaussian distribution in terms of log-likelihood. The samples generated from the VAE trained with IAF prior look less blurry compared to previous work using VAE.
This paper proposes a general solution to the variational auto-encoders (VAE) that enables more complicated and structured prior distribution. The experimental comparisons with previous work are solid enough. The paper is in general clearly written but the writing and formulation still needs polishing.  -- Line 150 -- 151 "where at each step we use a differently parametrized auto-regressive model \mathcal{N}(\mu^t, \sigma^t)" My understanding is that only z^0 is generated by diagonal Gaussian distribution \mathcal(\mu^0, \sigma^0), while the remaining z^t is obtained by transformation step z^t = L(x) \cdot z^{t-1}. Can you clarify the pipeline for generating a sample from VAE+IAF?  -- Line 158 -- Line 159: The current notation is quite confusing. The z^t has been defined in Eq.11 but not used here. It would be more clear if the author provide the final form rather than using "y" here. Also, "y" is often used for label rather than latent variable.   -- How many IAF steps are used in the experiments? The only clue I find is the notation "z^l" (line 227), where "l" is the l-th block (1 = 1..L). Please  elaborate on this point.  -- How does the proposed IAF compared to normalization flow (Rezende and Mohamed, 2015)? The previous work (Figure 3 of that paper) clearly demonstrated that proposed technique produces better approximation to true posterior. Please comment on the performance of IAF.


Preference Completion from Partial Rankings
We propose a novel and efficient algorithm for the collaborative preference completion problem, which involves jointly estimating individualized rankings for a set of entities over a shared set of items, based on a limited number of observed affinity values. Our approach exploits the observation that while preferences are often recorded as numerical scores, the predictive quantity of interest is the underlying rankings. Thus, attempts to closely match the recorded scores may lead to overfitting and impair generalization performance. Instead, we propose an estimator that directly fits the underlying preference order, combined with nuclear norm constraints to encourage low-rank parameters. Besides (approximate) correctness of the ranking order, the proposed estimator makes no generative assumption on the numerical scores of the observations. One consequence is that the proposed estimator can fit any consistent partial ranking over a subset of the items represented as a directed acyclic graph (DAG), generalizing standard techniques that can only fit preference scores. Despite this generality, for supervision representing total or blockwise total orders, the computational complexity of our algorithm is within a log factor of the standard algorithms for nuclear norm regularization based estimates for matrix completion. We further show promising empirical results for a novel and challenging application of collaboratively ranking of the associations between brain-regions and cognitive neuroscience terms.

The submission proposes a new approach for preference completion. Frequently used methods are based on low rank matrix completion or the `learning to rank' formulation, both of which rely heavily on the scores provided by the user. In contrast, the proposed approach does not directly rely on scores but rather on rankings between pairs of elements. Hence scores can be transformed in an arbitrary but monotonic way. More generally, the proposed approach is also directly applicable to given partial rankings, without any transformation to scores.  To this end, a set of vectors is defined which fulfill the provided ordering with at least an epsilon-margin. The estimator for the preference matrix is then obtained by minimizing its nuclear norm and its deviation from the entries in the set of vectors fulfilling the provided ordering.  A proximal gradient descent algorithm is suggested which results in two steps, (i) a singular value thresholding when optimizing w.r.t. the preference matrix, and (ii) a projection onto the feasible set of valid rankings. The latter projection is equivalent to isotonic regression, hence it can be solved using the `Pool Adjacent Violators' algorithm if we consider strict or blockwise total ordering of preferences. A more complex algorithm has to be used in case we are given an arbitrary (consistent) ordering.  The algorithmic complexity of the proposed algorithm is composed out of two parts, (i) singular value thresholding, which is also required in standard matrix completion techniques, and (ii) projection onto the feasible set, which is feasible in simple ordering cases but expensive generally.  The proposed approach is evaluated on the Movielens dataset and a neurosynth dataset and shown to outperform simple baselines such as standard low-rank matrix completion and Cofi-Rank.
Review summary: -------- The proposed technique does not directly take into account the provided preference scores which is a clear advantage since scores may often be noisy. However the proposed approach is generally very expensive (an experimental comparison hasn't been shown) and it might not handle inconsistent ordering very well. The set of feasible vectors may indeed be empty and no mechanism is provided for the algorithm to fail gracefully.  Technical quality: the feasible set might be empty if confusing rankings are provided as input, which hasn't been analyzed Novelty: an interesting technique for ranking Impact: a comparison with recent methods is missing Clarity: well described generally  Review details: -------- 1. The defined set of vectors fulfilling the provided ordering may be empty or contain only a few elements if the provided groundtruth ordering is not consistent. The proposed approach does not seem to fail gracefully in this setting, i.e., a confusing ranking may actually remove good solutions. Did the authors investigate the properties of the proposed algorithm in this setting?  2. While Lemma 2 is interesting, it's important to relate it back to the original problem. As a reader I'm therefore wondering whether the proposed approach still optimizes the original formulation.  3. Some more intuition could according to my opinion be provided by shortining the intro. In particular the description of the generalization bound is very short and follows from existing literature. This could be stated more clearly. Was the symbol `d' defined?  4. In the experimental evaluation, classical baselines are employed. As a reader I'm wondering about more recent approaches and a comparison w.r.t. the proposed algorithm. Can the authors comment?  5. Ideally it should also be evaluated whether it's worth to spend the additional effort of projecting onto the feasible set. An evaluation of the computation time would be desirable.  Minor comments: -------------- - typo in Eq. (2)? Shouldn't there be a minimization over z? - possible typo in Prop. 1? It seems like the second \gamma in the last equation shouldn't be inverted. Didn't check completely though. - possible typo in Lemma 2: a minus sign may be missing in the characterization of the constraint set to make it consistent with the definition of the feasible set in line 218. - the writing should be corrected here and there


Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Recurrent Neural Networks (RNNs) have become the state-of-the-art choice for extracting patterns from temporal sequences. However, current RNN models are ill-suited to process irregularly sampled data triggered by events generated in continuous time by sensors or other neurons. Such data can occur, for example, when the input comes from novel event-driven artificial sensors that generate sparse, asynchronous streams of events or from multiple conventional sensors with different update intervals. In this work, we introduce the Phased LSTM model, which extends the LSTM unit by adding a new time gate. This gate is controlled by a parametrized oscillation with a frequency range that produces updates of the memory cell only during a small percentage of the cycle. Even with the sparse updates imposed by the oscillation, the Phased LSTM network achieves faster convergence than regular LSTMs on tasks which require learning of long sequences. The model naturally integrates inputs from sensors of arbitrary sampling rates, thereby opening new areas of investigation for processing asynchronous sensory events that carry timing information. It also greatly improves the performance of LSTMs in standard RNN applications, and does so with an order-of-magnitude fewer computes at runtime.

LSTMs and related models (e.g GRUs) have become the workhorses of neural network research and applications involving sequences. The key idea of LSTMs is to use gating over the memory cells to mitigate the "vanishing gradients" problem that occurs in standard recurrent networks, allowing the network to learn tasks that require persisting memories over many timesteps.  This work proposes an extension to LSTMs, adding "time gates" which control updates to the memory. These gates are controlled by oscillators, which only allow updates to the memory during certain points in the oscillation. The periodicity and phase-shift of the gates are learned during training. One additional property these time gates have, is that as they are controlled by a continuous-time oscillator, the value of the gates can be sampled at irregular time intervals.  These phased LSTMs are tested in 4 experiments: classifying sinusoidal signals (a task tailored to the properties of this model), adding task (similar to the adding task in the original LSTM paper), event-based encoding of MNIST and a lip-reading dataset. In all of these tasks the phased LSTMs outperform the original LSTM.
LSTMs (and GRUs) are increasingly used as basic building blocks in neural network architectures, both in inherently sequential problems but also in other applications as many other problems can usefully be decomposed into sequential problems using mechanisms such as attention. Despite being devised some time ago, LSTMs have proved to be difficult to beat as a general purpose tools for modeling sequential structures (e.g. [1]).  This paper presents an interesting idea for improving the performance of LSTMs, particularly on tasks which contain cyclical structure. It is novel and explains the model and motivations well. There are aspect of the analysis and experimental results which could be improved on, but it is a novel approach that will be of interest to the field. I have several suggestions for improvements below, but these do not significantly detract from the work, which is of a high standard.  One aspect that should be clarified is exactly how time-sampling was performed in the cases where sampling was "asynchronous." It's clear that k_t can be computed for any time t, but the remainder of the LSTM still appears to be a discrete-time update (for example, two updates in quick succession when k_t ~ 1 will affect c_t differently than only 1 update is sampled during the k_t~1 phase).  It would be very interesting if the authors could include an any analysis or insight into the solution their model is learning on simple tasks such as adding or classifying sinusoids.  The manuscript mentions the potential for reduced computational load, particularly for high-frequency signals, due to the gating alleviating the need to compute update to the cell when k_t = 0. It would be ideal if this was supported by empirical analysis in one of the experiments at test time (where alpha = 0).  Although realized in a different way, these ideas bear some resemblance to reservoir computing. In particular, both maintain memories using fixed oscillators while learning the connections into/out of the oscillators. It would be helpful to cite some of this work and mention this connection/distinctions.  The choice of experimental comparisons is somewhat eclectic. It certainly makes sense to compare with one of the original LSTM tasks (adding). It would help to assess the potentially impact of this work if it were used on a larger, standard sequential benchmark that doesn't necessarily contain obvious oscillatory structure. For example, a language modeling or machine translation task. would improve the model substantially to compare with a more commonly used sequential benchmark (e.g. [3]).  One aspect of the model which the authors mention is the tendency to preserve memories over many timesteps due the gating effect of the time gates (particularly if tau is large). One might hope to obtain something similar in a standard LSTM by heavily biasing some of the forget gates to 1 and the input gates to 0. It would be a useful comparison to check that simple initialization tricks can't reproduce the results.  [1] LSTM: A Search Space Odyssey http://arxiv.org/pdf/1503.04069v1.pdf [2] A clockwork RNN. http://arxiv.org/abs/1402.3511 [3] Recurrent neural network regularization http://arxiv.org/pdf/1409.2329v5.pdf   ** After feedback ** Due to the large number of reviews, there wasn't space to address my comments in depth. Overall, my ratings are unchanged (they were already high), but I agree with the bulk of the other reviewers this is a good paper.


Faster Projection-free Convex Optimization over the Spectrahedron
Minimizing a convex function over the spectrahedron, i.e., the set of all d ⇥ d positive semidefinite matrices with unit trace, is an important optimization task with many applications in optimization, machine learning, and signal processing. It is also notoriously difficult to solve in large-scale since standard techniques require to compute expensive matrix decompositions. An alternative is the conditional gradient method (aka Frank-Wolfe algorithm) that regained much interest in recent years, mostly due to its application to this specific setting. The key benefit of the CG method is that it avoids expensive matrix decompositions all together, and simply requires a single eigenvector computation per iteration, which is much more efficient. On the downside, the CG method, in general, converges with an inferior rate. The error for minimizing a -smooth function after t iterations scales like /t. This rate does not improve even if the function is also strongly convex. In this work we present a modification of the CG method tailored for the spectrahedron. The per-iteration complexity of the method is essentially identical to that of the standard CG method: only a single eigenvector computation is required. For minimizing an ↵-strongly convex and -smooth function, the expected error of the method after t iterations is:
) are the rank of the optimal solution and smallest nonzero eigenvalue, respectively. Beyond the significant improvement in convergence rate, it also follows that when the optimum is low-rank, our method provides better accuracy-rank tradeoff than the standard CG method. To the best of our knowledge, this is the first result that attains provably faster convergence rates for a CG variant for optimization over the spectrahedron. We also present encouraging preliminary empirical results.
where rank(X ⇤ ), min (X ⇤ ) are the rank of the optimal solution and smallest nonzero eigenvalue, respectively. Beyond the significant improvement in convergence rate, it also follows that when the optimum is low-rank, our method provides better accuracy-rank tradeoff than the standard CG method. To the best of our knowledge, this is the first result that attains provably faster convergence rates for a CG variant for optimization over the spectrahedron. We also present encouraging preliminary empirical results. of a real symmetric matrix. Such a decomposition requires O(d 3 ) arithmetic operations for a d ⇥ d matrix and thus is prohibitive for high-dimensional problems. An alternative is to use first-order methods that do not require expensive decompositions, but rely only on computationally-cheap leading eigenvector computations. These methods are mostly based on the conditional gradient method, also known as the Frank-Wolfe algorithm [3,12], which is a generic method for constrained convex optimization given an oracle for minimizing linear functions over the feasible domain. Indeed, linear minimization over the spectrahedron amounts to a single leading eigenvector computation. While the CG method has been discovered already in the 1950's [3], it has regained much interest in recent years in the machine learning and optimization communities, in particular due to its applications to semidefinite optimization and convex optimization with a nuclear norm constraint / regularization 1 , e.g., [10,13,17,19,22,2,11]. This regained interest is not surprising: while a full eigen-decomposition for d ⇥ d matrix requires O(d 3 ) arithmetic operations, leading eigenvecor computations can be carried out, roughly speaking, in worst-case time that is only linear in the number of non-zeros in the input matrix multiplied by either ✏ 1 for the popular Power Method or by ✏ 1/2 for the more efficient Lanczos method, where ✏ is the target accuracy. These running times improve exponentially to only depend on log(1/✏) when the eigenvalues of the input matrix are well distributed [14]. Indeed, in several important machine learning applications, such as matrix completion, the CG method requires eigenvector computations of very sparse matrices [13]. Also, very recently, new eigenvector algorithms with significantly improved performance guarantees were introduced which are applicable for matrices with certain popular structure [5,8,20].
The main drawback of the CG method is that its convergence rate is, in general, inferior compared to projection-based gradient methods. The convergence rate for minimizing a smooth function, roughly speaking, scales only like 1/t. In particular, this rate does not improve even when the function is also strongly convex. On the other hand, the convergence rate of optimal projection-based methods, such as Nesterov's accelerated gradient method, scales like 1/t 2 for smooth functions, and can be improved exponentially to exp( ⇥(t)) when the objective is also strongly convex.
Very recently, several successful attempts were made to devise natural modifications of the CG method that retain the overall low per-iteration complexity, while enjoying provably faster convergence rates, usually under a strong-convexity assumption, or a slightly weaker one. These results exhibit provablyfaster rates for optimization over polyhedral sets [7,15] and strongly-convex sets [6], but do not apply to the spectrahedron. For the specific setting considered in this work, several heuristic improvements of the CG method were suggested which show promising empirical evidence, however, non of them provably improve over the rate of the standard CG method [19,17,4].
In this work we present a new non-trivial variant of the CG method, which, to the best of our knowledge, is the first to exhibit provably faster convergence rates for optimization over the spectrahedron, under standard smoothness and strong convexity assumptions. The per-iteration complexity of the method is essentially identical to that of the standard CG method, i.e., only a single leading eigenvector computation per iteration is required. Our method is tailored for optimization over the spectrahedron, and can be seen as a certain hybridization of the standard CG method and the projected gradient method. From a high-level view, we take advantage of the fact that solving a`2-regularized linear problem over the set of extreme points of the spectrahedron is equivalent to linear optimization over this set, i.e., amounts to a single eigenvector computation. We then show via a novel and non-trivial analysis, that includes new decomposition concepts for positive semidefinite matrices, that such an algorithmically-cheap regularization is sufficient, in presence of strong convexity, to derive faster convergence rates.

This paper provides a new method for strongly convex smooth minimization over the spectrahedron under Linear Minimization Oracle with provable rate of convergence. The expected error of this method in some regimes is better than for the existing conditional-gradient-type methods. The performance gain of the proposed approach in comparison to existing methods is demonstrated on a problem of matrix completion.
Initial review.  Technical quality:whether experimental methods are appropriate, proofs are sound, results are well analyzed. Besides the found errors in the proofs, I have the following comments which, I hope explain my evaluation. 1. The claim in the Lemma 1, that $b$'s form a distribution is not proved and doesn't hold. But I didn't find a place, where this claim is used in the sequel. 2. It is not clear why, for the numerical experiments, the authors choose a heuristic modification of their algorithm, despite that it doesn't have a proved convergence theorem. It would be interesting also to see how the provable method works. 3. Perhaps, the averaging over 5 independent experiments does not say much about the advantages of the new method.  Novelty/originality: in any aspect of the work, theory, algorithm, applications, experimental. I find the proposed approach new and interesting. It seems that it works at least in practice. The problem is the theoretical justification.  Potential impact or usefulness: could be societal, academic, or practical and should be lasting in time, affecting a large number of people and/or bridge the gap between multiple disciplines. Preliminary experiments show that this method could potentially solve practical problems faster. I also find the idea of rank-one regularization worth noting. The problem is the theoretical justification.  Clarity and presentation: explanations, language and grammar, figures, graphs, tables, proper references. My overall impression is quite positive. But several places were unclear for me and should be improved. 1. It would be nice to explain, why the last  identity in the first line of (6) holds.  2. It seems that in (6) the minimization is over the set of all unit vectors. But it is not stated there. 3. There are some misprints, e.g. on the line 179 "proprieties" should be "properties". 4. I could not understand, why in the long derivation between lines 346 and 347,  the first inequality holds. 5. It is not clear form the provided explanation, why the bound for the first term in the last inequality in (17) holds. 6. It is not clear, why the inequality on the line 410 holds. 7. It is not clear, why the inequality on the line 418 holds.     ================================================ Update after author feedback.  Technical quality: I still have some doubts about the numerical experiments. The authors choose a heuristic modification of their algorithm, despite that it doesn't have a proved convergence theorem. Their explanations about line-search modification seem to be reasonable. Still, in the experiment part, they use greedy choice of the rank-one matrix instead of random choice for the provable algorithm. It is not clear, why one should do that.  Novelty/originality:  I find the proposed approach new and interesting. It seems that it works at least in practice. The problem is the theoretical justification.  Potential impact or usefulness:  Preliminary experiments show that this method could potentially solve practical problems faster. I also find the idea of rank-one regularization worth noting. The problem is the theoretical justification.  Clarity and presentation:  Unfortunately, there were too many places in the text, which were unclear for me. In their feedback, the authors tried to explain them, but at least the explanations for items 4 and 5, are still not clear. Also I add the points, which I considered as errors in the first version of my review. I think they should also be clarified in the text. 1. It would be nice to explain in the text, why the last identity in the first line of (6) holds.  2. It seems that in (6) the minimization is over the set of all unit vectors. But it is not stated there. 3. There are some misprints, e.g. on the line 179 "proprieties" should be "properties". 4. I could not understand, why in the long derivation between lines 346 and 347, the first inequality holds. 5. It is not clear form the provided explanation, why the bound for the first term in the last inequality in (17) holds. 6. It would be nice to explain in the text, why the inequality on the line 410 holds. 7. It would be nice to explain in the text, why the inequality on the line 418 holds. 8. The claim in Lemma 1, that $b$'s form a distribution should be deleted. 9. I think, it still should be explained in the text, why ||A*B||_F < = ||A||_{spectral}*||B||_F and why ||P_{Y,\tau}||_{spectral} \leq 1. 10. I think, it should be explained in the text, how the third inequality in (19) follows from the optimality of $w^*$. 11. The estimate (25) should also be explained more explicitly. Unfortunately, the explanation in the author's feedback did not help me to understand, how it is proved.


Improving PAC Exploration Using the Median of Means
We present the first application of the median of means in a PAC exploration algorithm for MDPs. Using the median of means allows us to significantly reduce the dependence of our bounds on the range of values that the value function can take, while introducing a dependence on the (potentially much smaller) variance of the Bellman operator. Additionally, our algorithm is the first algorithm with PAC bounds that can be applied to MDPs with unbounded rewards.

The paper proposes a PAC algorithm for reinforcement learning in MDPs with potentially finite state and action spaces. The key difference to existing work is that this method uses median of means instead of the usual empirical mean to estimate the expected value of the next state in the Bellman operator. This allows to provide PAC bounds (in the form of TCE bounds) that only depend log-linearly on the range of the value function while introducing dependency on the variance of the values of successor states.
The authors' feedback clarified misunderstandings I had and addressed my concerns about usefulness of the presented results. With the remaining issues with clarity being removed, this paper can be a really nice contribution for the NIPS community.    Original Review: ---------------- The idea of using the medians of means estimator to obtain stronger PAC guarantees for reinforcement learning algorithms seems very appealing to me and the presented results appear plausible. Yet, I see two main issues with this paper: First, there is a lack of clarity and insufficient explanations and second, the usefulness of the presented  PAC guarantee is not entirely clear to me.  Usefulness: * The authors state in line 40 on page 2 that their algorithm does not depend on the number of successor states which can even be unbounded. However, the sample complexity bound depends linearly on the number of states. For unbounded number of successor states and therefore unbounded number of states, the sample complexity bound becomes meaningless. * Definition 2.1 states that Q_max is an upper bound on the value of any state (for any policy I assume) but no assumptions have been made about a lower bound. I therefore wonder whether adding a constant to every reward (which should not affect the difficulty of the RL problem) affects the bound in Theorem 1 or not. Is it assumed that rewards / all values are nonnegative? * Definition 5.1 requires sigma^2 to be an upper bound on the variance of the value of the next state for any value-function and policy. This seems rather demanding and raises the question when sigma^2 is actually  significantly smaller than (Q_max/2)^2 especially discount factors close to 1. For gamma close to 1, it is sufficient that there is a (s,a) pair and a subset S' of S so that p(s' in S' | s, a) \approx 1/2 for sigma to be close to (Q_max/2)^2 (by choosing \tilde Q and pi appropriately). This does seem to be the case in most relevant MDPs.  Clarity and presentation: * The paper claims the algorithm is PAC-optimal (w.g. line 200). However, a direct comparison to a lower-bound is not provided which makes it not clear to me why this algorithm has indeed optimal (= matching a lower bound up to constants and log-terms) sample-complexity. * Knowing a nontrivial bound on sigma is unrealistic, especially ahead of executing the algorithm. However, the current bound assumes parameters k, k_m to be chosen depending on sigma. The bound in Theorem 5.2 is therefore rather artificial. It is stated in line 199 that the bound can also be formulated for the more relevant case of choosing k, k_m fixed but not actually provided in the paper or appendix.  * The quantities k_m, k and epsilon_s are only defined implicitly (e.g. k_m appears on both sides of the equality sign. It is therefore hard to identify how changes in problem characteristics affect the provided sample complexity bound. These dependencies should be disentangled to make the dependency of the bound on the important quantities: 1/(1-gamma), |S|, |A|, Q_max and sigma clear. * The provided proof sketch discusses the general structure which is very similar to the analysis of most other Optimism-in-the-face-of-uncertainty methods. However, it provides no insight where the median of means actually changes the proof and allows (potentially) tighter bounds. While Section 3 is very helpful for understanding why median of means can be beneficial, more details on how this is leveraged in PAC RL would be very much appreciated.  Minor things: * Should the value of of n in line 110.5 not depend on 2/delta instead of 1/delta as each of the two failure cases has to be bounded by 1- delta/2? * Algorithm 1: \epsilon_a is also an input / parameter of the algorithm * Line 20 of Algorithm 1 redefines i in the sum, which is already defined in line 16


Structured Prediction Theory Based on Factor Graph Complexity
We present a general theoretical analysis of structured prediction with a series of new results. We give new data-dependent margin guarantees for structured prediction for a very wide family of loss functions and a general family of hypotheses, with an arbitrary factor graph decomposition. These are the tightest margin bounds known for both standard multi-class and general structured prediction problems. Our guarantees are expressed in terms of a data-dependent complexity measure, factor graph complexity, which we show can be estimated from data and bounded in terms of familiar quantities for several commonly used hypothesis sets along with a sparsity measure for features and graphs. Our proof techniques include generalizations of Talagrand's contraction lemma that can be of independent interest. We further extend our theory by leveraging the principle of Voted Risk Minimization (VRM) and show that learning is possible even with complex factor graphs. We present new learning bounds for this advanced setting, which we use to design two new algorithms, Voted Conditional Random Field (VCRF) and Voted Structured Boosting (StructBoost). These algorithms can make use of complex features and factor graphs and yet benefit from favorable learning guarantees. We also report the results of experiments with VCRF on several datasets to validate our theory.

This paper studies generalization bounds for structured prediction. In line with previous work, the authors adopt a structured prediction model based on a scoring function. In particular they allow for such function to factorize in terms of scoring function that account only for a smaller subset of the output coordinates. A variation of the classical Rademacher complexity is proposed, based on this formulation and generalization bounds on the generalization error are discussed. The authors draw connections with previous work and then present an optimization algorithm to approach structured prediction problems.
The paper is well written and motivated. In particular the problem considered is relevant. On the downside there are some issues related to the interpretability of the presented results:  - In Theorem 1 the generalization error is bounded in terms of the additive or multiplicative empirical margin losses. However their formulation at Eq. (5) and (6) is hard to interpret and would benefit from a comment. This is problematic since it is not clear how these quantities are related to the algorithmic approaches discussed in Sec. 5. In particular it is not clear how the approaches described in Sec. 5 are related to minimizing the additive or multiplicative empirical margin loss.  [EDIT]: I thank the authors' for clarifying the role of the additive and multiplicative empirical margin loss in perspective of the algorithms proposed in Sec. 5. I feel the   - Theorem 2 is quite hard to parse. In particular the role of sparsity factor s and its dependence on the maximum cardinality of each factor is not clear and deserves a deeper analysis. Indeed, in the worse case scenario it is bounded by m max_i |F_i| d_i, but in the case of NLP described in Line 192, max_i |F_i| = +\infty if I understood correctly the model (since sentences can have arbitrary length). This means that in the case of infinite dimensional output spaces only factor graphs with uniformly bounded cardinality can be used. This limit the applicability of the result and should be discussed.  [EDIT]: the authors reply did not clarify my doubts. In particular, when Y is infinite: 1) why a factor graph F should have finite cardinality? and 2) the dependency of max_i |F_i| from the number of examples "m" could dramatically affect the bound.   However, most structured prediction settings assume Y of finite cardinality so this is a second-order issue.  - Section 3.3 draws connection with previous work. While it is interesting and relevant to understand how previous results can be cast in the setting considered, the discussion is difficult to follow. For instance, in the case of the multi-class classification the paper does not provide a direct formulation of the problem within their notation and therefore it is not clear how to interpret the theoretical results in this setting. Moreover, to connect to previous work the authors use results (e.g. Lemma 6), which are reported in the appendices and not directly connected to the main results of this paper (Theorem 1 and 2). This makes the discussion harder to follow.  - Sec. 5: as mentioned above, one problem of the discussion in Sec. 5 is that it lacks a direct connection with the theoretical results presented in this work. A further issue is that the authors do not comment on the relation between the ideal empirical (non-convex) problem and the convex surrogates that are reviewed/proposed. In particular it would be interesting to understand how tight the relation of the VCRF method is with specific losses. For instance in the case of multi-class classification it is known that StructSVM - which belongs to the general framework proposed in Eq. (9) - is not consistent.   Minor notes:  - Line 158: the authors state that the empirical factor graph Rademacher complexity is sharply concentrated around its expectation but I was not able to find a proof of this in the supplementary material.   - Line 168: "linear functions", did the authors mean "linear spaces"?  - Line 296 - 297: the authors used \delta \psi(x,y,y') which was never defined.


Improved Dropout for Shallow and Deep Learning
Dropout has been witnessed with great success in training deep neural networks by independently zeroing out the outputs of neurons at random. It has also received a surge of interest for shallow learning, e.g., logistic regression. However, the independent sampling for dropout could be suboptimal for the sake of convergence. In this paper, we propose to use multinomial sampling for dropout, i.e., sampling features or neurons according to a multinomial distribution with different probabilities for different features/neurons. To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization. By minimizing a sampling dependent factor in the risk bound, we obtain a distribution-dependent dropout with sampling probabilities dependent on the second order statistics of the data distribution. To tackle the issue of evolving distribution of neurons in deep learning, we propose an efficient adaptive dropout (named evolutional dropout) that computes the sampling probabilities on-the-fly from a mini-batch of examples. Empirical studies on several benchmark datasets demonstrate that the proposed dropouts achieve not only much faster convergence and but also a smaller testing error than the standard dropout. For example, on the CIFAR-100 data, the evolutional dropout achieves relative improvements over 10% on the prediction performance and over 50% on the convergence speed compared to the standard dropout.

This paper proposes a new dropout scheme based on sampling features or neurons according to a multinomial distribution with different probabilities for different features or neurons.  For shallow learning, it proposes a data-dependent dropout by using the second-order statistics of the features to compute the sampling probabilities.  Theoretical analysis of the risk bound is provided.  For deep learning, it proposes an evolutional dropout by using the second-order statistics of each layer’s output based on a mini-batch of examples (for the purpose of reducing the computational demand) to compute the sampling probabilities of dropout for that layer.  Some experiments are presented to compare the proposed distribution-dependent dropout with standard dropout for both shallow and deep learning and with batch normalization.
TECHNICAL QUALITY  It is discussed in Section 1 (lines 31-33) that features with low/zero variance can be dropped more frequently or even completely.  How is this intuition supported by the theoretical analysis in Section 4 (particularly Eq. (8) or (9))?  Note that the features are not automatically zero-meaned.  The uniform dropout scheme described in line 135 (as a special case of multinomial dropout when all the sampling probabilities are equal) is only similar but not identical to standard dropout (the sampling probabilities for different features are not i.i.d.).  It may not be good to use it in the experiments as if it is indeed the standard dropout scheme.  There are some other concerns about the experiments reported.  Although the emphasis of this paper is not on obtaining better prediction performance by trying different network structures and tricks (line 265-266), it aims at improving standard dropout and hence is natural to also include other improved dropout schemes as baselines in the comparative study.  Such baselines should at least include the adaptive dropout method in Ba and Frey’s NIPS 2013 paper (which should have been cited) and the variational dropout method by Kingma et al. [6].  The experiments consider only one dropout rate (0.5).  Different dropout rates should be reported to give a more complete picture.  For shallow learning, I suggest that the author also includes a setting for s-dropout using data after Z-normalization.  As noted in the remark in Section 4.1 (lines 213-217), its performance is expected to be similar to that of d-dropout.  In Section 5.3, the paper concludes that e-dropout is roughly a randomized version of batch normalization plus standard dropout.  This seems to imply that combining e-dropout with batch normalization will not lead to further improvement in performance.  I suggest that this combination be also included in the comparison.  Also, from Figure 2 (right), as the number of iterations increases the test accuracy of e-dropout fluctuates more than other methods, including BN+dropout which, like e-dropout, is also a randomized scheme.  This seems to show that e-dropout is slightly worse than BN+dropout as far as learning stability is concerned.  Some discussions should be provided on this.  The performance gap between s-dropout and e-dropout is not always as large as that between s-dropout (without Z-normalization) and d-dropout.  In terms of the final test accuracy, e-dropout does not always win.  In fact, it seems to be worse than s-dropout for MNIST.  NOVELTY  The main theoretical result in the paper is Theorem 1.  Its proof is based on standard techniques for stochastic gradient descent.  The extension from shallow learning to deep learning is straightforward in that it simply uses mini-batches instead of the full data set to establish the sampling probabilities for each layer separately.  No theoretical guarantee is available for the deep learning case.  Nevertheless, as far as I know, this is the first data-dependent dropout method proposed with theoretical justification though only for the shallow learning case.  IMPACT  From the experiments, the performance gap between s-dropout and e-dropout is not always as large as that between s-dropout and d-dropout.  One may question whether the improvement obtained by distribution-dependent dropout will still be significant for deeper networks.  Compared with more realistic deep learning applications which need much deeper networks, I am afraid only small-scale experiments are presented for deep learning in the paper.  Consequently, the potential impact that this work can bring to the deep learning community is unclear.  CLARITY & PRESENTATION  The paper is generally well organized and easy to read.  Nevertheless, the writing has room to improve.  Besides problems with English usage, there are also some language/formatting errors.  Here are some examples: L22-23: “at random samples neurons and sets their outputs to be zeros” L32: “can dropout” L58: “acts similar to” L67: “a faster convergence” L73: “reminder” L80,85: seem to be errors when using citation in LaTeX L86: “to speed-up” L93-94: “developed evolutional dropout” L101: “in next section” L111: “where he operator” L135: “we refer as” L167-168: “a non-convex optimization” L172: “does not loss the generality” L179: “upper bond” L181: “The detailed proof of theorem” L185-186: “the second term also depend on” L191: “included in supplement”


Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
In this paper, we propose a very deep fully convolutional encoding-decoding framework for image restoration such as denoising and super-resolution. The network is composed of multiple layers of convolution and deconvolution operators, learning end-to-end mappings from corrupted images to the original ones. The convolutional layers act as the feature extractor, which capture the abstraction of image contents while eliminating noises/corruptions. Deconvolutional layers are then used to recover the image details. We propose to symmetrically link convolutional and deconvolutional layers with skip-layer connections, with which the training converges much faster and attains a higher-quality local optimum. First, the skip connections allow the signal to be back-propagated to bottom layers directly, and thus tackles the problem of gradient vanishing, making training deep networks easier and achieving restoration performance gains consequently. Second, these skip connections pass image details from convolutional layers to deconvolutional layers, which is beneficial in recovering the original image. Significantly, with the large capacity, we can handle different levels of noises using a single model. Experimental results show that our network achieves better performance than recent state-of-the-art methods.

This paper proposed a deep fully convolutional neural network for image restoration such as denoising and super-resolution. The network is composed of multiple layers of convolution and de-convolution operators, learning end-to-end mappings from corrupted images to the ground-truth. In addition, symmetrical links are used to connect the corresponding convolution and de-convolution layers. Experiments results show that the proposed method outperforms the state-of-the-arts on image denoising and super-resolution.
The proposed method is very related to Kim et al.’s work, but the later one is not mentioned at all. At the time of submission, the arXiv version of Kim’s paper was already online, which should be cited and discussed.  Kim, Jiwon, Jung Kwon Lee, and Kyoung Mu Lee. "Accurate image super-resolution using very deep convolutional networks." arXiv preprint arXiv:1511.04587 (2015). Accepted at CVPR 2016.    In Kim et al.’s work, the model is with multiple flat convolution layers. The input is connected with the output to form a residual learning. Compared with Kim’s model, half of convolutional layers are replaced with de-convolution layers in the proposed model and it has more skip connections between the layers. It seems that the overall idea is very similar to Kim’s paper that using very deep fully convolutional layers and residual learning for image restoration. Overall, there is some advancement made by the paper, but may be not significantly enough for a NIPS paper.   1. It is claimed that this paper different corruptions using a single model. While it provides some contribution in denoising, the strategy of mixed training set has already used in Kim’s paper for different levels of super-resolution. Therefore, the contribution is not that significant.   2. In section 2.2, to validate the benefits of skip connections, it is not very convincing by comparing residual networks [11] since [11] was proposed for recognition. A more proper choice could be the version of removing the connections in the middle, with only one input-to-end connection. It is like the residual learning in Kim’s work. This experiment can validate the necessity of the middle connections.   3. I think it is weird for a paper on image restoration without any visual comparisons in the paper.  although these results are included in the supplementary material. For some results, the improvements are not very clear. Highlighting some zoomed-in regions would be helpful.    Minor:  1. Tables 1,2,3 and 6,7 have the same captions, but actually they are trained with different strategies. The captions should be changed to make such difference clear without reading the text in the paper.  2. In Fig. 3(a), why the PSNR decreased when the number of epochs increased? 3. Another work using symmetric fully convolution and de-convolution layers may be related to this paper.   Edgar Simo-Serra et al., Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup. SIGGRAPH 2016.   __________________________________________________________________ Update:  After reading the authors’ rebuttal, I still have several concerns about this paper.   The authors said that the difference between this paper and Kim’s paper is that they used more skip connections instead of a single one from the input to the output. To me, the results are not necessarily getting better by doing this. In the paper, there is no comparison between a single connection and multiple connections. Therefore, it is hard to evaluate the benefit of this. The authors only compared the multiple connections with residual learning [11], which was a different structure with a skip connection in each small block.   Another import issue of this paper is that Kim’s work is never mentioned. It could mislead the readers: it seems that all the ideas of this paper (e.g., skip connection, very deep structure, handle different noise/scale by mixing the training data) are original, but actually very similar to Kim’s work. The arXiv version of Kim’s paper was online in 2015. I am not asking the authors to compare with an arXiv paper, but the difference should be discussed since the ideas are so similar.   The results are also not very convincing.  In the supplementary results, it is hardly to see any visual improvements over existing methods. While the “numbers” (e.g. PSNR) are slighted improved by 0.1-0.2dB, it does not mean a necessary improvement. For example, in Johnson’s paper, it shows a lower PSNR gives perceptually better looking result. I agree with the authors that this paper has slightly higher PSNR than Kim's(around 0.15dB) on Set5 and BSD 100, but lower by similar amounts on Set14.   Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." arXiv preprint arXiv:1603.08155 (ECCV) 2016.  Overall, I feel the major issue of this paper is the lack of proper discussions and evaluations.


Object based Scene Representations using Fisher Scores of Local Subspace Projections
Several works have shown that deep CNNs can be easily transferred across datasets, e.g. the transfer from object recognition on ImageNet to object detection on Pascal VOC. Less clear, however, is the ability of CNNs to transfer knowledge across tasks. A common example of such transfer is the problem of scene classification, that should leverage localized object detections to recognize holistic visual concepts. While this problems is currently addressed with Fisher vector representations, these are now shown ineffective for the high-dimensional and highly non-linear features extracted by modern CNNs. It is argued that this is mostly due to the reliance on a model, the Gaussian mixture of diagonal covariances, which has a very limited ability to capture the second order statistics of CNN features. This problem is addressed by the adoption of a better model, the mixture of factor analyzers (MFA), which approximates the non-linear data manifold by a collection of local sub-spaces. The Fisher score with respect to the MFA (MFA-FS) is derived and proposed as an image representation for holistic image classifiers. Extensive experiments show that the MFA-FS has state of the art performance for object-to-scene transfer and this transfer actually outperforms the training of a scene CNN from a large scene dataset. The two representations are also shown to be complementary, in the sense that their combination outperforms each of the representations by itself. When combined, they produce a state-of-the-art scene classifier.

The paper claims that mixture of factor analyzers (MFA) is a better approximation compared to Fisher vector polling for transfer learning of high-order CNN features between object detection and scene classification tasks. This is supported by bunch of experiments that show improvements over other methods with considerable margins.
1. There is no theoretical justification of why MFA outperforms FV on transfer learning form object level to holistic scene descriptor. The main argument of the paper about "... inability of the standard GMM ... to provide good approximation ..." in L73-75 needs proof or reference to appropriate literature rather than only experiment results.  2. Authors claim "... MFA-FS captures highly discriminative covariance statistics ..." in L80-81, but I could not find any support of this except L161-162 which refers to full covariance S compared to Gaussian diagonal covariance. It needs to clarify why full covariance in MFA is the key to transfer learning problem on CNN features. I reckon it as a week argument although it was considered as second contribution of the paper because; any other dictionary learning method with full covariance should generate the same improvement as MFA according to authors' reasoning.  3. To my knowledge, section 2 of the paper seems a review of FV and MFA which does not clarify the novelty of authors' claimed contributions. An experienced reader is already aware of these formulations; hence it is expected to see the focus of formulation towards main claims which I could not see them there.  4. The paper seems like an experimental work which just plugs an idea to bunch of experiments to beat state-of-the-arts. I personally think that it may be of interest of computer vision community or venues like CVPR or WACV, but NIPS is a top-tier machine learning event which engineering tricks could not be that appreciated without sound theory and understanding in the background.  5. My other concern is about deployment of PCA with MFA-FS scores. Authors claim that MFA is employed due to its better ability to approximate nonlinear manifold of CNN. I expect that applying PCA to this high-order descriptor should degrade the accuracy much higher than what's reported in table 4, because PCA gets rid of many local sub-spaces with second and higher derivatives. Actually, it must diminish the advantage of MFA on factorizing to high-order sub-spaces, but the results remain almost the same i.e. 79.9 vs 79.3 on MIT. If the key contribution of the paper is a better description of high-dimensional and nonlinear CNN feature space by MFA, I’m wondering that it still remains competitive after a simple PCA.


Unsupervised Risk Estimation Using Only Conditional Independence Structure
We show how to estimate a model's test error from unlabeled data, on distributions very different from the training distribution, while assuming only that certain conditional independencies are preserved between train and test. We do not need to assume that the optimal predictor is the same between train and test, or that the true distribution lies in any parametric family. We can also efficiently compute gradients of the estimated error and hence perform unsupervised discriminative learning. Our technical tool is the method of moments, which allows us to exploit conditional independencies in the absence of a fully-specified model. Our framework encompasses a large family of losses including the log and exponential loss, and extends to structured output settings such as conditional random fields.

The paper proposes a means to estimate the risk of a model on unlabeled data, and to optimise a model on the same. The approach relies on making a structural assumption of the underlying generating distribution, namely that it decomposes into label independent views. Preliminary experimental results show promise for the method.
  I found the paper very well presented and enjoyable to read. The basic problem is interesting, and the approach presented as some salient features, notably the fact that one does not have to make parametric assumption on the underlying distribution. The high-level idea of imposing structural assumptions but nonetheless relying on discriminative models was quite elegant.   The basic insight in estimating the risk from unlabelled data is that by encoding a certain structural assumption - namely, that the data comprises three independent views - one implicitly gets information about the class-conditional risks by considering the first three moments of the label vectors. This leads to a system of equations which may be solved to infer the class-conditional risks. One applies a similar trick in order to estimate the gradient of the risk from unlabelled data, allowing one to optimise models such as logistic regression without labels. The fact that one is able to do so under somewhat mild assumptions is quite surprising.   The mechanics of the proposed approach, as well as the overall inspiration for considering higher-order moments, derive from the tensor decomposition framework for latent variable models, e.g. Anandkumar et al. 2012, a literature that I am not familiar with. I am thus unable to confidently proclaim the novelty of the presented approach. However, from some preliminary reading, it does not appear as though existing work in the literature has use such methods for unsupervised risk estimation. It seems a nice conceptual insight that one can use the three view assumption to aid in estimating classification error.   The technical content is fairly clearly presented in the main body, with some high-level intuition for the proofs of the main theorems. The proofs of Theorem 1 and 2 to rely on Theorem 7 of Anandkumar et al. 2012, but I wasn't quite able to discern how much of the heavy lifting is accomplished by the latter. I also do not have much intuition on how good the precise form of the sample complexity is, in terms of dependence on various parameters, but the paper makes an attempt to explain why each of these is intuitively necessary.    I view the contribution of the paper as primarily theoretical, but appreciate that some preliminary experiments are presented. They indicate that the proposed method can be useful in covariate shift scenarios. The performance for semi-supervised learning is a little more erratic, but there are likely many practical extensions possible.   Overall, I found this an interesting, well-written paper that solves a non-trivial problem. Given that I am not an expert in the field, I cannot however specifically assess the novelty of the work.   Minor comments:   - the assumption that the last decomposes as a sum over the three views is intuitive. In the specific example of logistic regression, it was a little unclear why the feature maps for the three views have the same dimensionality. In general does one simply partition the ambient feature space three separate components for the view?    - I didn't quite follow the statement that assumption one doesn't hold for the hinge loss. Which specific version of multiclass hinge are you referring to?    - in equation 1 and elsewhere, maybe it would be good to make explicit that M depends on theta.    - for the extension to hidden Markov models, it might be worth adding a line explaining the difference to what is done in e.g. Anandkumar et al. 2012, which also considers estimating parameters for such models.    - the seed model in assumption 2 doesn't appear to be explicitly referred to beyond its definition in the body of the paper (i.e. it isn't said in the body what one actually does with it), which could be confusing.    - isn't the domain adaptation experiment more specifically one of covariate shift?    - it wasn't clear why the proposed model performed worse than the seed for small values of a in the domain adaptation experiments. Is this a finite sample effect?    - citation for Anandkumar et al. 2013 could be for the JMLR version.    - some citations in the main body only appear in the complete list of references in the supplementary. I suggest perhaps reducing spacing or font size for the references in the body to accommodate everything.


Online ICA: Understanding Global Dynamics of Nonconvex Optimization via Diffusion Processes
Solving statistical learning problems often involves nonconvex optimization. Despite the empirical success of nonconvex statistical optimization methods, their global dynamics, especially convergence to the desirable local minima, remain less well understood in theory. In this paper, we propose a new analytic paradigm based on diffusion processes to characterize the global dynamics of nonconvex statistical optimization. As a concrete example, we study stochastic gradient descent (SGD) for the tensor decomposition formulation of independent component analysis. In particular, we cast different phases of SGD into diffusion processes, i.e., solutions to stochastic differential equations. Initialized from an unstable equilibrium, the global dynamics of SGD transit over three consecutive phases: (i) an unstable Ornstein-Uhlenbeck process slowly departing from the initialization, (ii) the solution to an ordinary differential equation, which quickly evolves towards the desirable local minimum, and (iii) a stable Ornstein-Uhlenbeck process oscillating around the desirable local minimum. Our proof techniques are based upon Stroock and Varadhan's weak convergence of Markov chains to diffusion processes, which are of independent interest.

This paper aims to study the global dynamics of nonconvex statistical optimization based on diffusion processes. As an example of non-convex problems, this paper discusses in details the SGD applied on the tensor decomposition formulation of independent component analysis (ICA). An interesting finding in this paper is a three-phases phenomenon of the global dynamics of SGD to capture the transition of SGD solution from unstable initialization to finally stable local minimum. Overall, this paper is well written and it addresses a very challenge problem. This paper serves as a first step in the theoretical understanding of global dynamics of SGD, and is believed to stimulate many work on the global dynamic of non-convex optimizations.
This paper aims to study the global dynamics of nonconvex statistical optimization based on diffusion processes. As an example of non-convex problems, this paper discusses in details the SGD applied on the tensor decomposition formulation of independent component analysis (ICA). An interesting finding in this paper is a three-phases phenomenon of the global dynamics of SGD to capture the transition of SGD solution from unstable initialization to finally stable local minimum. Overall, this paper is well written and it addresses a very challenge problem. This paper serves as a first step in the theoretical understanding of global dynamics of SGD, and is believed to stimulate many work on the global dynamic of non-convex optimizations.    Q1: Discuss how to generalize it other non-convex models? As a concrete example, this paper only discussed SGD for ICA. Can the authors briefly discuss how to apply their framework to other non-convex optimization problems?  Q2: discuss the relationship to a series of literature which also provides global rate of convergence of non-convex optimzation problems. e.g.   (1) Zhang et al. (2014), Spectral Methods meet EM: A Provably Optimal Algorithm for Crowdsourcing (2) Anandkumar, A., Ge, R. and Janzamin, M. (2014), Guaranteed non-orthogonal tensor decomposition via alternating rank-1 updates. (3) Sun et. al. (2016), Provable sparse tensor decomposition.  Q3: Assumptions: In (iii) of Assumption 1, Y_i is assumed to have up to 8-th moment, i.e., k = 1,2,...,8. However, in Proposition 2, it requires the assumption for k = 1, 2,...,d, where d is the dimension of X. Can the authors discuss in details why such condition is required? When d diverges, is this condition strong?  Q4: Please further polish the paper. Below are some examples of typos:  (1)line 118. ``Y_1,\ldots,Y_n" should be X_1,\ldots,X_n. (2)line 123: ``...whose the number linearly grows... (3)line 139: ``...the well-known coupon collector problem [18] it takes..." (4)line 146: ``To work on the approximation we first conclude the following". This is not a complete setence.


k ⇤ -Nearest Neighbors: From Global to Local
The weighted k-nearest neighbors algorithm is one of the most fundamental nonparametric methods in pattern recognition and machine learning. The question of setting the optimal number of neighbors as well as the optimal weights has received much attention throughout the years, nevertheless this problem seems to have remained unsettled. In this paper we offer a simple approach to locally weighted regression/classification, where we make the bias-variance tradeoff explicit. Our formulation enables us to phrase a notion of optimal weights, and to efficiently find these weights as well as the optimal number of neighbors efficiently and adaptively, for each data point whose value we wish to estimate. The applicability of our approach is demonstrated on several datasets, showing superior performance over standard locally weighted methods.

This paper proposes a reweighting method of nearest neighbors (NNs) for NN classification and regression. The authors derive the upper bound of the difference between the model output and the target value, and the minimization of the upper bound is obtained by a simple and quick iterative process. As a result, the weights \alpha for k nearest neighbors are determined for given data, as well as the optimal number k.
The mathematical derivation in this work is novel and interesting, but the writing can be improved. The derivation uses Hoeffding’s inequality for noise on labels, with a mild assumption on the maximum noise. The analysis using KKT condition is straightforward, and it results in an interesting strategy for finding weights.  I want to support the acceptance of this paper, but I have several concerns: 1) According to the derivation, the upper bound of the risk (P1) is given as C\lambda (line 112). With some simple calculations using appropriate parameters, this upper bound does not give a small value. For example, with b=1, delta=0.1, and k*=10, and \lambda greater than the minimum (\beta_i + 1/k*) from Eq.(4), the minimum upper bound I roughly calculated is about 0.24. For example in classification, this does not seem to be a meaningful upper bound. Can I reduce the upper bound further with appropriate k* (k*=3-10)?  2) In high dimensional space, distance to the nearest neighbors is unreliable in many cases. In particular, the distance is highly concentrated, and the nearest neighbor does not appear very closely to the test point. How do the authors advocate the Lipschitz condition that the function deviation is bounded linearly by the distance within the range where the nearest neighbors appear?  3) Though the derivation and the result is interesting, the paper is not well-written. The structure and the writing can be improved.  4) In the discussion from line 170 to 179, the two presented examples are redundant. The first example is the special case of the second example.  5) The title has to be changed. From the title, I cannot have any information about this paper.  The derivation in this manuscript is new, and I like the way of obtaining the proposed optimization procedure. However, further honest analysis will make this paper more interesting when this method is useful and when it is not. Also, if the authors can provide some guarantee that the writing can be significantly improved, I want to recommend the acceptance of this paper.