Neural Voice Cloning with a Few Samples
Voice cloning is a highly desired feature for personalized speech interfaces. We introduce a neural voice cloning system that learns to synthesize a person's voice from only a few audio samples. We study two approaches: speaker adaptation and speaker encoding. Speaker adaptation is based on fine-tuning a multi-speaker generative model. Speaker encoding is based on training a separate model to directly infer a new speaker embedding, which will be applied to a multi-speaker generative model. In terms of naturalness of the speech and similarity to the original speaker, both approaches can achieve good performance, even with a few cloning audios.
2 While speaker adaptation can achieve slightly better naturalness and similarity, cloning time and required memory for the speaker encoding approach are significantly less, making it more favorable for low-resource deployment.

This work examines the problem of cloning a target speaker's voice given a trained generative model (with audio/speech and text). The broader context of the work is in Text to Speech (TTS) models in which rapid and excellent developments have occurred in the last few years. These models use DNNs as seq2seq models to predict speech to text. Other permutations are speech to speech/voice conversion (going from source voice to target voice), and voice cloning and adaptation, which this paper addresses. In the problem under consideration in the paper (voice cloning) one is able to produce voice that sounds like a certain 'target' or desired speaker using only a few samples presented which is then used to generate speech given arbitrary text. This generation is carried out using a multispeaker generative model that is trained over a large number of speakers and text/speech training data.   The reviewer feels that this work is an outstanding example of current TTS work and a massive undertaking both in terms of the quality of results produced and in the relevance of ideas it contains.  Prior work: The work is an extension of recent developments on seq2seq attention based TTS modeling works (e.g. Tacotron, DeepVoice-3 and lower). The models have several important improvements such as autoregressive models with dilated convolutions, trainable speaker embeddings, multiheaded self attention, and input feature processing (convolutions, filter banks etc from CBHG in tacotron), and using characters instead of phonemes to do away with segmentation.  The work uses the DeepVoice-3 architecture to carry out the voice cloning task. Cloning approach: Two approaches are presented. 1) Speaker adaptation - Step1: A multispeaker generative model is trained first with large amount of data (text/speech). Step 2: In order to clone voice of a new speaker (with few samples), the model learns the embedding of the speaker using the trained generative model learnt with large amount of data. This is what they call the speaker adaptation approach without fine tuning.  In addition to the above, one may also make the generative model tunable (along with the new speaker's embeddings). This is called 'whole model adaptation' in the paper. It is noted that the whole model adaptation produces the best speaker samples, and trains much faster than the case where only the speaker is implicitly adapted but the model remains fixed. Step 3: The adapted speaker embedding is now used for the inference task where text is passed along with speaker encoding to produce voice from the new speaker.  2) Speaker encoding - Step 1: The first step is to train the generative model along with speaker embeddings as in the speaker adaptation appraoch. The difference here is that the model also learns to produce as output an encoding for speakers. Step 2: For the voice cloning task, the model outputs a speaker encoding explicitly when presented with voice samples from the target to be cloned. Step 3: After an encoding/embedding is produced in the second step, one may use it for inference in the speaker adaptation approach where one passes in some text and the newly produced encoding to generate voice from the cloned speaker.   The paper goes on to show results to demonstrate that both approaches are viable (samples are available at https://audiodemos.github.io/ as indicated in the paper).   There are many details and minutiae in the work of very fine granularity. A few points came to the attention of this reviewer.  1) Training speaker encoding: Lines 128, 129 in Section 3.2 (Speaker encoding). Optimization challenges are mentioned while training the speaker encoding approach. The model tends to average out speaker voices to minimize overall generative loss. Instead, they device a workaround by training each speaker separately and fine tuning the model afterwards.   2) In addition to human evaluations, the architecture produces two comparison metrics to assist in the verification process.  a) Speaker classification - determining which speaker an audio sample belongs to b) Speaker verification - verifying the identity of the speaker. In other words, performing binary classification to identify whether the test audio enrolled is from the same speaker.  c) Latent space walks - While this work does not claim to explore stochastic (in the style of VAE) latent manifolds, it presents similar latent style transformations by manipulating embeddings (section 4.4 - Voice morphing via embedding manipulation). This could be useful pointers for investigations in tasks such as transforming voices.    Results: A fairly extensive set of experiments are conducted to compare quality of generated samples. Notably, the quality improves with the number of samples used for cloning (the number of samples used varies from 1-100). Also, the results consistently show that whole model adaptation is 'better' (when we listen to the samples) than single speaker embedding. The speaker embedding and encoding performance seem to be on par.   While I think that the paper's subject matter, ideas and results are truly exploring the limits of what is possible now in neural TTS, it seems that this would be an extremely difficult project to replicate owing to the complexity of the models presented and the generally pipelined nature of the space (it is not really end-to-end).   Comments on pertinent evaluation criteria:  1) Quality: Excellent. Encyclopedic presentation of current neural TTS landscape. Several novel ideas.  2) Clarity: Excellent. The paper is readable and well written.  3) Originality: Very Good. Voice cloning is an interesting (although these problems have been around for a long time) application for DNNs. The paper presents several original ideas in speaker adaptation, encoding and latent vector manipulation.  4) Significance: Excellent. The problems explored (text to speech, voice cloning, voice conversion, speaker verification etc.) in this paper are very pertinent to speech synthesis research. It demonstrates the massive progress made in recent times in DNN based TTS.


Learning Versatile Filters for Efficient Convolutional Neural Networks
This paper introduces versatile filters to construct efficient convolutional neural network. Considering the demands of efficient deep learning techniques running on cost-effective hardware, a number of methods have been developed to learn compact neural networks. Most of these works aim to slim down filters in different ways, e.g. investigating small, sparse or binarized filters. In contrast, we treat filters from an additive perspective. A series of secondary filters can be derived from a primary filter. These secondary filters all inherit in the primary filter without occupying more storage, but once been unfolded in computation they could significantly enhance the capability of the filter by integrating information extracted from different receptive fields. Besides spatial versatile filters, we additionally investigate versatile filters from the channel perspective. The new techniques are general to upgrade filters in existing CNNs. Experimental results on benchmark datasets and neural networks demonstrate that CNNs constructed with our versatile filters are able to achieve comparable accuracy as that of original filters, but require less memory and FLOPs.

The paper introduces two new types of convolutional filters, named versatile filters, which can reduce the memory and FLOP requirements of conv. nets. The method is simple and, according to the experimental results, it seems to be effective.  The text quality is OK, although it would definitively benefit from better explanations about the proposed method. For instance, Fig. 1 is a bit confusing (Are you showing 4 different filters in Fig 1 (b) ?).  My main concerns about this paper are related to the experiments and results, as detailed in the following questions: (1) Regarding the FLOP reduction, it is not clear how the reduction in the number of computations is really achieved. In the algorithm presented in the supplementary material, I cannot see any reduction in the number of FLOPs. Although you have less filters, these filters must be applied s times, and you still added the overhead of the masks. Please explain how you can achieve the FLOP reduction in practice. (2) The memory and FLOP reductions reported in the experiments are just nominal, correct? You must also report the actual memory and execution time of each one of the methods. For instance, ShuffleNet paper reports the actual inference time for different variations of their method as well as the baselines. Without this info, it is difficult to judge the real effect of the proposed method. (3) How did you implement the proposed conv. nets for the ImageNet experiment? There is no information about the implementation in the paper. I see you used Matlab for the MNIST experiment. How easy is it to implement the prosed method using one of the mainstream DL libraries (TensorFlow, Pytorch, Caffe) and have the proposed FLOP reduction?   --- Given the author feedback, I am willing to increase my score from 5 to 6. The author feedback properly addressed just some of my questions. Fig.1 is still unclear to me and the execution time comparison with at least one previous model (ShuffleNet) is not reported.


Constrained Graph Variational Autoencoders for Molecule Design
Graphs are ubiquitous data structures for representing interactions between entities. With an emphasis on applications in chemistry, we explore the task of learning to generate graphs that conform to a distribution observed in training data. We propose a variational autoencoder model in which both encoder and decoder are graphstructured. Our decoder assumes a sequential ordering of graph extension steps and we discuss and analyze design choices that mitigate the potential downsides of this linearization. Experiments compare our approach with a wide range of baselines on the molecule generation task and show that our method is successful at matching the statistics of the original dataset on semantically important metrics. Furthermore, we show that by using appropriate shaping of the latent space, our model allows us to design molecules that are (locally) optimal in desired properties.

Summary: This paper describes a model for generating graph-structured data, with molecule generation being the example task. This model is based around a variational autoencoder whose encoder/decoder are designed to handle graph-structured data. The decoder builds a graph sequentially by starting from an arbitrary node and sampling edges to other nodes, which are placed in a queue; upon sampling an edge to a “stopping node,” the next node is taken from the queue and the process continues until there are no more nodes to expand. The distributions from which these samples are taken are a function of the graph state (notably, not the specific steps taken to arrive at the current state), where the state vectors are encoded using a gated graph neural network (GGNN). Additionally, masking functions can be specified that serve as hard constraints on the sorts of edges that may be sampled (in case these would lead to graphs that are disallowed, e.g. that would lead to impossible molecules). The encoder is also comprised of a GGNN. Training is accomplished using a standard VAE objective; however, for computational efficiency reasons the reconstruction loss is only estimated. Additionally, a term can be added to the loss which is a function of the latent representation that attempts to optimize some numerical quantity related to the graphs being generated; at test time, gradient ascent can be used to find a latent representation with a “better” value of this quantity. Experiments were run on three molecule datasets, with comparisons being made against a variety of graph generation baselines regarding generated molecule uniqueness, validity, and novelty, as well as how the distributions of generated molecule statistics compare against the training dataset.  Comments: (using the same citation numbering as in the paper) This work is primarily a synthesis of ideas and techniques explored in other works: using a GGNN to model graphs as in [21]; training using a VAE objective as in [27]; enforcing hard constraints on generated graphs as in [27]; and learning a model that is invariant to graph construction order as in [27]. The real strength of this paper is that it combines these ideas and makes tweaks that allow the model to function well in practice, which seems to be primarily a combination of the modeling strength of GGNN (demonstrated by comparing the GGNN-based models against the non GGNN-based models during experiments) with the masking of invalid edges during generation and construction-order invariance. The provided experiments provide sufficient evidence that the proposed model has absorbed the benefits of these prior models (though it would have been nice to see more comparisons against [21] to ensure that the better results of this model are not an artifact of the one dataset used).  There are a few things I felt were missing that would have strengthened this paper a bit. The optimization of task-specific metrics (as described in section 4.3) was somewhat underexplored - section 6.2 presents a small experiment demonstrating that it is possible to learn a network that can optimize for a given metric, but only a few samples are provided. A full quantitative evaluation of this portion of the model would have been interesting to see (e.g. an ablation comparing the average metric for generated molecules with and without this additional modeling step and whether doing so hurts performance on other metrics at all). Additionally, details of training were completely omitted, including training hyperparameters and how they were chosen - this hurts reproducibility. Some further ablations related to training details, e.g. whether or not the number of path samples taken of the training data, would also have been interesting to see. Overall, though, these omissions do not detract significantly, and I feel that this paper is very worthy of submission.  It should also be noted that this paper is very well written, with the specific details presented in an exceptionally clear manner. None of the details presented were confusing.  POST AUTHOR RESPONSE EDIT: Thanks for the response. My opinions remain more or less the same, as your response did not address my primary concerns - that a comparison against Li et al on more datasets would further justify your claims and that the experiments in section 6.2 are somewhat preliminary in nature. However, I do appreciate the additional ablation experiments you did run, and your discussion of the evaluation metrics was important and should probably be added to the discussion presented during the experiments.


Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization
We present a unified framework to analyze the global convergence of Langevin dynamics based algorithms for nonconvex finite-sum optimization with n component functions. At the core of our analysis is a direct analysis of the ergodicity of the numerical approximations to Langevin dynamics, which leads to faster convergence rates. Specifically, we show that gradient Langevin dynamics (GLD) and stochastic gradient Langevin dynamics (SGLD) converge to the almost minimizer 2 within e O nd/( ✏) and e O d 7 /( 5 ✏ 5 ) stochastic gradient evaluations respectively 3 , where d is the problem dimension, and is the spectral gap of the Markov chain generated by GLD. Both results improve upon the best known gradient complexity 4 results [45]. Furthermore, for the first time we prove the global convergence guarantee for variance reduced stochastic gradient Langevin dynamics (SVRG-LD) to the almost minimizer within e O p nd 5 /( 4 ✏ 5/2 ) stochastic gradient evaluations, which outperforms the gradient complexities of GLD and SGLD in a wide regime. Our theoretical analyses shed some light on using Langevin dynamics based algorithms for nonconvex optimization with provable guarantees.
, where d is the problem dimension, and is the spectral gap of the Markov chain generated by GLD. Both results improve upon the best known gradient complexity 4 results [45]. Furthermore, for the first time we prove the global convergence guarantee for variance reduced stochastic gradient Langevin dynamics (SVRG-LD) to the almost minimizer within e O p nd 5 /( 4 ✏ 5/2 ) stochastic gradient evaluations, which outperforms the gradient complexities of GLD and SGLD in a wide regime. Our theoretical analyses shed some light on using Langevin dynamics based algorithms for nonconvex optimization with provable guarantees. question in nonconvex optimization and machine learning: is there an efficient algorithm that is guaranteed to converge to the global minimum of (1.1)?
Recent studies [17,16] showed that sampling from a distribution which concentrates around the global minimum of F n (x) is a similar task as minimizing F n via certain optimization algorithms. This justifies the use of Langevin dynamics based algorithms for optimization. In detail, the first order Langevin dynamics is defined by the following stochastic differential equation (SDE) dX(t) = rF n (X(t))dt + p 2 1 dB(t),
2) where > 0 is the inverse temperature parameter that is treated as a constant throughout the analysis of this paper, and {B(t)} t 0 is the standard Brownian motion in R d . Under certain assumptions on the drift coefficient rF n , it was showed that the distribution of diffusion X(t) in (1.2) converges to its stationary distribution [14], a.k.a., the Gibbs measure ⇡(dx) / exp( F n (x)), which concentrates on the global minimum of F n [29,26,47]. Note that the above convergence result holds even when F n (x) is nonconvex. This motivates the use of Langevin dynamics based algorithms for nonconvex optimization [45,53,50,49]. However, unlike first order optimization algorithms [42,27,46,3], which have been extensively studied, the non-asymptotic theoretical guarantee of applying Langevin dynamics based algorithms for nonconvex optimization, is still under studied. In a seminal work, Raginsky et al. [45] provided a non-asymptotic analysis of stochastic gradient Langevin dynamics (SGLD) [52] for nonconvex optimization, which is a stochastic gradient based discretization of (1.2). They proved that SGLD converges to an almost minimizer up to d . In a concurrent work, Zhang et al. [53] analyzed the hitting time of SGLD and proved its convergence to an approximate local minimum. More recently, Tzen et al. [50] studied the local optimality and generalization performance of Langevin algorithm for nonconvex functions through the lens of metastability and Simsekli et al. [49] developed an asynchronous-parallel stochastic L-BFGS algorithm for non-convex optimization based on variants of SGLD. Erdogdu et al. [23] further developed non-asymptotic analysis of global optimization based on a broader class of diffusions.
In this paper, we establish the global convergence for a family of Langevin dynamics based algorithms, including Gradient Langevin Dynamics (GLD) [17,20,16], Stochastic Gradient Langevin Dynamics (SGLD) [52] and Stochastic Variance Reduced Gradient Langevin Dynamics (SVRG-LD) [19] for solving the finite sum nonconvex optimization problem in (1.1). Our analysis is built upon the direct analysis of the discrete-time Markov chain rather than the continuous-time Langevin diffusion, and therefore avoid the discretization error.

After the author rebuttal: Thank you for your detailed and satisfying responses to all of my comments. I keep my overall score and congratulate you for a very nice paper that I enjoyed reading.  The authors propose a new framework to analyze the global convergence of Langevin dynamics based algorithms in a nonconvex optimization context. Three type of algorithms are considered, including gradient Langevin Dynamics, its stochastic version and, finally, a stochastic version that incorporates variance reduction. The key idea is a novel decomposition of the optimization error that directly analyses the ergodicity of the numerical approximation of the Langevin diffusion, rather than measuring the error of the true dynamics and its discretization, which is a slowly converging term. Moreover, focusing on the numerical approximation bypasses the analysis of the convergence of the true dynamics to its stationary distribution. Bypassing these two steps, the authors are able to show improved convergence rates compared to previous work for gradient Langevin dynamics and its stochastic version. In addition, the authors provide a first global convergence guarantee for the variance reduced version in this nonconvex setting. They show that, under certain conditions, the variance reduced version is superior to the other two.  My comments: 1. line 51: I understand this as you mean that you take the discrete-time diffusion to be the "ground truth" and do subsequent analysis based on this? If so there should be no discretization error, i.e. you bypass it by construction? What do you mean with "get around to a large extent"?  2. You refer to sigma2 as the "variance of the stochastic gradient". The variance of the gradient should formally be a matrix (variance-covariance) matrix since the gradient is a vector. Or is it another variance metric you have in mind?  3. In general, how does sigma2 (however you define it) scale with d and n?  4. Does the rate d^7 (or d^5 after variance reduction) mean that the stochastic versions are basically non-feasible for high-dimensional datasets? I am not an expert in optimization methods, but the consensus seem to be that optimization methods scale much better with respect to the dimension of the problem compared to say simulation methods. These rates with respect to d seem to suggest otherwise.  5. Do you have any empirical results to validate your mathematical derivations? Note that this is the reason that I responded "Not confident" on the reproducibility question.  6. You have assumed that eta is a constant "step-size". You could probably improve your rates a lot by clever step-size choices, for example, using second order methods. I suppose that you would have to make more assumptions and your results would be less general, but it would be interesting to hear your thoughts on how to extend this work beyond a constant step-size.  7. Should there be dependence on n in the rates for gradient Langevin dynamics in Section 1.1?  Other comments: This paper is very well written, in particular considering that it is purely theoretical. The authors have done an excellent job in presenting the work in a comprehensible way (using 8 pages only!).


Reparameterization Gradient for Non-differentiable Models
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the reparameterization trick, one of the most effective techniques for addressing the variance issue for differentiable models, so that the trick works for non-differentiable models as well. Our algorithm splits the space of latent variables into regions where the density of the variables is differentiable, and their boundaries where the density may fail to be differentiable. For each differentiable region, the algorithm applies the standard reparameterization trick and estimates the gradient restricted to the region. For each potentially non-differentiable boundary, it uses a form of manifold sampling and computes the direction for variational parameters that, if followed, would increase the boundary's contribution to the variational objective. The sum of all the estimates becomes the gradient estimate of our algorithm. Our estimator enjoys the reduced variance of the reparameterization gradient while remaining unbiased even for non-differentiable models. The experiments with our preliminary implementation confirm the benefit of reduced variance and unbiasedness.

EDITED AFTER REBUTTAL PERIOD:  The rebuttal addresses concerns 3 and 4. I like the plots in Figure 1' and suggest to include them in the paper, together with table 2'.  Regarding concerns 1 and 2, I disagree with the authors' rebuttal in that Gumble-softmax (and related) approaches are "orthogonal" to the paper. As noted in the rebuttal, any model with categorical variables can be rewritten in terms of if/else statements. That is exactly what I had in mind when I wrote my initial review, which is why I suggested to include an experiment involving a model with categorical latent variables.  ==============================  In this paper, a VI stochastic gradient estimator for non-differentiable models is proposed. The proposed estimator contains two terms; one resembles the reparameterization gradient and the other involves surface integrals over the non-differentiable boundary. The estimator is compared on three models involving if statements to the score function estimator as well as a naive approach that takes into account the reparameterization component only.  To the best of my knowledge, the method presented in this paper is novel. The paper is very well written and the ideas are easy to follow, although I am not familiar with the results in [3] about exchanging integration and differentiation under moving domain, so I can't really assess the correctness of the math and the proofs. They are technically sound, though.  I have some comments and questions, mostly about the experimental evaluation.  1) I would have liked to see an experiment with categorical latent variables on a more "classical" model, similar to the ones considered in [9,32,6].  2) I was missing a comparison with other gradient estimators for non-reparameterizable distributions, such as REBAR [32] and RELAX [6]. Can you say something about how these methods would compare?  3) I was confused about the numbers in Table 2. What does it mean exactly "to complete a single optimization trajectory"? I suggest to replace these numbers with the time per iteration of each approach, because otherwise it is hard to read if the numbers presented involve different number of iterations in each cell of the table.  4) I appreciate the honesty about the computation time being the worst for the proposed approach. However I would have liked to see a better implementation rather than an early unoptimized prototype. Based on the behavior of the "Repar" method in Figure 1, my guess is that the suggested idea of subsampling the L terms of the boundary component will perform well enough. I also suggest to implement at least one simple model in a library other than Python+autograd that avoids dynamic computational graphs. Even if the resulting implementation is not generic, it will allow to learn something about the true computational cost of each approach.


Leveraging the Exact Likelihood of Deep Latent Variable Models
Deep latent variable models (DLVMs) combine the approximation abilities of deep neural networks and the statistical foundations of generative models. Variational methods are commonly used for inference; however, the exact likelihood of these models has been largely overlooked. The purpose of this work is to study the general properties of this quantity and to show how they can be leveraged in practice. We focus on important inferential problems that rely on the likelihood: estimation and missing data imputation. First, we investigate maximum likelihood estimation for DLVMs: in particular, we show that most unconstrained models used for continuous data have an unbounded likelihood function. This problematic behaviour is demonstrated to be a source of mode collapse. We also show how to ensure the existence of maximum likelihood estimates, and draw useful connections with nonparametric mixture models. Finally, we describe an algorithm for missing data imputation using the exact conditional likelihood of a DLVM. On several data sets, our algorithm consistently and significantly outperforms the usual imputation scheme used for DLVMs.

Updated Review after Rebuttal: After reading the authors response and re-evaluate the paper I do agree that most of my concerns that there was a fundamental issue with some of their statements were wrong, hence I'm changing my score from 3 to 6. From going into detail of the proof it resides on constructing a generative model where for half of the latent variables (w^t z < 0) the integral is bounded for all data points and for the other half for 1 data point the integral diverges while for the other goes to zero. This split allows them to say that the one integral diverges and the all of the others are finite hence the likelihood is infinite.  However, I'm still not convinced that this issue actually arises at all in practical settings.  First, in practice, we are optimizing an ELBO which is never tight, hence for this to be convincing argument the authors should investigate whether there are settings of the ELBO where it diverges except when it can perfectly reconstruct the posterior.  Furthermore, I still stand that I do not think that the results on the Frey Faces dataset are interpreted correctly and given that this is a fairly small dataset it is highly likely that the generative model overfits to the data (but not in the way for the divergence to happen).  The experimental section in this direction seems to be a bit weak, nevertheless, the paper is worth being accepted.   ================================================================ Summary of paper: The paper investigates various properties of the exact likelihood of Deep Latent Variable Models (DLVM). In the first part of the exposition, the authors argue that the likelihood of standard DLVMs is ill-posed in the sense that there exist parameters for which it diverges for continuous data and is well-posed for discrete. In the second part, they propose to use Metropolis-within-Gibbs instead of the standardly used pseudo-Gibbs sampler for missing data imputation. The experimental section on demonstrating unboundedness of the likelihood is very brief and needs more evidence, while the ones comparing different samplers is much more convincing.   Summary of review: The paper's arguments and conclusions about regarding the likelihood being unbounded for DLVMs seem to be fundamentally wrong. Although the proved theorems are mathmeatically correct their assumptions do not match the standard way these models are defined in the literature. Additionally, the experiments in this direction are minimal and I suspect that the reason for the generalization gap is that by not treating the network parameters in a Bayesian way we are still performing MLE on those, hence we are not guaranteed that the network can not overfit.  The proposed method for doing missing data imputation - Metropolis-within-Gibbs - is not known to me to have been used previously in the literature, hence there is a valid contribution here. The ideas presented clearly and are very easy to implement. Results to support the theoretical intuition that this method should perform well versus the pseudo-Gibbs most widely used in practice.  My general opinion is that the authors should reconsider their stance on the unboundness of the likelihood, based on my details comments below, and focus a lot more on the missing data imputation. The idea seems novel and interesting to me. More results and better presentation in this direction would definitely make the paper worth for publishing. However, given the inaccuracies in the first idea, which takes about half of the paper, at the current stage, I propose this work to be rejected.     Comments by sections:  On the boundedness of the likelihood of deep latent variable models ====================================================== The authors argue that standard DVMs as the one presented in Kingma and Welling [25] have unbounded loglikelihood. Although Theorem 1 is mathematically correct as well as Proposition 1 there is a major flaw in the logic and conclusions extracted from this. The parameterization considered in these and written out in equation (9) has been presented to be reflecting the model described in equation (8) which is a 2 layer generator network of a standard VAE. However, there is a fundamental mistake here - equation (9) assumes that there is a different set of parameters $\theta_i$ **per datapoint**, specifically by their definition $\theta_i$ contains $x_i$. This is significantly flawed as the standard VAE model of Kingma and Welling as well as any other model derived from this framework always assumed that the parameters are tight together between all data points (e.g. as the authors originally define $\theta$ below equation (8)). Under this wrong assumption, the following Theorem 1 seems very natural as this non-parametric model would just output a delta function around each data point. However, in any VAE in the literature, where this is not the case, the arguments of this section would not apply and the likelihood is most-likely bounded (I'm not aware of any way of proving this rigorously either way). Hence and the conclusions drawn regarding models known in the community are wrong.    Towards data-dependent likelihood upper bounds ============================================== In this section, the authors propose an upper bound on the likelihood based on finite mixture models. Theorem 2 is a valid bound, however, the assumptions in Theorem 3 are very unrealistic: 1. It is still under the assumption that the generative model parameters are per-data point. 2. It cites the universal approximation abilities of Neural Networks as a reason why the bound is tight. Although this is technically possible, if we assume 1. is not a flaw, however even we never actually train networks with infinite capacity, hence the bound is in fact arbitrarily (and most likely) very loose.   Additionally, the authors fail to cite previous works on upper bounds of the likelihood of DLVMs such as "Sandwiching the marginal likelihood using bidirectional Monte Carlo", "Approximate Bayesian inference with the weighted likelihood bootstrap". Specifically, the Bi-directional Monte Carlo Method can provide increasingly more accurate and tighter upper bounds based on the amount of computing resources provided.   Missing data imputation using the exact conditional likelihood ============================================================== The de-facto "standard" way for doing data-imputation with DLVMs is to use what the authors call pseudo-gibbs sampling. This invloves sequentially sampling p(z|x_o,x_m) and p(x_m|z,x_o) where one replaces the true posterior with the learned approximation. The authors propose to "fix" the method by rather than just sampling from the approximate posterior to use Metropolis-Hastings to account for the approximation gap. This is as correctly pointed out by the authors an instantion of Metropolis-within-Gibbs and indeed is a very valid observation which I have not seen so far in the literature. Although simple, such connections are indeed important.   Witnessing likelihood blow-up ============================================================== Here I think that the experiments are not enough - the authors only demonstrate work on the Frey faces. This is both relatively small as well as not too popular in order to compare with other results in the literature. Additionally, the authors conclude that the big generalization gap is a confirmation of the unboundedness of the likelihood. As discussed earlier, the likelihood is most likely bounded. Secondly, the most likely reason for the generalization gap is the fact that the training on the ELBO still performs an MLE estimation of the parameters $\theta$ of the generative network. This implies that if the network has enough capacity it **will** overfit the data and has been observed before in the literature. This is indeed an issue with the naive formulation of Variational Inference where one does not also provide prior for the network parameters. However, for more complicated datasets, it seems that in practice generating good images is difficult enough that the models used tend to not have enough capacity to do so. Thus my suspicion is that the Frey faces is too simple.    Comparing the pseudo-Gibbs and Metropolis-within-Gibbs samplers ============================================================== The results in this section provide good evidence that the Metropolis-within-Gibbs sampler performs bettern tha the pseudo-Gibbs. It would be interesting to show more statistics than just the F1-score (e.g. the raw accuracy etc...). Further, showing actual reconstructed images would make the presentation even better.


Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate
Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain near-optimal (but non-zero) test error. This phenomenon of strong generalization performance for "overfitted" / interpolated classifiers appears to be ubiquitous in high-dimensional data, having been observed in deep networks, kernel machines, boosting and random forests. Their performance is consistently robust even when the data contain large amounts of label noise. Very little theory is available to explain these observations. The vast majority of theoretical analyses of generalization allows for interpolation only when there is little or no label noise. This paper takes a step toward a theoretical foundation for interpolated classifiers by analyzing local interpolating schemes, including geometric simplicial interpolation algorithm and singularly weighted k-nearest neighbor schemes. Consistency or near-consistency is proved for these schemes in classification and regression problems. Moreover, the nearest neighbor schemes exhibit optimal rates under some standard statistical assumptions. Finally, this paper suggests a way to explain the phenomenon of adversarial examples, which are seemingly ubiquitous in modern machine learning, and also discusses some connections to kernel machines and random forests in the interpolated regime.

On further reflection and seeing the responses, I have substantially increased my score. I think the takeaway "interpolating classifiers/regressors need not overfit" is quite important, even if the algorithms studied here are fairly different from the ones people are actually concerned about. I would suggest re-emphasizing that this is the main point of the paper in the introduction, and additionally toning down some of the discussion about a "blessing of dimensionality" as mentioned in your response / below.  Original review is below. =======   This paper studies generalization bounds for certain classifiers which interpolate training labels. This is related to the recent "controversy" in learning theory, brought to prominence by [43] and continued in [9, 41], that practical deep learning models (and some kernel-based models) lie far outside the regime of performance explained by current learning theory, for example having extremely high Rademacher complexity, and yet perform well in practice. This paper gives bounds for two particular nearest-neighbor-like models, which interpolate the training data and yet are argued to generalize well in high dimensions. This is perhaps a step towards understanding the regime of interpolating classifiers. The 1-nearest-neighbor model, of course, is not consistent (having asymptotic risk up to twice the Bayes risk).  -----  The first model studied is one based on simplicial interpolation, related to a nearest-neighbor model, studied for both regression and classification. The model is not argued to be practical, particularly in high-dimensional, but is studied as an analogue. The bounds shown, however, do not actually show consistency. The excess risk is bounded by 2 sigma^2 / (d + 2), with sigma^2 >= sup_x var(Y | X = x); this is argued to decrease in high dimensions, a "blessing of dimensionality," which of course it does if sigma is fixed and d increases. But keeping sigma fixed as the dimension increases is not necessarily particularly natural; for example, if we consider independent N(0, s^2) noise in each new dimension as we grow d, then sigma^2 would grow linearly with d, giving a constant excess risk. Also, it seems likely that even the terms in the bound of Theorem 3.2 which go to zero in the setting of Corollary 3.3 do so at a rate which gets exponentially worse in the dimension, so referring only to a so-called "blessing of dimensionality" in this case seems misleading.  A similar case is true for classification: neither special case of Theorem 3.4 gives consistency. Corollary 3.5's results are somewhat hard to interpret in the interplay between the smoothness conditions and the dimension; using Remark 3.6, we see that the direct dependence on d is approximately (d + 1)^{- beta / 2}, but of course (A, alpha, B, beta) will be affected by d.  Corollary 3.7 seems to show a simpler-to-interpret result, except that its conditions are actually impossible to satisfy! Holder continuity implies continuity (for alpha > 0), so eta must be continuous; but the condition of 3.7 implies that eta never hits 1/2 on the support of mu. Since mu is also assumed to be uniform on a simple polytope, i.e. supp(mu) is connected, Corollary 3.7 can only possibly apply to the case where eta > .5 on the whole distribution or eta < .5 on the whole distribution, in which case f^* is uniformly 0 or 1, and the problem is trivial.  We could imagine replacing the condition of 3.7 with something like Pr( |eta(x) - 1/2| > h ) >= 1 - delta, giving an excess risk of delta + exp(-(d + 1) h^2 / 2). But even in this case, it seems likely that the relationship between h and delta would worsen with increasing dimension for reasonable functions eta.  Thus, although there are regimes where high-dimensional simplicial interpolation becomes essentially-consistent, they are only very limited. Given that (a) the algorithm is utterly impractical in high dimensions, and (b) the sample complexity is likely also exponential in the dimension, it's unclear what this analysis really tells us about the cases that we actually care about.  The connection to Laplace kernel methods and PERT-like trees in one dimension (in the discussion) is interesting, but it seems that the correspondence is lost in dimensions greater than one, while the rest of the paper focuses on the high-dimensional regime. It would be interesting to know if there is, for example, some other kernel than Laplace that gives a model like simplicial interpolation in higher dimensions.  -----  The other model analyzed is for a weighted nearest neighbor classifier, with weights chosen such that the classifier interpolates the training labels. This classifier is perhaps more realistic to actually apply to data, and interestingly is also shown to be consistent in any dimension.  The introduction (line ~93), however, claims that this technique "also benefits from the blessing of dimensionality as conditions on the weights to insure consistency become less restrictive in higher dimension." Though it is true that more weight functions are allowed as d increases, it seems like a stretch to call Corollary 4.4 an instance of "blessing of dimensionality": the convergence rate is exponential in d! (Also, as before, alpha and beta will likely increase with d for "reasonable" eta functions.)  It would also be good to clarify (line 286) exactly what at least the rate for k is that provides the given rate; there doesn't seem to be a reason not to put at least the respective choices for gamma, p, and k in the appendix, to save the interested reader some time.  -----  Overall: the problem setting of this paper is interesting, and the consistency of the weighted nearest neighbors classification scheme is intriguing (though it could use a little more comparison to the Hilbert kernel scheme). But its claims about a "blessing of dimensionality," and the overall value of the results about simplicial interpolation are misleading, and I'm not entirely sure what the greater takeaway of most of this work is really supposed to be.  I did not carefully check the proofs.  -----   Minor presentation notes and typos:  - Line 28: "training training deep networks" - Line 91: "singular weigh functions" - Line 178: "can yield to classifiers" - should be "can yield" or "can lead to" - Theorem 4.3: The square in the denominator of kappa_p should be inside the expectation, not after the conditioning; it would also be clearer to add another set of brackets in the proof.


Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance. This is especially true with high-capacity parametric function approximators, such as deep networks. In this paper, we study how to bridge this gap, by employing uncertainty-aware dynamics models. We propose a new algorithm called probabilistic ensembles with trajectory sampling (PETS) that combines uncertainty-aware deep network dynamics models with sampling-based uncertainty propagation. Our comparison to state-of-the-art model-based and model-free deep RL algorithms shows that our approach matches the asymptotic performance of model-free algorithms on several challenging benchmark tasks, while requiring significantly fewer samples (e.g., 8 and 125 times fewer samples than Soft Actor Critic and Proximal Policy Optimization respectively on the half-cheetah task).

This paper describes a model-based reinforcement learning approach which is applied on 4 of the continuous control Mujoco tasks.  The approach incorporates uncertainty in the forward dynamics model in two ways: by predicting a Gaussian distribution over future states, rather than a single point, and by training an ensemble of models using different subsets of the agent's experience.  As a controller, the authors use the CEM method to generate action sequences, which are then used to generate state trajectories using the stochastic forward dynamics model.  Reward sums are computed for each of the action-conditional trajectories, and the action corresponding to the highest predicted reward is executed.  This is thus a form of model-predictive control.   In their experiments, the authors show that their method is able to match the performance of SOTA model-free approaches using many fewer environment interactions, i.e. with improved sample complexity, for 3 out of 4 tasks. It also performs better than another recent model-based RL approach which uses a deterministic forward dynamics model and a random shooting methods, rather than CEM, for producing candidate actions to be evaluated by the forward model.   Quality: The paper performs a thorough experimental evaluation on the 4 tasks and compares against several strong baselines, including model-based and model-free methods. The authors also perform ablation studies which disambiguate the effects of different design decisions, such as probabilistic vs. deterministic outputs of the forward model and single model vs. ensemble of models. These actually show that most of the performance gains come from the use of a probabilistic output, rather than the model ensembling, which somewhat weakens the authors' story in the introduction that modeling both aleatoric and epistemic uncertainty (captured by probabilistic models and ensembling, respectively) is useful, although the gains of using the ensembling are clear for 1 out of 4 tasks.   Clarity: The paper is clear and well-written, and results are well-communicated through the figures.   Originality: As the authors point out, the components of their method are not novel (probabilistic neural networks, ensembling, using dynamics models for planning), but this particular combination of components applied to these particular tasks is.   Significance:  It seems to me that the main contribution of this paper is to provide new evidence that with a well-designed experimental setup, model-based RL methods can, indeed, match model-free methods with much better sample complexity.  The components of this setup aren't particularly novel, but the authors have done a good job of making their method work empirically which sets a higher standard for future model-based methods.   One limitation is that the tasks themselves all have low-dimensional state and actions spaces, which allow the authors to use simple and small fully-connected models together with an explicit paramaterization of the output distribution (as a mean and variance over output states). It is unclear whether the method in its current form would be applicable to more challenging tasks involving higher-dimensional state or action spaces (such as images). Using an explicit paramaterization of a high-dimensional Gaussian is harder to train, and it is not clear if the CEM method they use would scale to high-dimensional action spaces. This may require gradient-based optimization of a paramaterized policy.   Overall, I would suggest a weak accept. The contributions of the paper aren't fundamental, but the respectable empirical results help validate the model-based approach to RL, which I believe to be a useful direction overall for adapting RL methods to more practical applications where improved sample complexity is important.


Practical Deep Stereo (PDS): Toward applications-friendly deep stereo matching
End-to-end deep-learning networks recently demonstrated extremely good performance for stereo matching. However, existing networks are difficult to use for practical applications since (1) they are memory-hungry and unable to process even modest-size images, (2) they have to be trained for a given disparity range. The Practical Deep Stereo (PDS) network that we propose addresses both issues: First, its architecture relies on novel bottleneck modules that drastically reduce the memory footprint in inference, and additional design choices allow to handle greater image size during training. This results in a model that leverages large image context to resolve matching ambiguities. Second, a novel sub-pixel crossentropy loss combined with a MAP estimator make this network less sensitive to ambiguous matches, and applicable to any disparity range without re-training. We compare PDS to state-of-the-art methods published over the recent months, and demonstrate its superior performance on FlyingThings3D and KITTI sets.
: Number of parameters, inference memory footprint, 3-pixels-error (3PE) and mean-absoluteerror on FlyingThings3D (960 × 540 with 192 disparities). DispNetCorr1D [20], CRL [24], iResNeti2 [18] and LRCR [12] predict disparities as classes and are consequently over-parameterized. GC [13] omits an explicit correlation step, which results in a large memory usage during inference. Our PDS has a small number of parameters and memory footprint, the smallest 3PE and smallest or second smallest MAE, depending on evaluation protocol, and it is the only method able to handle different disparity ranges without re-training. Note, that for our method we report two results. The result outside of brackets is obtained using protocol of PSM [2] method, according to which the errors are calculated only for ground truth pixel with disparity < 192. The result in the brackets is calculated according to protocol of CRL [24], DispNetCorr1D [20] and iResNet-i2 [18] methods, according to which the error is calculated only for images where less than 25% of pixels have disparity > 300, as explained in [24]. Inference memory footprints are our theoretical estimates based on network structures and do not include memory required for storing networks' parameters (real memory footprint will depend on implementation). Error rates and numbers of parameters are taken from the respective publications.

Summary -------  This paper presents a method for computing a disparity map from a rectified pair of images. The authors propose an end-to-end learning approach consisting of three modules: the embedding module, the matching module, and the regularization module. The embedding module extracts feature vectors from the image par. The feature vectors from the left image and right image are horizontally offset and concatenated (this is repeated for all disparities under consideration) and the matching module is applied to compress the concatenated feature vectors. The resulting volume is processed by the regularization module which returns a distribution over disparities for each spatial location. The network is trained on full-resolution images. The target for training is a discretiezed Laplace distribution centered around the ground truth disparity and the cross-entropy loss is used to train the network.  At inference the disparity is computed using sub-pixel MAP approximation, which is defined as the expected value of disparity taking into consideration only disparity values around the disparity with maximum posterior probability.  The parts of the architecture that are novel are the matching module, sub-pixel MAP approximation, and using a discretized Laplace distribution as target.  Strengths ---------  - The authors submitted the generated disparity maps to the KITTI online   evaluation server. The method performs well and is bested by only a couple of   recently published papers. The KITTI leaderboard is very competitive and   placing 3rd and 4th is respectable. The results on KITTI are the main reason    why I leaned towards accepting the paper.  - The code will be open-sourced after publication, which makes it easy to   reproduce the results and build upon the ideas presented in this work.  - Ablation studies on the sub-pixel MAP estimator are performed.  Weaknesses ----------  - The contributions of this work (the matching module, sub-pixel MAP   approximation, and using a discretized Laplace distribution during training)   are solid, but not ground-breaking.  Comments --------  Line 62 and line 153: The authors say that using cross-entropy is more "natural" than using L1 loss. Can you explain what you mean by natural?  Line 70: "training with a vanilla L1 penalty". I'm not sure what the word "vanilla" is referring to. Maybe the authors should consider omitting it?  In equation after line 91 the authors apply the softmax function to a three dimensional tensor, whereas the standard definition softmax is on vectors. My guess is that the softmax function is applied on each spatial location. Perhaps the text could mention this or the equation should be written more precisely.  Line 181: "by processing the test set using the ground truth for benchmarking". I'm slightly confused by the meaning of this sentence. My interpretation was that that "they processed the test set, which is used for benchmarking".  Line 182 "without mentioning it" and line 184 "although this is not commendable". I would consider omitting these two sentences, because they  don't really convey any meaning and come across as slightly negative.  Figure 4 and line 210: "Note that SoftArgmin estimate, though completely wrong, is closer to the ground truth than sub-pixel MAP estimate". Maybe consider omitting the work "completely" or re-editing this sentence. After all, If an estimate is completely wrong it can't be better than another estimate.  Spelling --------  Line 46: "It composed of" -> "It is composed of" Line 47: "and not treat" -> "and does? not treat" Line 51: "their disparity range is still is non-adjustable" -> "their disparity range is still non-adjustable" Line 61: "it allows to train network to" -> "it allows to train the network to" Line 68: "to train on a full-size images" -> "to train on full-size images" Line 80: "Than, in 3.4 we" -> "Then, in 3.4 we" Line 205: "we firstly show" -> "we first? show" Line 206: "we train the our PDS network" -> "we train our PDS network" Line 210: "SoftArgmin estimate are" -> "SoftArgmin estimates? are" Figure 2: "Comparison the proposed" -> "Comparison of? the proposed" Figure 4: "The first column shows image" -> "The first column shows the? input? image"


Factored Bandits
We introduce the factored bandits model, which is a framework for learning with limited (bandit) feedback, where actions can be decomposed into a Cartesian product of atomic actions. Factored bandits incorporate rank-1 bandits as a special case, but significantly relax the assumptions on the form of the reward function. We provide an anytime algorithm for stochastic factored bandits and up to constants matching upper and lower regret bounds for the problem. Furthermore, we show how a slight modification enables the proposed algorithm to be applied to utilitybased dueling bandits. We obtain an improvement in the additive terms of the regret bound compared to state-of-the-art algorithms (the additive terms are dominating up to time horizons that are exponential in the number of arms).

The paper proposes to take a step back from several existing combinatorial bandit problems involving factored sets of actions (e.g. Rank-1 bandits). A meta algorithm based on sequential elimination is given and analyzed in two major settings: in general factored problems and in the dueling bandits setting.   TL;DR: The whole algorithmic structure is extremely hard to parse… The improvement over Rank-1 bandits is not clear at all, and I am not an expert in Dueling Bandits to fully judge the impact of TEM on the ``constant” term in the analysis.   Major comments:  1/ One recurrent inconsistency in the writing is that you define atomic arms at the beginning of section 2 and you barely use this vocabulary thereafter ! Arms and atomic arms are often called arms, especially in Definition 1 that actually defines gaps for atomic arms on each dimension.   1/ Algorithm description: For me, this is the main problem of this paper. The algorithm is extremely hard to parse for many reasons: many small inconsistencies (some are listed below) that do not help reading and a quite unclear description (l 130-137). Typically, I 134 : do you use comparisons between arms or atomic arms ? Algorithm 3 (function feedback) seems to do comparisons between atomic arms.  - Algorithm 1 (and 2) :      - M_s is not defined     - This encapsulation of every step of your algorithm is actually more confusing than helping: basically you’re pulling all remaining atomic arm of each factor on each round. This is quite similar to Rank1Elim as far as I understand, and rather natural, but it took me quite some time making sure it was not more complicated than that.  - Algorithm 3: If I understand well, you’re keeping track of all couples of atomic arms in each Module (so for each factor). That’s a quadratic memory complexity, typically of the order of LxK^2. Isn’t it an issue when K (number of atomic arm per factor) becomes large ? This is actually the interesting case of this type of structure I believe.      - In function feedback, the indexing of N^i_s is inconsistent (sometimes N^s_i).      - Using N_{i,j} and N^s_i for two different counters is quite confusing  2/ Link with Rank-1 bandits: TEA does not behave like rank1Elim but it could have been nice to explain better where is the improvement. Indeed, the rank-1 bandits is not a closed problem yet: there is still a gap between the lower bound of [Katariya et al.] and their algorithms (rank1Elim(KL)). If you think you are improving on this gap, it should be stated. If you are not, we would like you to tell us in which setting you are improving on the state of the art. I am really surprised by the empirical results. I see no reason why the rank1ElimKL would perform similarly as your policy, this is not discussed.   3/ Regret bound : It seems like your bound scales in 1/\Delta^2, which looks optimal but it is actually not. Typically, take the stochastic rank-1 bandits problem (which is a subclass of your setting). The deltas you defined are actually the smallest gaps while in [Katariya et al.] they consider the largest gaps (u_1v_1 - u_iv_1 = \Delta^U_i). They show a lower bound in 1/\Delta for such largest-gap definition. Their regret bound does not match their lower bound but it is still a bit tighter than yours as it scale as 1/(mean{u_1,…,u_K}) instead of 1/(min{u_1,…,u_K}).   Minor comments:  - l.22 Player The  - It’s just my personal opinion but I think this paper tries to cover too wide of a scope. I am not an expert in dueling bandit but I know quite well the combinatorial / factored bandits literature in general. I think both contributions you make would be clearer if split in two separate papers. That would give you some space to explain better the behavior of your algorithm and where the improvement comes from.   EDIT: Thank you for the detailed answers to my comments and remarks in the rebuttal. I raised my score a little bit because they were convincing :)


Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
Large batch size training of Neural Networks has been shown to incur accuracy loss when trained with the current methods. The exact underlying reasons for this are still not completely understood. Here, we study large batch size training through the lens of the Hessian operator and robust optimization. In particular, we perform a Hessian based study to analyze exactly how the landscape of the loss function changes when training with large batch size. We compute the true Hessian spectrum, without approximation, by back-propagating the second derivative. Extensive experiments on multiple networks show that saddle-points are not the cause for generalization gap of large batch size training, and the results consistently show that large batch converges to points with noticeably higher Hessian spectrum. Furthermore, we show that robust training allows one to favors flat areas, as points with large Hessian spectrum show poor robustness to adversarial perturbation. We further study this relationship, and provide empirical and theoretical proof that the inner loop for robust training is a saddle-free optimization problem. We present detailed experiments with five different network architectures, including a residual network, tested on MNIST, CIFAR-10, and CIFAR-100 datasets.

Summary:  This paper presents many empirical results on the effects of large batch training based on second order analysis. The results focus on the generalization gap and robustness to adversaries under the setting of large batch training. The first major conclusion is that large batch size training tends to get attracted to areas with higher Hessian spectrum, rather than saddle points. Secondly, large batch training drastically reduces model robustness, and robust training favors parameters with smaller Hessian spectrum.  Strength:  This paper presents many intriguing results under the setting of large batch training. Experiments are comprehensive. Despite lacking theoretical support, the results in this paper help us understand large batch training better and develop new theoretical frameworks. This is also a pioneering result on empirically investigating Hessian-based analysis to understand large batch training, especially on its connections to model robustness.  The presentation of the paper is nice, with many nicely drawn figures and tables. The paper is overall well written. Good job!  Empirical studies in this paper are also done very carefully with sufficient details presented. So I am confident that the experimental results can be reproduced by others.  Weakness:  Despite the Hessian based analysis shows some intriguing properties of the networks under investigation, all experiments in this paper use ReLU networks. In fact, the only contributing term in Hessian of the network comes from the softmax layer (Appendix line 388), which does not contain any trainable parameter. Removing the softmax layer, the network still works as a valid classifier (and we pick the class with largest logit value as the prediction result), but the analysis in this paper would fail as the Hessian is 0 almost everywhere. Given this fact, I am in doubt if the results discovered in this paper actually reflects the property of ReLU networks. In fact, the power of the ReLU network comes from the singular points where Hessian does not exists, which provides non-linearity; without these points the network would become a pure linear network.  I would suggest that conducting empirical analysis on networks with twice-differentiable activations, for example, ELU or softplus, to give more convincing conclusions. Especially, softplus function can approximate ReLU in its limit. The authors can change the scaling parameter in softplus to simulate the case where the Hessian is close to the one of ReLU networks, and compare this result to the existing results presented in this draft.  The theoretical contribution of this paper is also weak; the PSD property of Hessian matrices comes from a simple fact the Hessian of softmax function is  PSD.  Overall Evaluation:  I appreciate the comprehensive experiments, detailed experiment setup, and intriguing findings in this paper. Empirical results are interesting, but are not completely sound theoretically (see the weakness about). Overall I am leaning slightly towards an rejection, but accepting it would not be that bad.  Comments after author feedback: I have carefully read the author feedback. I am glad that the authors conduct additional experiments on twice differentiable activation functions and provide some supporting evidence. I believe the results of this paper are basically correct (despite using only ReLU), but I think the main texts should include more experiments using twice differentiable networks to make the conclusions more convincing (and this requires a relatively large change to this paper). I am fine if the AC wants to accept this paper, and I hope the authors can revise the paper accordingly if accepted.


A theory on the absence of spurious solutions for nonconvex and nonsmooth optimization
We study the set of continuous functions that admit no spurious local optima (i.e. local minima that are not global minima) which we term global functions. They satisfy various powerful properties for analyzing nonconvex and nonsmooth optimization problems. For instance, they satisfy a theorem akin to the fundamental uniform limit theorem in the analysis regarding continuous functions. Global functions are also endowed with useful properties regarding the composition of functions and change of variables. Using these new results, we show that a class of nonconvex and nonsmooth optimization problems arising in tensor decomposition applications are global functions. This is the first result concerning nonconvex methods for nonsmooth objective functions. Our result provides a theoretical guarantee for the widely-used 1 norm to avoid outliers in nonconvex optimization.

This paper studies the condition for absence of spurious optimality. In particular, the authors introduce 'global functions' to define the set of continuous functions that admit no spurious local optima (in the sense of sets), and develop some corresponding definitions and propositions for an extending characterization of continuous functions that admit no spurious strict local optima. The authors also apply their theory to l1-norm minimization in tensor decomposition.   Pros: In my opinion, the main contribution of this paper is to establish a general math result and apply it to study the absence of spurious optimality for a specific problem.     I also find some mathematical discoveries on global functions interesting, which include:     -- In section 2, the paper provides two examples to show that: (i). there exist global functions that are nowhere differentiable; (ii). there may not exist a strictly decreasing path from any point to a global minimum for a coercive global function defined on compact sets. These two non-trivial examples imply that 'global function' may have bad properties.     -- From definition 2.4 to definition 2.6, the paper introduces 'minimum in the sense of sets' and 'weakly global functions'. These definitions are non-trivial in the sense that the crucial property of 'all strict local minima are global minima' is not consistent in the sense of points and sets. This non-equivalence means that the set of 'weakly global functions' is a proper subset of functions whose strict local minima are all global minima, which implies that we may expect more detailed characterization of weakly global functions.  Cons: There are a few places that can be improved. See detailed comments below.  (1) Although the mathematical tools and examples are interesting, the studied problem is not as interesting as the math part. Often we expect a powerful theory to solve a general class of problems, but I find it a bit disappointing that this paper only solves a relatively narrow and less interesting problem. The significance of the main general theorem Proposition 2.7 would be much more appreciated if there are other interesting examples.   (2) Algorithm. In Line 61, the paper claims that one can minimize the function in proposition 1.1 using a local search algorithm. Which local search algorithm can guarantee local optimality for a non-smooth function? To my knowledge, at least subgradient method and coordinate descent method can not.    It is clearly an issue that classical methods cannot guarantee local optimality in general. Thus proving a non-smooth function is a global function is not enough. Whether this is a big issue or not is debatable, but it is misleading to claim local minima of non-smooth functions are easy to find in general.   (3) Some comments on the wording.    (i) The title “A theory on the absence of spurious optimality” is a bit too general. This is not the first theoretical paper that studies spurious optimality, and not the first one that attempts a general theory. It is also not a comprehensive study. Something more specific may be more accurate (perhaps something related to Prop. 2.7?)   (ii) The authors claim “This is the first result concerning nonconvex methods for nonsmooth objective functions”. This claim is questionable as there are quite a few previous papers that analyze neural networks with ReLU activations.    (iii) The statement “It has received a lot of attention recently due to the discovery in [25, 10] stating that the problem generally admits no spurious local minima (i.e. local minima that are not global minima)” is not true. [25] requires extra regularizers (which is crucial), and [10] requires RIP assumption. But the statement seems to imply (2) is a global function, given that the whole paper is about global functions. This is misleading. In fact, without extra restrictions (2) can be a non-global function.   (4) Some technical comments.   --The description of proposition 2.4 is a bit confusing. According to definition 2.4, the global minimum for a set is defined with respect to a particular function f. However, the conclusion focuses on the set of global functions over some particular sets. The assumption and conclusion don't match perfectly. I guess what you intend to claim is that: if f is a global function on X_alpha for some alpha in some index set A, and X_alpha are global minima of f on the union of X_alpha, then f is a global function on the union of X_alpha. This claim is true.    --A point in the proof of lemma 3.2 and proposition 2.3: to utilize proposition 2.3, you need to show that S'' is a global minimum of f_odd on S'. This is not proven in the proof. However, I guess this is implicitly true due to the continuity of f_odd. Correspondingly, is assumption 1 really needed in proposition 2.3?  (5) Some typos. -- Line 411: alpha should be alpha/2. -- Line 473: the term in the beginning should be corrected. Same is it with equation (37). -- In the first equation of (42): 0 in the right hand side should be |yi|. --Equation (4): It is not the clear to me where matrices A in (2) are. Do you assume here that A=I?


Online Reciprocal Recommendation with Theoretical Performance Guarantees
A reciprocal recommendation problem is one where the goal of learning is not just to predict a user's preference towards a passive item (e.g., a book), but to recommend the targeted user on one side another user from the other side such that a mutual interest between the two exists. The problem thus is sharply different from the more traditional items-to-users recommendation, since a good match requires meeting the preferences at both sides. We initiate a rigorous theoretical investigation of the reciprocal recommendation task in a specific framework of sequential learning. We point out general limitations, formulate reasonable assumptions enabling effective learning and, under these assumptions, we design and analyze a computationally efficient algorithm that uncovers mutual likes at a pace comparable to that achieved by a clairvoyant algorithm knowing all user preferences in advance. Finally, we validate our algorithm against synthetic and real-world datasets, showing improved empirical performance over simple baselines.

The authors consider the problem of online reciprocal recommendation (ORR), where a recommendation is successful if it satisfies both parties.  The online reciprocal recommendation is of practical importance as it can model online systems for job-matching, online dating, roommate matching and so on.   The authors model ORR as a planted bipartite graph where the two parties form the two partites and edges are hidden.  A query is directional between partites and the system gets to place two queries u->v’ and v->u’ on each time slot, where u and v are randomly chosen by nature. Moreover, an edge uv is established iff both u->v and v->u queries are made.   Under this model, the authors investigate how many such edges can be established in T = o(n^2) rounds.  They present negative results when a planted graph is unrestricted. In particular, there are graphs where an oblivious algorithm is optimal.  Whereas, when the feedbacks are clusterable they provide an efficient algorithm, namely SMILE, that learn an order-wise optimal number of edges.    The paper also validates the theoretical guarantees by testing them on a real-world dataset.  Pros: * The author initiates a theoretical study of the practically important problem of online reciprocal recommendation. * The proposed model of the ORR is novel and allows for key structural insights.  The quantification of clustering using a notion of covering number is also interesting. * The author provides a negative result: without structural assumption learning is not possible. Specifically, an oblivious online matching strategy matches the best possible performance. * On the positive side the authors show that when the received feedback is clusterable then through joint cluster learning and user matching order-wise optimal learning is possible, i.e. the number of edges uncovered by SMILE is of the same order of an algorithm that knows the planted bipartite graph.  * The paper is well written and easy to follow.   Cons: * The model falls short in capturing the real system as it  * The problem is considered from a learning perspective, whereas the quality of recommendation is overlooked. It is unclear how the learning can be used to improve performance under the proposed model.  The authors do point out this shortcoming.   * The authors claim that the experimental results validate the model. However, in the results section, the authors preprocess the dataset to remove users that have few edges. The authors should defend their validity claim when such alterations are made.  * The definition of a match through the two directed edges seem cumbersome. The authors may consider a different description that can be generalized easily to non-binary ratings. * The proposed model is far from practical, such as binary preference. It is not clear whether the current insights will remain significant in such a setting. The author may consider commenting on that.


The Price of Privacy for Low-rank Factorization
In this paper, we study what price one has to pay to release differentially private low-rank factorization of a matrix. We consider various settings that are close to the real world applications of low-rank factorization: (i) the manner in which matrices are updated (row by row or in an arbitrary manner), (ii) whether matrices are distributed or not, and (iii) how the output is produced (once at the end of all updates, also known as one-shot algorithms or continually). Even though these settings are well studied without privacy, surprisingly, there are no private algorithm for these settings (except when a matrix is updated row by row). We present the first set of differentially private algorithms for all these settings. Our algorithms when private matrix is updated in an arbitrary manner promise differential privacy with respect to two stronger privacy guarantees than previously studied, use space and time comparable to the non-private algorithm, and achieve optimal accuracy. To complement our positive results, we also prove that the space required by our algorithms is optimal up to logarithmic factors. When data matrices are distributed over multiple servers, we give a non-interactive differentially private algorithm with communication cost independent of dimension. In concise, we give algorithms that incur optimal cost across all parameters of interest. We also perform experiments to verify that all our algorithms perform well in practice and outperform the best known algorithm until now for large range of parameters.

SUMMARY: This is mostly a theory paper proposing new algorithms for low-rank matrix factorisation (LRF) that satisfy approximate (delta > 0) differential privacy (DP) and are also suitable for settings where the input matrix is distributed, or streamed, or the output is produced continually. The paper analyses the runtime, space requirements and the multiplicative and additive errors of the proposed algorithms.  There is an attempt in the paper to provide the reader with sufficient background to understand how the proposed algorithms compare to prior work and what are the key new ideas. Unfortunately, the paper is at times difficult to read due to unclear writing.   MAIN COMMENTS: 0/ [suitability] The main paper (8 pages) is essentially an advertisement for the actual, substantial journal paper appearing in the Appendix. Although I realize this appears to be common practice nowadays due to the page limitation, submitting to a journal instead would provide the benefit of the actual technical content of this work (currently in the appendices) being peer-reviewed.  1/ [significance] This could appear as another paper that takes an existing task X and simply looks at what happens when one wants to solve it under DP, proposing a new DP-X algorithm. However, I think X=LRF can be a particularly interesting problem to look at under the DP constraint, for the following reason. In DP we are constantly looking for ways to reduce the amount of noise required in order to reach a given privacy guarantee, and one folk wisdom general advice seems to be to avoid privatising high-dimensional objects (especially when some dimensions can carry little useful statistical information, but still need to be privacy-protected as we don't know a priori which dimensions these are). In studying DP-LRF one is forced to look at this question whether interest in a lower-dimensional description of the data allows one to get away with less noise being added, and this paper answers this question in the positive for the settings studied. I wonder if there might be some substance in this view and if it could help differentiate this paper from other DP-X papers.  2/ [significance] Is the proposed algorithm practical, or is it theoretical at this stage? Specifically, could you compute exact values of epsilon and delta when the algorithm is run? I couldn't spot the values of epsilon or delta in either the main text or in Appendix K.  3/ [originality] To my knowledge the paper is novel, and it is well discussed against existing work, describing the differences in assumptions, theoretical results and algorithmic ideas used. However, there is a bit of confusion especially in the Abstract about whether the proposed algorithms are the first ones to consider a given setting under the privacy constraint (lines 7-9), or whether they are improvements on top of existing ones (line 19).  4/ [clarity] The clarity of writing could be substantially improved at places (for example, in all of the important Section 2). The submission would also really benefit from just careful proof-reading for grammar (although I'm not penalising in my review for bad grammar, fixing even singular vs plural in many places would have made reading the submission easier). The submission has probably not even gone through a spell-checker. Please see also minor comments below for some concrete suggestions.  5/ [significance] Motivation for the newly introduced neighbouring relation definitions Priv1 and Priv2 currently only appears in Appendix B, but I think a summary would deserve to appear in the main paper (space can definitely be gained e.g. in the final Empirical evaluation section), so that the significance of targeting these new privacy settings can be better assessed.  6/ [quality] Is the code for reproducing experiments (available in the supplementary material upload) planned to be released to the public?   STRENGTHS: [+] LRF seems to be an interesting problem to study under DP, and it appears the particular settings considered here had either not been studied before, or the algorithms proposed here perform better.  WEAKNESSES: [-] The quality of the writing (and grammar) unfortunately leaves a lot to be desired. I don't think the paper could be published in the exact same form as it has been submitted for review. [-] Being a theory paper first and foremost, a more suitable venue than NIPS could be a journal where the reviewing process scope would be such that the proofs (currenty in the Appendix) are also carefully peer-reviewed.   MINOR COMMENTS: (Problem 1) Missing range for the variable gamma. (Definition 1) If the output of M is not discrete, a rigorous definition would only consider measurable sets S, rather than all subsets of the range. (36) The sentence is false as written. (73-74) Please provide context about the problem considered by Dwork et al. and Blum et al. (90) Does Big-O with tilde only hide poly log factors in the variable n, or also in other variables (e.g. poly log k)? (170) What is "Question 3"? (205) Do you mean "Algorithm 1" instead of "Figure 1"? (Figure 1) Using different marker shapes would help plot readability when printed in black and white.  ===== Update after author response and reviewer discussion =====  Thank you for the responses to our reviews. I'm happy with the answers to our technical comments, and where appropriate I would like to suggest perhaps incorporating some of these clarifications into the next revision. I don't think the quality of writing was good enough for NIPS in the submitted version of the paper, but I believe this can be fixed in the next revision. Please do have your paper proof-read for style and grammar issues. Assuming these changes are made, I've increased my score from 5 to 7.


Information-based Adaptive Stimulus Selection to Optimize Communication Efficiency in Brain-Computer Interfaces
Stimulus-driven brain-computer interfaces (BCIs), such as the P300 speller, rely on using a sequence of sensory stimuli to elicit specific neural responses as control signals, while a user attends to relevant target stimuli that occur within the sequence. In current BCIs, the stimulus presentation schedule is typically generated in a pseudo-random fashion. Given the non-stationarity of brain electrical signals, a better strategy could be to adapt the stimulus presentation schedule in real-time by selecting the optimal stimuli that will maximize the signal-to-noise ratios of the elicited neural responses and provide the most information about the user's intent based on the uncertainties of the data being measured. However, the high-dimensional stimulus space limits the development of algorithms with tractable solutions for optimized stimulus selection to allow for real-time decision-making within the stringent time requirements of BCI processing. We derive a simple analytical solution of an information-based objective function for BCI stimulus selection by transforming the high-dimensional stimulus space into a one-dimensional space that parameterizes the objective function -the prior probability mass of the stimulus under consideration, irrespective of its contents. We demonstrate the utility of our adaptive stimulus selection algorithm in improving BCI performance with results from simulation and real-time human experiments.

The authors consider the problem of adaptively selecting stimulus presentations online for a BCI based on evoked potentials.   They focus on the problem of presenting the best “flash groups” in a P300 spelling application.  They derive a method to select stimuli that maximize mutual information between a target character and the information likely to be extracted from EEG signals (in the form of classifier scores).  They derive their methods with an eye towards computational efficiency to allow their method to work online during BCI use and demonstrate their method with simulated and real, online experiments.   The authors do a very nice job of motivating the need for online stimulus selection for evoked BCI work and it is clear the authors are addressing an important problem.  However, I feel the paper, in it’s current form, may not be appropriate for NIPS.     My first concern is the work presented here may be very application specific.  It’s not clear how the methods presented here would generalize to problems outside of the evoked BCI paradigm.  Additionally, it seems that many of the technical contributions of the paper (in particular the form of optimization objective) are motivated by the need to run their algorithm in real time and leverage special structure (binary representation) specific to the P300 spelling paradigm.    My other second concern with the paper is that while the authors have simplified the objective function, it’s not clear to me the importance of this. The search space is still (if I understand correctly) over a set of 2^M possible stimulus configurations, which can potentially be quite large.  It seems the authors are currently searching this space with a Greedy algorithm.  It then seems the authors have turned a hard search problem into a slightly less difficult one, but from a technical point of view, it seems this may be an incremental improvement.    Finally, the authors are to be commended for honestly reporting their results on real data, but as they point out, the current results leave room for improvement, as classification accuracy of the target stimuli with their adaptive strategy decreases with time.   The authors indicate they have good reason to believe they may understand the cause of this, but it would be helpful to see this demonstrated in practice.    Finally, a minor concern with the paper is one of technical clarity.  I found the derivations throughout sections 2.2 – 2.3 a bit hard to follow.   For example, it is not clear to me that 3a is the correct equation for the conditional mutual information between Y_{t+1}^h and C*.  It may very well be, but it seems to be there is a term missing in the denominator of the log term.      Response to authors rebuttal: I thank the authors for their detailed and clear response.  After reviewing their response, I now see how their formulation of the objective fundamentally reduces the complexity of the objective which must be calculated during run-time.  Additionally, while I still find the performance during real experiments somewhat concerning, I also acknowledge the substantial decreases in expected stopping times achieved with their method in these real experiments.  For this reason, I am revising my overall score upward.