Hamiltonian Variational Auto-Encoder
Variational Auto-Encoders (VAEs) have become very popular techniques to perform inference and learning in latent variable models: they allow us to leverage the rich representational power of neural networks to obtain flexible approximations of the posterior of latent variables as well as tight evidence lower bounds (ELBOs). Combined with stochastic variational inference, this provides a methodology scaling to large datasets. However, for this methodology to be practically efficient, it is necessary to obtain low-variance unbiased estimators of the ELBO and its gradients with respect to the parameters of interest. While the use of Markov chain Monte Carlo (MCMC) techniques such as Hamiltonian Monte Carlo (HMC) has been previously suggested to achieve this [25,28], the proposed methods require specifying reverse kernels which have a large impact on performance. Additionally, the resulting unbiased estimator of the ELBO for most MCMC kernels is typically not amenable to the reparameterization trick. We show here how to optimally select reverse kernels in this setting and, by building upon Hamiltonian Importance Sampling (HIS) [19], we obtain a scheme that provides low-variance unbiased estimators of the ELBO and its gradients using the reparameterization trick. This allows us to develop a Hamiltonian Variational Auto-Encoder (HVAE). This method can be re-interpreted as a target-informed normalizing flow [22] which, within our context, only requires a few evaluations of the gradient of the sampled likelihood and trivial Jacobian calculations at each iteration.

I am satisfied with the authors' reply, especially about the clarification of volume preservation and Hamiltonian conservation. The additional experiments are also promising.   I raise my score to 7 and agree to accept this paper.   *** original review ********  In this paper, the authors exploit the Hamiltonian sampling updates to construct the posterior family. Due to the volume is preserved asymptotically, the posterior can be explicitly represented, which leads to closed form of the entropy term in the ELBO. With such property, the gradient w.r.t. the parameters of both posterior and the graphical model can be calculated through the Hamiltonian updates, i.e, reverse-mode differentiation trick [5, 6], and thus, stochastic gradient descent can be utilized for parameters updating. The authors tested the proposed algorithm on a simple Gaussian model on sythetic dataset and the generative model on MNIST to demonstrate the benefits of the proposed algorithm.   There are three major issues the authors should addressed.  1, The volume preservation property only holds asymptotically, i.e., the stepsize goes to zero. When introducing the discretization step, i.e., the leapfrog update, such property no longer holds. (Otherwise, there is no need the MCMC step in the Hybrid Monte-Carlo algorithm) Therefore, the claim in line 145-146, "the composition of equations (7)-(9) has unit Jacobian since each quation describes a shear transformation", is not correct. There will be extra error introduced due to the discretization, which leads to the posterior formulation in the paper is only an approximation.   Please correct such misleading claim and disucss the effect of discretization in details.   2, There are severl important references missing. As far as I know, the reverse-mode differentiation (RMD) [5, 6] has been applied to variational inference [1, 2, 3, 4]. (These are only partial work in this direction.) None of such closely related work has been cited, discussed, and empirically compared.   Please check the liturature and cite the related work appropriately.  Moreover, the RMD has explict flaw that the memory cost will be huge if there are many iterations of the Hamiltonian update.   Please discuss such effect in the paper too.  3, The experiment part is weak. The authors only compared to vanilla VAE with CNN on MNIST. The other state-of-the-art variational inference methods, e.g., normalized flow, adversarial variational Bayes, and the most related one proposed in [3, 4, 7]. Without the comprehensive comparison with these algorithms, it is not supportive to claim the benefits of the proposed algorithm.   In sum, I like the idea to exploit the Hamiltonian updates to construct posterior. However, this paper can be improved in many places.  [1] Justin Domke. Generic methods for optimization-based modeling. In Artificial Intelligence and Statistics, pages 318–326, 2012.  [2] Veselin Stoyanov, Alexander Ropson, and Jason Eisner. Empirical risk minimization of graphical model parameters given approximate inference, decoding, and model structure. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 725–733, 2011.  [3] Yisong Yue, Stephan Mandt, and Joseph Marino. Iterative Amortized Inference. International Conference on Machine Learning (ICML), July 2018.  [4] Kim, Y., Wiseman, S., Miller, A. C., Sontag, D., and Rush, A. M. Semi-amortized variational autoencoders. In Proceedings of the International Conference on Machine Learning (ICML), 2018.   [5] Atilim Gunes Baydin and Barak A Pearlmutter. Automatic differentiation of algorithms for machine learning. arXiv preprint arXiv:1404.7456, 2014.  [6] Yoshua Bengio. Gradient-based optimization of hyperparameters. Neural computation, 12(8):1889–1900, 2000.  [7] Tim Salimans, Diederik P Kingma, and Max Welling. Markov chain Monte Carlo and variational inference: Bridging the gap. In International Conference on Machine Learning, pages 1218–1226, 2015.


Discretely Relaxing Continuous Variables for tractable Variational Inference
We explore a new research direction in Bayesian variational inference with discrete latent variable priors where we exploit Kronecker matrix algebra for efficient and exact computations of the evidence lower bound (ELBO). The proposed "DIRECT" approach has several advantages over its predecessors; (i) it can exactly compute ELBO gradients (i.e. unbiased, zero-variance gradient estimates), eliminating the need for high-variance stochastic gradient estimators and enabling the use of quasi-Newton optimization methods; (ii) its training complexity is independent of the number of training points, permitting inference on large datasets; and (iii) its posterior samples consist of sparse and low-precision quantized integers which permit fast inference on hardware limited devices. In addition, our DIRECT models can exactly compute statistical moments of the parameterized predictive posterior without relying on Monte Carlo sampling. The DIRECT approach is not practical for all likelihoods, however, we identify a popular model structure which is practical, and demonstrate accurate inference using latent variables discretized as extremely low-precision 4-bit quantized integers. While the ELBO computations considered in the numerical studies require over 10 2352 log-likelihood evaluations, we train on datasets with over two-million points in just seconds.

Update: read the author feedback and all reviews and still agree the paper should be accepted.  This paper addresses the problem of performing Bayesian inference on mobile hardware (e.g., self-driving car, phone) efficiently. As one would imagine, approaches that operate with discrete values have an advantage in hardware. Variational inference, a method for approximate Bayesian inference, often involves continuous latent variables and continuous variational parameters. This paper’s contribution is to cast everything in the discrete space with an approximating discrete prior. The authors explore and develop variational inference in such a setting and this appears to be a new idea in the field.   I like the approach (called DIRECT) but a few things remain in my mind that were not addressed in the paper. First, let me state that I did not understand all the details of the technique. It was not clear to me how to choose the discretization granularity: with discrete approximations of continuous variables it usually matters how granular the points are. I also don’t understand what exactly DIRECT gives up with the Kronecker matrix representation. It’s clear that it is impossible to operate in high dimensional discrete space so there is probably a smoothness assumption embedded in the representation but I do not have a good intuition about that.   The paper is well written and I like the worked examples with GLMs and neural networks. However, I found the beginning of Section 3 was hard to follow because of notation and the way the matrix W is introduced. Organizationally, the section on “predictive posterior computations” seems like it’s in the middle of the two worked examples. The experimental section was very thorough. With regards to related work, it would also be nice to compare to a more recent method than REINFORCE, which is known to have high variance. Also, whenever it is claimed that you compute the exact ELBO it needs to be clarified that it is only exact *after* the discrete approximation is applied. One minor addendum: a small niggle throughout the paper was claiming that you calculate “unbiased, zero-variance gradient estimators”. It’s not wrong but it just sounds like a roundabout way of saying “exact gradients”.


FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction
The basic principles in designing convolutional neural network (CNN) structures for predicting objects on different levels, e.g., image-level, region-level, and pixellevel, are diverging. Generally, network structures designed specifically for image classification are directly used as default backbone structure for other tasks including detection and segmentation, but there is seldom backbone structure designed under the consideration of unifying the advantages of networks designed for pixellevel or region-level predicting tasks, which may require very deep features with high resolution. Towards this goal, we design a fish-like network, called FishNet. In FishNet, the information of all resolutions is preserved and refined for the final task. Besides, we observe that existing works still cannot directly propagate the gradient information from deep layers to shallow layers. Our design can better handle this problem. Extensive experiments have been conducted to demonstrate the remarkable performance of the FishNet. In particular, on ImageNet-1k, the accuracy of FishNet is able to surpass the performance of DenseNet and ResNet with fewer parameters. FishNet was applied as one of the modules in the winning entry of the COCO Detection 2018 challenge. The code is available at https://github.com/kevin-ssy/FishNet.

The paper proposes a new backbone architecture more suited to various tasks, e.g. image-, region- and pixel-level tasks, than traditional pre-trained backbones. It allows direct backpropagation from deep layers to shallow layers by solving the issue caused by isolated convolutions happening in common architectures, e.g. ResNet or DenseNet. Experimental validation confirms that FishNet backbone is better than comparable ResNet and DenseNet backbones on image classification, and on both object detection and instance segmentation.          The paper is rather clear and easy to follow. The motivations are exposed well and the justification of the idea to solve the issue of gradient propagation in common backbones is appealing. However, the mathematical notations are cumbersome, and not consistent across equations, e.g. the superscripts 'u' disappear between equations (4) and (5). They are also different notations between the equations and Figure 2, see k and K, or different fonts for M function. It could also help to write the name of the variables directly in Figure 2, making the link between the two easier to follow.          Regarding the experimental section, the results presented are convincing on the usefulness of FishNet, especially for object detection and segmentation, where the use of recent frameworks (FPN, Mask R-CNN) is welcome. It would be interesting to have comparisons with deeper versions of ResNe(X)t backbone on MS COCO dataset, as the 101-layer versions are the most common used. It would also be good to evaluate FishNet on the test sets for easier comparisons with the literature, and to have the results from other state-of-the-art methods in the same tables, for both classification on ImageNet and object detection/segmentation on MS COCO. The ablation study seems a bit light, and does not show the relative importance of the main parts introduced, i.e. the fish body and head.        ************* Post-rebuttal recommendation The rebuttal is a bit disappointing since the authors only answer to one of my concern. This said, I like the proposed method and the experiments are overall convincing. I thus stick on my weak accept recommendation.


-regression with Heavy-tailed Distributions
In this paper, we consider the problem of linear regression with heavy-tailed distributions. Different from previous studies that use the squared loss to measure the performance, we choose the absolute loss, which is capable of estimating the conditional median. To address the challenge that both the input and output could be heavy-tailed, we propose a truncated minimization problem, and demonstrate that it enjoys an O( d/n) excess risk, where d is the dimensionality and n is the number of samples. Compared with traditional work on 1 -regression, the main advantage of our result is that we achieve a high-probability risk bound without exponential moment conditions on the input and output. Furthermore, if the input is bounded, we show that the classical empirical risk minimization is competent for 1 -regression even when the output is heavy-tailed.

In this manuscript the authors consider the problem of linear regression in the case where the distributions of the input and output are heavy-tailed. The goal of this task is to find a linear function that is gives the best description for transforming the input into the observed output. Here the term best means that this function should minimize the risk, which is the expected difference between the function applied to the input and the observed output under a specific loss function. Although in many standard work on linear regression the squared loss is considered, the authors argue that for heavy-tailed in- and output the absolute loss is more suited.  To approach the linear regression problem the authors use a truncation method, which finds a linear function that minimizes a rescaled empirical average of a so-called truncation function. The difference between the risk of the true linear regression and that of the truncation method is called the excess risk. The main results of this manuscript provide bounds on the excess risk, under certain conditions on the input distribution. In particular, using epsilon nets, they obtain an improved excess risk bound for linear regression with absolute loss, which they further improve using additional assumptions on the input data. The overall setup of the manuscript is clear. The authors explain the problem of linear regression and comment on different approaches and techniques in the literature, especially related to heavy-tailed distributions. The proofs are easy to follow and I did not find any errors.  The problem of linear regression with heavy-tailed distributions for input and even output data is relevant since many interesting systems in the real-world exhibit such behaviors. The innovative aspect of this work is the combination of absolute loss linear regression and truncation methods for a wide class of truncation functions. My main concern with this work is that although the title and many parts of the text refer to heavy-tailed distributions, these are never defined. I understand that one has an intuitive idea of what heavy-tailed means, but these are well defined objects, see for instance Definition 2.2 or Theorem 2.6 in Foss et al. An introduction to Heavy-tailed and subexponential distributions. The reason I bring this up is that nowhere in any of the proofs could I find any argument that used properties of heavy-tailed distributions. All results in this manuscript are proven under the three main assumptions. It is also worth noting that often real-world data is heavy-tailed with diverging second moment, which violates Assumption 2. Moreover, it is not clear at all if one should expect Assumption 3 to hold for simple heavy-tailed distributions such as Pareto or Log-normal. This is crucial since if this is not the case then it would make the results stated here much less useful.  To summarize, the paper is well-written and addresses a relevant problem by making a novel combination of absolute loss linear regression with truncation methods. The proofs are easy to read and no errors were found in them. However, the manuscript does not make use of any properties of heavy-tailed distributions and does not even discuss the validity of its assumptions for some standard examples of such distributions. Therefore, I would only weakly recommend this paper for acceptance to NIPS 2018. ADDED AFTER FEEDBACK: The authors addressed my questions and concerns regarding the use of the term heavy-tailed and the existence of second moments. Although I personally still believe that working with heavy-tailed (more specifically regularly-varying) distributions in the setting of infinite second moment seems more realistic, I see the authors point of view and am fine with the setting used in this paper. My other small questions where also addressed properly and hence I stand by my previous decision on this paper. Below is a small list of additional comments. Main text: Line 29, [heavy-tailed]: Please provide a definition of this. Equation (2): Why do you need the \frac{x^2}{2} term inside the function. It seems that this forces the finite second moment condition? Could you elaborate a bit on this? Line 77: Here you seem to indicate that a paper from 2011 extends a result from 2012. I did not find any information that the 2012 was available before the one from 2011. Could you maybe rephrase this sentence. Assumption 2: How realistic is the finite second moment assumption with respect to real-world data? Especially in the heavy-tailed regime this might be false most of the times. Could you discuss this a bit? Line 143-144: I do not understand why you would not simply use this relaxed assumption. Could you explain this? Line 201, [the omitted ones…]: What does ones refer to here. This is not really clear from the text. Are they lemmas or equations?


Stochastic Chebyshev Gradient Descent for Spectral Optimization
A large class of machine learning techniques requires the solution of optimization problems involving spectral functions of parametric matrices, e.g. log-determinant and nuclear norm. Unfortunately, computing the gradient of a spectral function is generally of cubic complexity, as such gradient descent methods are rather expensive for optimizing objectives involving the spectral function. Thus, one naturally turns to stochastic gradient methods in hope that they will provide a way to reduce or altogether avoid the computation of full gradients. However, here a new challenge appears: there is no straightforward way to compute unbiased stochastic gradients for spectral functions. In this paper, we develop unbiased stochastic gradients for spectral-sums, an important subclass of spectral functions. Our unbiased stochastic gradients are based on combining randomized trace estimators with stochastic truncation of the Chebyshev expansions. A careful design of the truncation distribution allows us to offer distributions that are varianceoptimal, which is crucial for fast and stable convergence of stochastic gradient methods. We further leverage our proposed stochastic gradients to devise stochastic methods for objective functions involving spectral-sums, and rigorously analyze their convergence rate. The utility of our methods is demonstrated in numerical experiments.

Spectral optimization is defined as finding $\theta$ that minimizes $F(A(\theta)) + g(\theta)$ where $A(\theta)$ is a symmetric matrix and $F$ typically the trace of an analytic function i.e. $F(A) = tr(p(A))$ where $p$ is a polynomial.   They propose an unbiased estimator of $F$ by randomly truncating the Chebyshev approximation to $F$ and doing importance sampling. Moreover, they calculate the optimal distribution for this importance sampling.  They demonstrate how this method would be used for SGD and stochastic Variance Reduced Gradient. Finally, they demonstrate two applications---matrix completion and minimizing log determinant for fitting Gaussian processes.  This paper is very clearly written, given the theoretical depth of the work. The main result is the optimal distribution for random truncation. The authors do an excellent job of defining and motivating this problem, proving their result, and demonstrating its effectiveness in two applications.  I would strongly argue for accepting this paper.  ## Questions ##  1. As pointed out in the paper, several other papers have taken similar approaches to approximating $tr(f(A))$ using Chebyshev approximations e.g. [6, 16, 28, 25]. However, I haven't seen a proof that importance sampling truncated series is strictly better than importance sampling individual terms. Have you been able to prove such a result? 2. In algorithm 1 and 2, it seems like the only stochasticity comes from sampling Rademacher vectors. We can imagine a case where $A(\theta)$ can only be calculated relative to a specific batch of data (e.g. the Hessian of a NN for a specific batch). Are you specifically not treating this case, or do you believe you have accounted for it here? 3. To make the two applications more explicit for people not very familiar with GPs or matrix completion, I would suggest writing out the objective that is being minimized in a form similar to $F(A(\theta)) + g(\theta)$ just to make the correspondence explicit.


Deep Generative Markov State Models
We propose a deep generative Markov State Model (DeepGenMSM) learning framework for inference of metastable dynamical systems and prediction of trajectories. After unsupervised training on time series data, the model contains (i) a probabilistic encoder that maps from high-dimensional configuration space to a small-sized vector indicating the membership to metastable (long-lived) states, (ii) a Markov chain that governs the transitions between metastable states and facilitates analysis of the long-time dynamics, and (iii) a generative part that samples the conditional distribution of configurations in the next time step. The model can be operated in a recursive fashion to generate trajectories to predict the system evolution from a defined starting state and propose new configurations. The DeepGenMSM is demonstrated to provide accurate estimates of the long-time kinetics and generate valid distributions for molecular dynamics (MD) benchmark systems. Remarkably, we show that DeepGenMSMs are able to make long timesteps in molecular configuration space and generate physically realistic structures in regions that were not seen in training data.

The manuscript introduces a new technique for learning Markov State Models from molecular trajectory data. The model consists of several novel ideas, including a generative component that makes it possible to reconstruct molecular structures from the latent space.  Quality: The methodological contributions are convincing, and backed up by both source code and detailed derivations in supporting material (and references to relevant literature). Although the experiments focus on systems of modest size, and are thus mostly proof-of-concept, the modelling strategy itself has several non-standard components, which in my view makes the manuscript a significant Machine Learning contribution. The authors are honest about the strenths and weaknesses of their work.  Clarity: The manuscript is of high quality. It is well written, and the text is accompanied with helpful figures. My only minor concern regarding clarity is that some of the subfigures in Fig 2 - 4 are difficult to read due to their small size.  Originality: The presented methods are new, and represent a clear deviation from earlier work in this area.  Significance: The authors present a convincing case that their approach has fundamental advantages over current techniques for estimating Markov State Models. It is thus likely to have considerable impact on the MSM field. The modelling strategy itself should be of interest more broadly in the Machine Learning community.   Suggestions for corrections:  In the text following eq (1), perhaps you should more explicitly introduce the hyperparameter m as the number of metastable states, to make it clearer to readers not familiar with the MSM literature.  The discussion regarding the rewiring trick is a little brief. Perhaps you could explicitly write out the approximation either in the text or in supporting material. It's also not quite clear how many samples you draw in this approximation.  Eq 7 is not quite clear to me. You write that \rho(y) is the empirical distribution, but how should I think of an empirical distribution over a continuous space y? Or are we talking about a distribution over the fixed set of input structures (in which case I would assume they are all equally likely - i.e. the exact same structure is never visited twice). Please elaborate.  I found your Energy Distance procedure interesting, but could you perhaps briefly state why you would choose this over e.g. a Kullback-Leibler based approach? (since you use the Kullback-Leibler divergence later for measuring the difference between distributions (fig 2d).  The dialanine system seems to be of rather modest size compared to the systems typically studied with MD. It would be nice if the paper concluded with some some statements on the extent to which the DeepResampleMSM and DeepGenMSM scale to such systems.   Minor corrections: Line 42: "is conserves" -> "conserves"  Line 130: "an generated trajectory" -> "any generated trajectory"?  Update after rebuttal period:  I only raised minor issues in my review, which the authors have addressed in their rebuttal. I therefore stand by my original (high) rating of the manuscript.


Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition
The design of a reward function often poses a major practical challenge to realworld applications of reinforcement learning. Approaches such as inverse reinforcement learning attempt to overcome this challenge, but require expert demonstrations, which can be difficult or expensive to obtain in practice. We propose variational inverse control with events (VICE), which generalizes inverse reinforcement learning methods to cases where full demonstrations are not needed, such as when only samples of desired goal states are available. Our method is grounded in an alternative perspective on control and reinforcement learning, where an agent's goal is to maximize the probability that one or more events will happen at some point in the future, rather than maximizing cumulative rewards. We demonstrate the effectiveness of our methods on continuous control tasks, with a focus on highdimensional observations like images where rewards are hard or even impossible to specify.

**Summary of the paper The paper considers a problem of learning a policy that reaches specified goal states (known as events) without an access to the true reward function. The paper proposes a method that alternates between learning a reward function and learning a policy. Algorithmically, the proposed method resembles inverse reinforcement learning/imitation learning. However, unlike existing methods that requires expert trajectories, the proposed method only requires goal states that the expert aims to reach. Experiments show that the proposed method reaches the goal states more accurately than an RL method with a naïve binary classification reward.  **Detailed comments: The paper extends the problem setting of inverse reinforcement learning and proposes a novel method to solve it. The proposed method is interesting, and I enjoy reading the paper. However, I have a concern regarding a gap between the theorems and the proposed practical algorithm.  - By using logistic regression to classify an event label given a state-action pair, the classifier D(s,a) is an estimate of a positive class (event happens) posterior p(e=1|s,a), and 1 – D(s,a) is an estimate of a negative class (event does not happen) posterior p(e=-1|s,a). However, in line 6 of Algorithm 1, both positive and negative class posteriors are used to update \hat{p}(e=1|s,a). This is strange unless log \hat{p}(e=1|s,a) should be understood as a reward function and not as a log of class posterior. For such a case, this choice of reward function seems fine since minimizing log(1-D(s,a)) implies maximizing log(D(s,a)). However, it is not consistent with the theorems. Specifically, the reward function for the Q-value in Theorem 1 is the log of positive class posterior and not the difference between the log posteriors of the two classes which is used in Algorithm 1. I think that this is a big gap between theory and practice, and should be explained better in the paper.  **Minor comments - In line 247, the left side of the relation should be conditioned on the event e_t=1. This also applied to the subsequent gradient equation. - In the IRL method of Fu et al., 2018, the discriminator depends on policy, D(s,a) = f(s,a) / (f(s,a) + pi(a|s)). It is unclear in the paper whether it uses this policy dependent discriminator or the policy independent one used by GAIL and GAN.   =======================================================  I read the rebuttal letter and have a better understanding of the method. I am more confidence on the paper and increase the score.


Reversible Recurrent Neural Networks
Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which can be trained. Reversible RNNs-RNNs for which the hidden-to-hidden transition can be reversed-offer a path to reduce the memory requirements of training, as hidden states need not be stored and instead can be recomputed during backpropagation. We first show that perfectly reversible RNNs, which require no storage of the hidden activations, are fundamentally limited because they cannot forget information from their hidden state. We then provide a scheme for storing a small number of bits in order to allow perfect reversal with forgetting. Our method achieves comparable performance to traditional models while reducing the activation memory cost by a factor of 10-15. We extend our technique to attention-based sequence-to-sequence models, where it maintains performance while reducing activation memory cost by a factor of 5-10 in the encoder, and a factor of 10-15 in the decoder.

Summary: Authors investigate reversible architectures for RNN to reduce the memory requirement of their  training. They build upon the reversible residual network architecture. By allowing reversibility,  RNN can recompute the hidden activation during the backpropagation through time, trading computation for memory.   Authors argue that a naïve reversible RNNs are not able to forget, hence have trouble to solve some simple learning tasks. To tackle this issue, they propose a variant, that store the lost information due to the forgetting. Authors then perform an extensive empirical evaluation on Penn TreeBank, Wikitext-2, Multi30K and IWLST, where they show that the reversible GRU/LSTM are on part and sometime outperform their non-reversible counterparts.   Clarity,  The paper is clearly written and pleasant to read overall.   Section 4  clarity could be improved. It would be informative to report more details about the experiments (model architectures (size of the hidden states), dataset splits..) and what are the final performances of the different model. In addition, the statement ‘the remember task for an ordinary model with forgetting reduces to the repeat task for a perfectly reversible RNN’ is confusing to me?   Quality, Authors perform an extended empirical evaluation to validate their claims on several language modeling and machine translation tasks. My main concern is that the authors do not report previous works scores on those different tasks. Therefore, it is not clear if the baselines studied in this paper are competitive with respect to previous work.   Few remarks: - While section 4 results indicates the memorization capability of NF-RevGRU/NF-RevLSTM, it does not really show their ‘Impossibility of Zero Forgetting’ ?  - Given a fix hidden state size, a reversible architecture have more parameters than its non-reversible counterpart. Is this factor taken into account in the comparison? - It would be informative to reports that additional computational cost in addition to the memory saving.  Originality: To my knowledge, this is the first work exploring reversible architecture for recurrent neural networks.  Significance: The memory cost associated with training RNN is an important limitation. Tackling this issue could allow the exploration of larger RNN models.  Other approaches have try to reduce the memory cost associated with training a RNN, by investigating real-time recurrent learning or recurrent backpropagation (see ‘Unbiased Online Recurrent Optimization (Tallec et al., 2017) or ‘Reviving and Improving Recurrent Back-Propagation’ (Liao, 2018)).  How does reversible architecture compare in term of computation/memory cost to those approaches?   Overall, I think the paper tackles an important problem and I appreciate the extended empirical evaluation. For those reason I lean toward acceptance. Some aspect of the paper could be improved as reporting previous work performances on the different tasks and a comparison with other approach that also tackle the problem of RNN training  memory cost.  After Rebuttal: Thanks for the feedback, as it addresses some of my concern, I will revise my score accordingly. Also it would be nice to refer to state-of-art model for the different task in the main text for completeness.


Adaptive Learning with Unknown Information Flows
An agent facing sequential decisions that are characterized by partial feedback needs to strike a balance between maximizing immediate payoffs based on available information, and acquiring new information that may be essential for maximizing future payoffs. This trade-off is captured by the multi-armed bandit (MAB) framework that has been studied and applied when at each time epoch payoff observations are collected on the actions that are selected at that epoch. In this paper we introduce a new, generalized MAB formulation in which additional information on each arm may appear arbitrarily throughout the decision horizon, and study the impact of such information flows on the achievable performance and the design of efficient decision-making policies. By obtaining matching lower and upper bounds, we characterize the (regret) complexity of this family of MAB problems as a function of the information flows. We introduce an adaptive exploration policy that, without any prior knowledge of the information arrival process, attains the best performance (in terms of regret rate) that is achievable when the information arrival process is a priori known. Our policy uses dynamically customized virtual time indexes to endogenously control the exploration rate based on the realized information arrival process.

Summary: The paper proposes a new multi-armed bandit (MAB) formulation, in which the agent may observe reward realizations of some of the arms arbitrarily before each round. The paper studies the impact of information flows on regret performance by deriving the regret lower bound with respect to information flows. Moreover, the paper proposes an adaptive exploration policy matching the regret lower bound. However, the insight under this setting is somewhat unclear to the reviewer. Therefore, the reviewer suggests voting for “weak accept”.   Significance: The paper studies the problem of MAB with auxiliary information for some arms. The problem is interesting and has not been studied before. The paper provides an adaptive exploration policy with a near-optimal regret upper bound. However, in each round, it is assumed that the player obtains the knowledge of the arms with auxiliary information before choosing an arm to pull. It is hard to find the motivation for this assumption.   Clarity: This paper is easy to follow. However, the clarity of the motivation can be further improved if the authors can give one or two real-world examples of this MAB formulation, especially for the case where the player observes the impact of information flows before choosing arms to pull. Another minor issue is that this paper does not have a conclusion section to summarize the contributions.   Originality: The originality of this paper comes from the new, generalized MAB problem setting that considers auxiliary information of the arms may arrive arbitrarily between pulls.   Technical Quality: The theoretical analyses are solid and the results are elegant. The algorithms are designed to keep the number of observations of a suboptimal arm be of order log(T), as the number of observations is unequal to the number of pulls. The proofs are regular and correct, with nothing new in the methodology.


Empirical Risk Minimization Under Fairness Constraints
We address the problem of algorithmic fairness: ensuring that sensitive information does not unfairly influence the outcome of a classifier. We present an approach based on empirical risk minimization, which incorporates a fairness constraint into the learning problem. It encourages the conditional risk of the learned classifier to be approximately constant with respect to the sensitive variable. We derive both risk and fairness bounds that support the statistical consistency of our methodology. We specify our approach to kernel methods and observe that the fairness requirement implies an orthogonality constraint which can be easily added to these methods. We further observe that for linear models the constraint translates into a simple data preprocessing step. Experiments indicate that the method is empirically effective and performs favorably against state-of-the-art approaches.

Overview: This paper presents an approach to fairness in statistical learning based on surrogate losses. They work in a generalization of the "equality of opportunity" (EO) framework of Hardt, Price, and Srebro in which the learner performs an ERM problem subject to a constraint that (perhaps only approximately) enforces equality of opportunity. The EO framework is relaxed so that we require equality of some abstract loss function across protected attributes rather than equality of probability that the classifier outputs a specific class. The EO constraint is also relaxed so that it only needs to hold approximately.   After introducing this framework, the authors show relax zero-one losses appearing in the objective or the EO constraint to surrogate losses. Specifically, they show how to bound approximation error when the zero-one loss in the ERM objective is replaced with the hinge loss and the zero-one loss in the EO constraint is replaced with the linear loss. For the special case where the function class is an RKHS they show to directly solve the fair ERM problem under this relaxation.  Finally, the authors conduct extensive experiments against comparable fair learning algorithms and find that their surrogate loss approach is competitive for linear classes.   Originality and Significance: On the mathematical side, I was somewhat disappointed with the limited scope of the paper's theory results relative to the breadth suggested by the title. In particular, the results appearing in the main body, theorem 1 and proposition 2, use mostly standard calculations from learning theory and in particular surrogate losses. It is indeed unfortunate that proposition 2 is limited to the special case of hinge loss and linear loss, and does not give a general framework. My conclusion here is that in terms of analysis tools there are not many new results or tricks that a broader audience might hope to take away.  I am willing to accept however that this paper makes a nice conceptual contribution, namely for the special case of their general problem (hinge/linear loss, linear function class) that is the focus of page 5 onward, the experimental evaluation the authors perform suggests that this method indeed outperforms that of comparable algorithms from Hardt et al. and Zafar et al. So I believe the method has practical merit and could be a useful baseline/benchmark for future development in this area. There is one caveat which is that all the algorithms compared against use the same linear representation. It would not suprise me if using one of the algorithms from [18-26] that direct solves the nonconvex problem for a more expressive function class (eg decision forests) can obtain higher accuracy and EO simultaneously, so this seems like an important benchmark.  To make the theory results stronger in the future it would be nice to give comparison results in the vein of proposition 2 for more general loss functions, as well as to try to establish proper classification calibration results in the vein of Bartlett, Jordan, and McAullife.  Quality and Clarity: I found the paper easy to read and follow overall. A couple minor notes on the writing: * People not already familiar with surrogate losses may find the discussion on page 4 unclear. * Try to define all the named losses (\ell_h, \ell_c, \ell_l) in the same place to avoid making people search for their definitions. * Please try to find a way to highlight the conclusions of Table 1 visually, eg highlight highest accuracy and EO for each dataset.  I appreciate the fairly extensive discussion of the experimental setup in the main paper and appendix.


Generative Neural Machine Translation
We introduce Generative Neural Machine Translation (GNMT), a latent variable architecture which is designed to model the semantics of the source and target sentences. We modify an encoder-decoder translation model by adding a latent variable as a language agnostic representation which is encouraged to learn the meaning of the sentence. GNMT achieves competitive BLEU scores on pure translation tasks, and is superior when there are missing words in the source sentence. We augment the model to facilitate multilingual translation and semi-supervised learning without adding parameters. This framework significantly reduces overfitting when there is limited paired data available, and is effective for translating between pairs of languages not seen during training.

Summary  This paper proposes a generative latent variable model for neural machine translation, where inference is performed with variational inference. This extends the work of Zhang et al., 2016, who proposed a conditional model with variational inference. The advantage of the generative model is to force the latent variable to capture more of the semantics of the sentence than the conditional model was able to do. The main disadvantage of this approach is that the value of the latent variable has to be infered during decoding (based on candidate generations).   The paper also shows that a version of this model can be trained in a multilingual setting, that monolingual data can be used as semi-supervised training, and that the inference algorithm can be extended to perform translation with missing words.      Results show that the model generates better translations than the Variational NMT baseline across multiple language pairs and training data sizes, although the improvement is larger in a small data setting. Multilingual training and semi-supervised training also gives substantial improvements in small data settings.   Quality  An alternative version of this model would be to condition the inference network q only on x, not on x and y. This would make inference during decoding more efficient, and as z should encode the semantics of either x or y, it is not clear what the additional benefit is of conditioning on both.   Experiments: why not also compare directly with previous results on more standard datasets, especially with Zhang et al. 2016's reported results? The paper is also missing a comparison with a standard NMT model without latent variables, without which its performance cannot fully be assessed.   Originality  The extension over Zhang et al., 2016 is relatively small in terms of the technical contribution, but I do think that the generative formulation has clear advantages over previous work.  Significance  The main advantage of this approach seems to be in better sampling complexity (more data-efficient learning) than baselines, which is an important direction in machine translation research.     More analysis of what semantics the latent vector is learning to capture would be interesting - for example by sampling x and y given z inference on a observed sentence or sentence pair.   Overall I think this is an interesting contribution that I'd like to see accepted.


Sample Efficient Stochastic Gradient Iterative Hard Thresholding Method for Stochastic Sparse Linear Regression with Limited Attribute Observation
We develop new stochastic gradient methods for efficiently solving sparse linear regression in a partial attribute observation setting, where learners are only allowed to observe a fixed number of actively chosen attributes per example at training and prediction times. It is shown that the methods achieve essentially a sample complexity of O(1/ε) to attain an error of ε under a variant of restricted eigenvalue condition, and the rate has better dependency on the problem dimension than existing methods. Particularly, if the smallest magnitude of the non-zero components of the optimal solution is not too small, the rate of our proposed Hybrid algorithm can be boosted to near the minimax optimal sample complexity of full information algorithms. The core ideas are (i) efficient construction of an unbiased gradient estimator by the iterative usage of the hard thresholding operator for configuring an exploration algorithm; and (ii) an adaptive combination of the exploration and an exploitation algorithms for quickly identifying the support of the optimum and efficiently searching the optimal parameter in its support. Experimental results are presented to validate our theoretical findings and the superiority of our proposed methods.

This looks an interesting result.  Authors suggest a couple of methods, inspired by "mini batch strategy",  for solving linear regression when the signal of interest is sparse and one can observe the data partially.  The problem and their proposed methods are clearly described in the paper.   Theoretical guarantees for convergence of the algorithms are provided which is supported by practical evidence given in the paper.  As the authors suggest, these methods look to outperform other existing methods in term of "sample complexity".  I think result presentation can be improved by paying attention to the following points :  1) I think hiding effect of noise parameter $ \sigma^2 $ in the result presented in table 1 is not a good idea at all. It shows the robustness of the algorithm and omitting it may cause some confusions.  For instance, in the presented result in table 1, it seems just with scaling the problem (sending $ r_min $ to infinity), we can improve the performance of the proposed Hybrid algorithm and make sample complexity independent of $ \epsilon $.  It is not the case because scaling blows up the noise level which is not visible in the way that they present the complexity comparison.   2) It seems choice of the parameter $ s $, defined in line 116, plays a crucial role in the performance of the algorithm.  However, it is not clear to me how the authors choose this parameter for the practical results.  Especially, because the level of sparsity $ s* $ is not known and I assume $ s >= s* $.  It is reasonable to assume an upper bound for $ s* $ and tune $ s $ with standard methods, but neither are mentioned in the paper.  3) I think the comparison in the simulation part is not fair because of the following reasons,   i)  There is no run-time data for the numerical experiments.  Sample complexity is not the only factor that shows the complexity of the algorithm.  In many problems, including compressive linear regression, it is possible to solve the problem with information theoretic bounds for sample complexity at the cost of exponential run-time.  If the claim is having efficient sample complexity, some computational complexity should be included as well.  Moreover, in comparison of different methods, this should be considered.  ii) I think that Dantzig is performing horribly in the synthetic data but pretty well in the real data can be a sign that synthetic data is chosen to artificially fit the proposed method.  4)  The maximum chance that the analysis can guarantee for the success of the Hybrid algorithm, as Theorem 4.4 suggests, is less than 67%, regardless of the number of samples we have.  Although the main point of the work is solving the problem with partially finite observations, not having asymptotic consistency is a weakness for the theoretical result.  In general, I think this is an interesting result while there is a large room for improvement.


Towards Deep Conversational Recommendations
There has been growing interest in using neural networks and deep learning techniques to create dialogue systems. Conversational recommendation is an interesting setting for the scientific exploration of dialogue with natural language as the associated discourse involves goal-driven dialogue that often transforms naturally into more free-form chat. This paper provides two contributions. First, until now there has been no publicly available large-scale dataset consisting of real-world dialogues centered around recommendations. To address this issue and to facilitate our exploration here, we have collected REDIAL, a dataset consisting of over 10,000 conversations centered around the theme of providing movie recommendations. We make this data available to the community for further research. Second, we use this dataset to explore multiple facets of conversational recommendations. In particular we explore new neural architectures, mechanisms, and methods suitable for composing conversational recommendation systems. Our dataset allows us to systematically probe model sub-components addressing different parts of the overall problem domain ranging from: sentiment analysis and cold-start recommendation generation to detailed aspects of how natural language is used in this setting in the real world. We combine such sub-components into a full-blown dialogue system and examine its behavior.

Summary: This paper provides a large-scale data set of 10,000 conversations for the domain of movie recommendation.  Strengths: + The authors will release the first large-scale dataset for conversational movie recommendations, which can be significant for follow-up work on conversational recommendation.   + The authors propose an end-to-end trainable architecture, with sub-components of new neural-based models for sentiment analysis and cold-start recommendations, along with natural language.   Weaknesses: - As this work has the perspective of task-oriented recommendation, it seems that works such as  []  Li, Xiujun, et al. "End-to-end task-completion neural dialogue systems." arXiv preprint arXiv:1703.01008 (2017). are important to include, and compare to, at least conceptually. Also, discussion in general on how their work differs from other chatbox research works e.g.  []  He, Ji, et al. "Deep reinforcement learning with a natural language action space." arXiv preprint arXiv:1511.04636(2015). would be very useful.  -  It is important that the authors highlight the strengths as well as the weaknesses of their released dataset: e.g. what are scenarios under which such a dataset would not work well? are 10,000 conversations enough for proper training?  Similarly, a discussion on their approaches, in terms of things to further improve would be useful for the research community to extend -- e.g.  a discussion on how the domain of movie recommendation can differ from other tasks, or a discussion on the exploration-exploitation trade-off. Particularly, it seems that this paper envisions conversational recommendations as a goal oriented chat dialogue. However, conversational recommendations could be more ambiguous..  - Although it is great that the authors have included these different modules capturing recommendations, sentiment analysis and natural language, more clear motivation on why each component is needed would help the reader. For example, the cold-start setting, and the sampling aspect of it, is not really explained. The specific choices for the models for each module are not explained in detail (why were they chosen? Why is a sentiment analysis model even needed -- can't we translate the like/dislike as ratings for the recommender?)  - Evaluation -- since one of the contributions argued in the paper is "deep conversational recommenders", evaluation-wise, a quantitative analysis is needed, apart from user study results provided (currently the other quantitative results evaluate the different sub-modules independently). Also, the authors should make clearer the setup of how exactly the dataset is used to train/evaluate on the Amazon Turk conversations -- is beam-search used as in other neural language models?  Overall, although I think that this paper is a nice contribution in the domain of movie conversational recommendation, I believe that the authors should better  position their paper, highlighting also the weaknesses/ things to improve in their work, relating it to work on neural dialogue systems, and expanding on the motivation and details of their sub-modules and overall architecture. Some discussion also on how quantitative evaluation of the overall dialogue quality should happen would be very useful.  ==  I've read the authors' rebuttal. It would be great if the authors add some of their comments from the rebuttal in the revised paper regarding the size of the dataset, comparison with goal-oriented chatbots and potential for quantitative evaluation.


Recurrent World Models Facilitate Policy Evolution
A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatiotemporal representations. The world model's extracted features are fed into compact and simple policies trained by evolution, achieving state of the art results in various environments. We also train our agent entirely inside of an environment generated by its own internal world model, and transfer this policy back into the actual environment. Interactive version of paper: https://worldmodels.github.io

Summary:  This paper proposes a new way to develop a world model for reinforcement learning. The focus is on the encoding of the visual world, coupled with a world model that learns based on the compressed representation. The world model is a recurrent version of Bishop’s (1995, neural networks book, chapter 6) mixture of gaussians network. That network outputs the weights of an MOG (using softmax), the means of the gaussians (linear outputs), and the variance (modeled as e^var, so it is a scale parameter). I had not seen a recurrent version of this network before.  The model is very simple: 0) The world is explored using a random policy with many rollouts. 1) The visual world is encoded using a VAE using the visited images gotten in the rollout, call it V. 2) The world model is trained using a recurrent NN with a five-component MOG to predict the next hidden state of V given the current hidden state. Call it M. 3) A very simple (one or two layer) controller is trained using the current hidden state of V and the current hidden state of M, using an evolution strategie method (the Covariance-Matrix Adaptation Evolution Strategy (CMA-ES), which has been shown to work well in high dimensional spaces.  The network achieves state of the art (“solves”) the OPENAI Gym CarRacingv0 task, the first system to do so.  A slightly modified network also achieves state of the art on the DoomTakeCover-v0 task, by first training V using a random policy, and then training M, but then training the controller on it’s own world model, rather than training directly on the task. I.e., the world model is iterated to produce the next hidden state of V, without ever using V, and the controller learns from that. In order for this to work, the controller also has to return a prediction of the agent’s death, as that is how rewards are calculated. This is very cool - it is like a tennis player who simulates playing in his or her mind, and improves as a result. The other difference from the previous model is that the controller not only gets h from the world model, but the c’s as well (it’s an LSTM network).   Quality:  Strengths  This is a well-written and well-reasoned paper, and the results are clear. The use of a variety of techniques from different authors shows a mastery of the field, and many approaches are discussed. As a result, the references are humongous for a NIPS paper: 106 references! This alone, while a bit over the top, creates a valuable reading list for new PhD students interested in RL.  While many of the details of the networks and training are in the supplementary material (there is also an interactive web site I did not look at), I think this is ok.   Weaknesses  In the references, about 20% of them are to the author’s own work (it’s obvious who’s work this is). Dial back that ego, guy! ;-)  Clarity:  The paper is quite clear.   Originality: The combination of techniques is novel. The work combines techniques from many researchers.   Significance: This is in some ways, a very simple model that solves a difficult task. I plan to teach it in my next iteration of my deep learning class.


Deep Structured Prediction with Nonlinear Output Transformations
Deep structured models are widely used for tasks like semantic segmentation, where explicit correlations between variables provide important prior information which generally helps to reduce the data needs of deep nets. However, current deep structured models are restricted by oftentimes very local neighborhood structure, which cannot be increased for computational complexity reasons, and by the fact that the output configuration, or a representation thereof, cannot be transformed further. Very recent approaches which address those issues include graphical model inference inside deep nets so as to permit subsequent non-linear output space transformations. However, optimization of those formulations is challenging and not well understood. Here, we develop a novel model which generalizes existing approaches, such as structured prediction energy networks, and discuss a formulation which maintains applicability of existing inference techniques.

This paper studies the problem of training deep structured models (models where the dependencies between the output variables are explicitly modelled and some components are modelled via neural networks). The key idea of this paper is to give up the standard modelling assumption of structured prediction: the score (or the energy) function is the sum of summands (potentials). Instead of using the sum the paper puts an arbitrary non-linear (a neural network) transformation on top of the potentials. The paper develops an inference (MAP prediction) technique for such models which is based on Lagrangian decomposition (often referred to as dual decomposition, see details below). The training of the model is done by combining this inference technique with the standard Structure SVM (SSVM) objective. The proposed approach is evaluated in 4 settings: OCR, multilabel classification, image tagging, sematic segmentation and is compared against several baselines.  Overall the paper is well written (I think I understood the method without too much effort) and there is a novel component. However, I don't think that the paper is significant enough to warrant a NIPS publication. My skepticism is based on the following three points:  1) The proposed method has no theoretical justification on the inference side. The Lagrangian relaxation technique usually works well when the duality gap is small (Kappes et al., 2015), but can provide arbitrarily bad results otherwise. There is no study of whether the inference problem is solved well or not.  2) The paper uses non-standard datasets and does not compare to any baselines reported in the literature. For OCR, there is a standard dataset of Taskar et al. (2003) and a lot of baselines (see, "Lacoste-Julien et al., Block-Coordinate Frank-Wolfe Optimization for Structural SVMs, ICML 2013" for linear baselines; "Perez-Cruz, Ghahramani, Pontil, Conditional Graphical Models, 2006, http://mlg.eng.cam.ac.uk/zoubin/papers/CGM.pdf" for kernel-based baselines, "Leblond et al., SEARNN: Training RNNs with Global-Local Losses, 2017, https://arxiv.org/abs/1706.04499" for some RNN-based baselines). I do not understand the reasoning for not using the standard dataset on the same tasks.  3) The novelty of the paper is somewhat limited, in part because learning with SSVM and Lagrangian dual of the inference procedure was done before. In particular, the work of Komodakis (2011) did this, but connections to it are not discussed at all (the paper is cited however). The work of Komodakis&Paragios (2009) used an inference method, which is very similar to the one proposed, and is directly comparable in some settings. For example, the OCR datasets (both the version used for experiments and the standard one of Taskar et al. (2003)) contain only 50 different words, this ideally matches the patter-based potentials used by Komodakis&Paragios (2009). I do agree that the proposed method is not identical to the one of Komodakis&Paragios (2009), but I would insist on fair discussion and comparison.   === after the response I would like to thank the authors for addressing some of my questions and hope the corresponding clarifications would be added to the next revision. However, I still think that the contribution is somewhat incremental because the setting is quite limited, but anyway I'm raising my score to 6.


Ridge Regression and Provable Deterministic Ridge Leverage Score Sampling
Ridge leverage scores provide a balance between low-rank approximation and regularization, and are ubiquitous in randomized linear algebra and machine learning. Deterministic algorithms are also of interest in the moderately big data regime, because deterministic algorithms provide interpretability to the practitioner by having no failure probability and always returning the same results. We provide provable guarantees for deterministic column sampling using ridge leverage scores. The matrix sketch returned by our algorithm is a column subset of the original matrix, yielding additional interpretability. Like the randomized counterparts, the deterministic algorithm provides (1 + ) error column subset selection, (1 + ) error projection-cost preservation, and an additive-multiplicative spectral bound. We also show that under the assumption of power-law decay of ridge leverage scores, this deterministic algorithm is provably as accurate as randomized algorithms. Lastly, ridge regression is frequently used to regularize ill-posed linear least-squares problems. While ridge regression provides shrinkage for the regression coefficients, many of the coefficients remain small but non-zero. Performing ridge regression with the matrix sketch returned by our algorithm and a particular regularization parameter forces coefficients to zero and has a provable (1 + ) bound on the statistical risk. As such, it is an interesting alternative to elastic net regularization.

--------------------------After rebuttal---------------------------- I maintain my score. I also strongly approve of the suggestion of highlighting more the experimental results in the main paper. -----------------------------------------------------------------------  The paper describes a novel deterministic selection rule based on ridge leverage scores (RLS). RLSs are well known quantities used in randomized sketching and coreset selection to identify influential samples. Similarly, it is known that sampling **and reweighting** rows of a matrix A according to their RLS produces a sketch that whp approximates the true matrix up to a small multiplicative and additive error.  The authors prove that sorting the rows in descending order (by RLS), and deterministically selecting them until the RLS falls under a carefully chosen threshold is also sufficient to obtain a provably accurate sketch. Therefore, they propose deterministic RLS selection as a convenient and provably accurate rule for column selection, with improved interpretability over optimization based alternative such as lasso and elastic net.  Contribution: The additive-multiplicative spectral bound (Theorem 1) this is the core contribution of the paper. Note that Thm. 2,3,4 are direct consequences of Thm. 1, and are obtained with a straightforward application of results from (Boutsidis et al. 2011), (Cohen et al. 2015), (Cohen et al. 2017), (Papailiopoulos et al. 2014) and (Alaoui and Mahoney, 2015). Similarly, Thm. 5 is almost a direct consequence of (Papailiopoulos et al. 2014). This, together with the only two modifications necessary to existing results, is explained in detail in the complete appendix, which matches and exceed NIPS's standards for clarity. Note that it is probably possible to leverage this method into even more results, for example statistical risk guarantees stronger than Thm 4 using results from  Rudi, Alessandro, Raffaello Camoriano, and Lorenzo Rosasco. "Less is more: Nyström computational regularization." Advances in Neural Information Processing Systems. 2015.  While the paper introduces a new method with several small improvement over (Papailiopoulos et al. 2014), the work remains overall incremental and the significance of the contribution from a technical point of view is limited. The authors should try to stress more and clarify any significant improvement over (Papailiopoulos et al. 2014), and any advantage of RLS over k-rank LS. For example, RLS methods are widely successful in Reproducing Kernel Hilbert Spaces, while k-rank LS are not.  Nonetheless, I still believe this paper could find its audience at NIPS, and weakly suggest acceptance.


NAIS-NET: Stable Deep Networks from Non-Autonomous Differential Equations
This paper introduces Non-Autonomous Input-Output Stable Network (NAIS-Net), a very deep architecture where each stacked processing block is derived from a time-invariant non-autonomous dynamical system. Non-autonomy is implemented by skip connections from the block input to each of the unrolled processing stages and allows stability to be enforced so that blocks can be unrolled adaptively to a pattern-dependent processing depth. NAIS-Net induces non-trivial, Lipschitz input-output maps, even for an infinite unroll length. We prove that the network is globally asymptotically stable so that for every initial condition there is exactly one input-dependent equilibrium assuming tanh units, and multiple stable equilibria for ReL units. An efficient implementation that enforces the stability under derived conditions for both fully-connected and convolutional layers is also presented. Experimental results show how NAIS-Net exhibits stability in practice, yielding a significant reduction in generalization gap compared to ResNets.

Summarization: This paper presents a deep network that is stable and robust against small perturbations. The model is called Non-autonomous input-output Stable Network (NAIS-Net). The network has two characteristics: the first one is non-autonomous in the sense that skip connections are implemented for each step of the unrolled block. The second characteristic is time-invariance because of the tied weights. The proposed model is able to achieve an input-dependent stable state for the output as well as reduce the generalization gap. The stableness can be enforced by special design of the network parameters. In the end, the authors demonstrate their claims by supportive experiments conducted on MNIST and CIFAR datasets.  Strength: 1. The explanatory descriptions and technical details make the work worth of a publication. It is worth to mention that the authors state the motivations clearly by comparing their model with existing works and are able to guide the way directly to the core of their method. Explaining all the necessary technical details and selecting the conclusive ones in the main text without confusing the reader is also a plus.  2. The paper is well structured and explains clearly the background and contributions together with the solutions. Proposals for both fully-connected and convolutional layers are presented.  3. The work presented has its own originality in terms of both model architecture and stability solutions, which makes it a valuable contribution.  4. The authors conduct extensive experiments to support their claims, which include both quantitative and qualitative illustrations. Firgure 3 and 4 demonstrate the convergence stability of the propose method. Specially, Figure 3 shows the effect of each component clearly by comparing different model settings which is convincing.  Weakness: 1. For the technical details, although the authors provide sufficient information in the supplementary material, it's still better to state some of them clearly in the main text. E.g., can you explain more on set P (what it represents? Does it include all the pairs that do not saturate? Why the element (x,u) belongs to R^{n*m}, according to supplementary material?), the function of state \hat{x} for both tanh and Relu activations, etc.   2. It would be better to explain more on general stability analysis, the motivation, intuition, what condition is deemed as stable, etc, in order to have an idea on what you are trying to solve.  3. In the beginning, the authors compare their model with ResNet and existing dynamical systems to emphasize the non-autonomous characteristics. However, it lacks some mathematical comparisons. Specifically, why is your method better than ResNet in terms of stability? How is ResNet's formulation hard to achieve stability?  4. The authors may want to explain the what the dotted curve represents in Figure 3. From the method introduction and experiment section, it's not clear to me the effect of time-invariant construction, e.g., compare with ResNet-NA-Stable.   5. Since the model should benefit from better generalization gap, a direct proof could be injecting some noise and compare your model with ResNet. I wonder if you have done this experiment and how's the results? It seems in Table 1, your model does not outperform ResNet. Can you explain possible reasons? And any quantitative results for patter-dependent processing depth?


Doubly Robust Bayesian Inference for Non-Stationary Streaming Data with β-Divergences
We present the first robust Bayesian Online Changepoint Detection algorithm through General Bayesian Inference (GBI) with β-divergences. The resulting inference procedure is doubly robust for both the parameter and the changepoint (CP) posterior, with linear time and constant space complexity. We provide a construction for exponential models and demonstrate it on the Bayesian Linear Regression model. In so doing, we make two additional contributions: Firstly, we make GBI scalable using Structural Variational approximations that are exact as β → 0. Secondly, we give a principled way of choosing the divergence parameter β by minimizing expected predictive loss on-line. Reducing False Discovery Rates of CPS from over 90% to 0% on real world data, this offers the state of the art.
increases as they move away from the posterior mean, mimicking the KLD. However, once they move far enough, their influence decreases again. This can be interpreted to mean that they are (increasingly) treated as outliers. As β increases, observations are registered as outliers closer to the posterior mean. Conversely, as β → 0, one recovers the KLD which cannot treat any observation as an outlier. In addressing misspecification and outliers this way, our approach builds on the principles of General Bayesian Inference (GBI) [see 6,21] and robust divergences [e.g. 4,15]. This paper presents three contributions in separate domains that are also illustrated in Figs. 1 and 3:
(1) Robust BOCPD: We construct the very first robust BOCPD inference. The procedure is applicable to a wide class of (multivariate) models and is demonstrated on Bayesian Linear Regression (BLR). Unlike standard BOCPD, it discerns outliers and CPS, see Fig. 1 B.
(2) Scalable GBI: Due to intractable posteriors, GBI has received little attention in machine learning so far. We remedy this with a Structural Variational approximation which preserves parameter dependence and is exact as β → 0, providing a near-perfect fit, see Fig. 3.
(3) Choosing β: While Fig. 1 A shows that β regulates the degree of robustness [see also 21,15], it is unclear how to set its magnitude. For the first time, we provide a principled way of initializing β. Further, we show how to refine it on-line by minimizing predictive losses.
The remainder of the paper is structured as follows: In Section 2, we summarize standard BOCPD and show how to extend it to robust inference using the β-D. We quantify the degree of robustness and show that inference under the β-D can be designed so that a single outlier never results in false declaration of a CP, which is impossible under the KLD. Section 3 motivates efficient Structural Variational Inference (SVI) with the β-D posterior. Within BOCPD, we propose to scale SVI using variance-reduced Stochastic Gradient Descent. Next, Section 4 expands on how β can be initialized before the algorithm is run and then optimized on-line during execution time. Lastly, Section 5 showcases the substantial gains in performance of robust BOCPD when compared to its standard version on real world data in terms of both predictive error and CP detection.

This paper builds on recent work in the area of “General Bayesian Inference” (GBI). GBI has been proposed to deal with the “M-open world”, where the class of models we choose does not capture the actual sampling distribution. A remarkable fact is that in this setting standard Bayesian updating can be seen as a method which learns a model by minimising the predictive KL-divergence from the model from which the data were sampled. The problem with the KL-divergence in the case where the model is mis-specified the penalties are much worse in the tails, meaning that the posteriors that are derived can be unsatisfactory for e.g. decision making. Hence, the use of “robust” divergence measures are proposed, which focus instead on the areas of greatest mass. $\beta$-divergences have been proposed to do exactly this. The $\beta$-divergence can be derived from the more general $\alpha,\beta$-divergence by setting $\alpha$ to 1, or from the Bregman divergence family. There are two nice properties of the $\beta$-divergence: posterior inference does not require estimating the data generating density; and it results in robust inference, since observations with low predicted probability under the model are automatically down weighted. This is not for free, and is at the cost of statistical efficiency.   The paper’s first contribution is to take GBI, and apply it to sequential Bayesian inference, where the prior at time t is the posterior from t - 1, to produce a general online inference method that is naturally robust to model misspecification. I feel that some of this could be made clearer. For example, M-open worlds are cited but not explained, and to the general reader it is probably not at all obvious why standard Bayesian inference corresponds to the KL between the fitted model and the data generating mechanism. I also think that the point that using $\beta$-divergence gives your robustness *irrespective of the model* is quite subtle. It is mentioned in the paper (e.g. at the end of 2.1) but I think this point could get missed. The technical contribution here is then to take a specific model: the Product Partition Model (aka Bayesian Changepoint Detection), and derive the inference procedure for the $\beta$-divergence. This essentially boils down to a regression model that is computed over the period since the last changepoint (the run length), with a corresponding model for the probability of the run continuing from one time step to the next. Note that the (extremely long and detailed) derivations are given in the appendix.  The second contribution is to improve the efficiency of GBI in this model using stochastic variance reduced gradient descent (SVRG). This is actually general to GBI, rather than specific to BOCPD, although it’s also a fairly trivial modification from SVRG with KL. There is an extra nuance where the gradient estimates are anchored every few steps, which presumably comes from an insight (failure?) whilst running vanilla SVRG. It would be interesting to know how much is lost (if anything) versus performing full optimisation of the variational parameters (as we know, sometimes SGD provides an additional regularisation effect).   The final technical contribution is of optimisation of $\beta$, including doing so in an online fashion. There are a lot of moving parts in this, even for the static version. It would be interesting in the examples to see how $\beta$ ends up getting set, and how it evolves through the time course of the experiments. Without this is somewhat difficult to estimate the utility of this method of selection, and whether the online updating is necessary or not.  The experiments themselves are two classical examples in CPD. Unsurprisingly the proposed method is more robust to outliers, so mission accomplished in this regard. However, I feel there are multiple ways that these could be improved. Firstly, both of these examples are 1-dimensional; whilst the methods are clearly applicable to higher dimensions, and it would be really interesting to see how they behave. Also, as described in the appendix, there are quite a few hyper parameters and preprocessing steps, and little intuition as to how these are reached. I fear that a lot of experimentation was performed to get to this stage, and that application to a new domain would remain a difficult task.  Specific comments:  Figure 1a is a bit unintuitive. It’s clear that the KL is focusing more on the tails of the distribution. However, the influence has a strange profile for the $\beta$-divergences - going up as you move away before falling away again. It’s hard to say what the practitioner should take from this: one might think to use as small a $\beta$ as possible. However when $\beta$->0 it reduces to the KL-Divergence (again, not obvious at all from the plot). I realise that section 4 discusses optimisation of $\beta$, but I think there is some intuition missing here In the definition of the product partition model, there is a $\Sigma_0$ which has no definition (it is multiplied by $\sigma^2$, which has a conjugate prior) In the algorithm, what is B vs b? What is Geom? Is the $\mathcal{I}$ sampled with or without replacement? In the for loop, it looks like the indexer i isn’t used, and clashes with the $i \in \mathcal{I}$ [22] looks like it has been published under the title "Principles of Bayesian Inference Using General Divergence Criteria" references missing in appendices (latex build)  Note: I didn’t check all of the appendix, but the parts that I did check were all correct.  Note: I have reviewed author's feedback, and glad to see that it addresses the concerns raised both by myself also by R2. I had a misconception of the data being used in the 2nd example. Accordingly I have raised my score from a 7 to an 8.


Near-Optimal Policies for Dynamic Multinomial Logit Assortment Selection Models
In this paper we consider the dynamic assortment selection problem under an uncapacitated multinomial-logit (MNL) model. By carefully analyzing a revenue potential function, we show that a trisection based algorithm achieves an item-independent regret bound of Op ? T log log T q, which matches information theoretical lower bounds up to iterated logarithmic terms. Our proof technique draws tools from the unimodal/convex bandit literature as well as adaptive confidence parameters in minimax multi-armed bandit problems.

Summary:  In the paper, the authors consider an online assortment problem, where each customer chooses his/her purchase from the offered assortment according to a multinomial logit choice model. The preference weights in the choice model are unknown to the seller, who aims to maximize the total revenue under the model uncertainty. The authors consider the unconstrained setting, which allows the seller to offer any subset of products as the assortment. This is different from most online assortment optimization literature, which consider the constrained setting where there is a cardinality constraint on the number of products in an assortment.  The authors first note the fact that, in the unconstrained setting, the optimal assortment lies in the sequence of revenue-ordered assortments. Using the established facts that the sequence of revenue-ordered assortments is nested and the sequence of mean revenue associated with the sequence of assortments, the authors transform the assortment optimization problem to a certain unimodal bandit problem. The authors accomplish a O(sqrt{T}log log T) regret bound, where T is the number of rounds, and the bound is independent of the number of products.   Major Comments:  While online assortment optimization is an important problem in the realm of e-commerce, I am not too convinced if the submission is sufficient for publication. Indeed, cardinality constraints are an important aspect in the e-commerce setting, since the seller can only display a limited number of products out of the set of all products, and the set is typically much larger than the display capacity. Therefore, with the absence of cardinality constraints, the submission shall be mainly weighted in terms of its theoretical contributions.  Given that the transformation to a unimodal optimization is a well-known result in the revenue optimization literature (for example, see the classical papers [TalluriVR04, RusmevichientongT12]), the main theoretical result shall be on addressing the unimodal bandit problem in the case of assortment optimization.  My major concern about the work is my difficulty in comparing the submission with existing works on unimodal bandits. While the submission claims (in Lines 96-97) that existing unimodal bandit literature requires structural assumptions such as convexity and inverse Lipschitz continuity, to my knowledge the design and analysis of Algorithm OSUB in [CP14] (see Sections 4.2, 4.3) seem not require any such sort of structural assumption. In fact, [CP14] seems to only require unimodality, same as the current paper. [CP14] provides a “gap-dependent” regret bound, while the current paper provides a “gap-independent” regret bound, where “gap” here refers to the difference between the optimal reward and a suitably defined second best reward. Understandably, we cannot directly compare a gap-independent regret bound with a gap-dependent one. Thus, we cannot conclude that the proposed algorithm in the submission achieves a better bound than Algorithm OSUB.  In relation to the previous comments, for assessment it will be beneficial for the authors to provide concrete examples (say in the supplementary material) on showing that structural assumptions such as convexity and inverse Lipchistz continuity (I believe that the literature are essentially [AFHKR13, YM11]) do not hold in the current problem setting.   (Post Rebuttal) I am convinced of the paper's technical novelty after the author's feedback. I feel that the improvement from log t to loglogt somehow overcome the models' limitation on the lack of capacity constraint.   Minor Comments: It shall be interesting to numerically compare the proposed algorithms with existing unimodal bandit algorithms [CP14, YM11] (in particular Algorithm OSUB in the former), and investigate if the proposed algorithms indeed suit better for the unconstrained assortment optimization problem empirically.  References [AFHKR13] A. Agarwal, D. P. Foster, D. Hsu, S. M. Kakade, and A. Rakhlin. Stochastic convex optimization with bandit feedback. SIAM Journal on Optimization, 23(1):213–240, 2013. [CP14] R. Combes and A. Proutiere. Unimodal bandits: Regret lower bounds and optimal algorithms. In ICML, 2014. [RT12] P. Rusmevichientong and H. Topaloglu. Robust assortment optimization in revenue management under the multinomial logit choice model. Operations Research, 60(4):865–882, 2012. [TVR04] Talluri, K., and G. J. van Ryzin. 2004. Revenue management under a general discrete choice model of consumer behavior. Management Science 50 (1): 15–33. [YM11] J. Y. Yu and S. Mannor. Unimodal bandits. In ICML, 2011.


Overcoming Language Priors in Visual Question Answering with Adversarial Regularization
Modern Visual Question Answering (VQA) models have been shown to rely heavily on superficial correlations between question and answer words learned during training -e.g. overwhelmingly reporting the type of room as kitchen or the sport being played as tennis, irrespective of the image. Most alarmingly, this shortcoming is often not well reflected during evaluation because the same strong priors exist in test distributions; however, a VQA system that fails to ground questions in image content would likely perform poorly in real-world settings.
In this work, we present a novel regularization scheme for VQA that reduces this effect. We introduce a question-only model that takes as input the question encoding from the VQA model and must leverage language biases in order to succeed. We then pose training as an adversarial game between the VQA model and this question-only adversary -discouraging the VQA model from capturing language biases in its question encoding. Further, we leverage this question-only model to estimate the increase in model confidence after considering the image, which we maximize explicitly to encourage visual grounding. Our approach is a model agnostic training procedure and simple to implement. We show empirically that it can improve performance significantly on a bias-sensitive split of the VQA dataset for multiple base models -achieving state-of-the-art on this task. Further, on standard VQA tasks, our approach shows significantly less drop in accuracy compared to existing bias-reducing VQA models.

Bias in Visual Question Answering dataset can harm the image-grounding of the model, by relaying on the question modality, and prone to failure in the general real-life case. The paper deals with this issue by discouraging the VQA model from capturing language biases in its question encoding, alternating the question embedding to reduce question-only model in an adversarial setup. Moreover, the paper deals with the nuance of language bias in VQA, by encouraging visual grounding via mutual information of image and answer given the question.     Strengths:  • Compared to GVQA method, the adversarial loss is simpler and more elegant.  • The mutual information between answer and image is interesting. To my knowledge this is original component. It captures the nuance in question bias for VQA, whether the bias is attributed to annotations, or it is a real-world bias, e.g, sky is blue. This component improves the results and also stabilize training over different hyper-parameters.  • Comprehensive evaluation on SOTA UpDn, and not only SAN. Also, evaluation on both VQA v2 and VQA v1.  • The paper is well organized and easy to read, the contributions clearly laid out.   Weaknesses:  • Evaluation on Oracle(Ours, SAN) reported only in the supplementary. In my opinion, it should be part of the paper. To me, this is the most significant score, significant model should be useful to improve the test score of VQA2.0 dataset, which is still far from being solved. If I understand correctly from the supplementary, your model improves by 3.46%, while Oracle(GVQA, SAN)  by 4.42%. The paper should discuss weaknesses as well.  • A diagnose of stronger f_Q networks will be interesting. A 2-NN network is way too naïve. Why not trying some sort of LSTM variation, which is the common practice? • I would appreciate qualitative analysis, i.e, showing images that your model succeeds on and GVQA failed.  • Technical information is missing, size of train/test split, training parameters: optimization method used, time of training, hardware, number of parameters for each model, framework used.  • I wonder, is it possible to run your model on the train+val/test split of VQA2.0, which usually not correlates with score on train/val split.  Conclusion: The paper discuss an interesting loss to deal with vqa bias while training.  While it works better on VQA-CP v2, surprisingly the Oracle version is weaker than GVQA, a better discussion is expected. Also technical information given is limited.  After author response:  I'm happy the authors agreed to add suggested important results to the revised version. I believe this paper will be a good contribution to the conference, and hopefully will enlighten a future research related to bias in QA tasks.


On Binary Classification in Extreme Regions
In pattern recognition, a random label Y is to be predicted based upon observing a random vector X valued in R d with d ≥ 1 by means of a classification rule with minimum probability of error. In a wide variety of applications, ranging from finance/insurance to environmental sciences through teletraffic data analysis for instance, extreme (i.e. very large) observations X are of crucial importance, while contributing in a negligible manner to the (empirical) error however, simply because of their rarity. As a consequence, empirical risk minimizers generally perform very poorly in extreme regions. It is the purpose of this paper to develop a general framework for classification in the extremes. Precisely, under non-parametric heavy-tail assumptions for the class distributions, we prove that a natural and asymptotic notion of risk, accounting for predictive performance in extreme regions of the input space, can be defined and show that minimizers of an empirical version of a non-asymptotic approximant of this dedicated risk, based on a fraction of the largest observations, lead to classification rules with good generalization capacity, by means of maximal deviation inequalities in low probability regions. Beyond theoretical results, numerical experiments are presented in order to illustrate the relevance of the approach developed.

Summary This paper introduces theoretical analysis of learning a classifier in an extreme region, where the norm of input vector is very large. The authors describe a new definition of natural and asymptotic notion of risk for predictive performance in extreme regions. The theoretical properties of definition are fully investigated and empirical performance was validated by synthetic datasets.  Strengths [Motivation] Applicable well to clinical data analysis. Sometimes, large feature values are very rarely observed, but its meaning is significantly important.  [Theoretical analysis] The authors presented some interesting theoretical results on learning a classifier in extreme regions, though it’s difficult to follow every step in the proof.  Weaknesses [Positive orthant assumption] I’m not quite convinced to analyze a classifier by restricting an input space to the positive orthant. We could normalize the training data to fit them in the positive orthant, but it’s rarely conducted. Moreover, in section 3, a classifier is defined on the unit sphere. All theories in section 2 could be generalized to the unit sphere?  [Fixing alpha] It seems to be too restrictive to fix alpha=1. It could be possible to generalize the theory to smaller alpha values?  [Lack of comparison] There exist many similar works to learn a classifier with heavy-tail distributions. This paper doesn’t compare any previous works and discussion on the difference between the proposed algorithm and existing ones. Moreover, the proposed algorithm is just learning a classifier by sub-sampling instances with large values.  [Lack of real-world experiments] Since this paper doesn’t contain any results on real-world datasets, it’s very difficult to understand a practical importance of the proposed method.  == Updated review == I’ve read authors’ response on all reviewers’ comments. My original concerns are fully addressed, though I still believe that empirical justification is relatively weaker than the novelty of theoretical analysis. During discussion, I’ve changed my original score to 6 (weak accept).


Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Generating long and coherent reports to describe medical images poses challenges to bridging visual patterns with informative human linguistic descriptions. We propose a novel Hybrid Retrieval-Generation Reinforced Agent (HRGR-Agent) which reconciles traditional retrieval-based approaches populated with human prior knowledge, with modern learning-based approaches to achieve structured, robust, and diverse report generation. HRGR-Agent employs a hierarchical decisionmaking procedure. For each sentence, a high-level retrieval policy module chooses to either retrieve a template sentence from an off-the-shelf template database, or invoke a low-level generation module to generate a new sentence. HRGR-Agent is updated via reinforcement learning, guided by sentence-level and word-level rewards. Experiments show that our approach achieves the state-of-the-art results on two medical report datasets, generating well-balanced structured sentences with robust coverage of heterogeneous medical report contents. In addition, our model achieves the highest detection precision of medical abnormality terminologies, and improved human evaluation performance.

Summary The paper presents an approach using hierarchical reinforcement learning to address the problem of automatically generating medical reports using diagnostics images. The approach first predicts a sequence of hidden states for each sentence, and deicdes when to stop, and a low level model takes the hidden state and either retrieves a sentence and uses it as an output or passes control to a generator which generates a sentence. The overall system is trained with rewards at both sentence level as well as word-level for generation. The proposed approach shows promise over ablations of the proposed model as well as some sensible baseline CNN-RNN based approaches for image captioning.  Strengths + Paper provides the experimental details of the setup quite thoroughly. + Paper clearly mentions the hyperparameters used for training.  Weaknesses  Motivation 1. It would be nice to explain the motivation for the model more clearly. Is the idea that one wants to be able to retrieve the templated sentences and generate the less templated ones? How is mixing generation and retrieval helping with more common sentences vs less common sentences problem?  2. It would be good to perform evaluation with actual patient outcomes instead of evaluation using metrics like CIDEr or even human evaluation on Mechanical Turk. In a lot of ways, it seems important to get validation in terms of patient outcomes instead of M-Turk since that is what matters at the end of the day. It would also be good to take different risk functions (for saying different things) into account in the model.    Notation 3. Eqn. 14 and 15, the notation for conditioning inside the indicator variable is somewhat confusing, and there does not seem an obvious need for it to be there. Also, it would be good to make the bracket after I(z_i > 1 / 2) larger to indicate that it is over both the terms in the gradient.   Formulation  4. Choice of delta-CIDEr, especially for providing the reward at each timestep of the generation seems very odd and unjustified. Ideally the reward that is optimized for should be the reward provided at the end of an entire sequence CIDEr(w_1, …, w_T, gt), instead of that we are optimizing a proxy, which is the described delta CIDEr score, which might have a problem, that the reward for a word might be low at the given timestep, but might be a part of say, a 4-gram which would get us really high reward (at the sequence level), yet it would not be given importance because the marginal improvement at the world level is low. A more principled approach might be to get intermediate rewards by training a critic using an approach similar to [A].  5. How long do the reports tend to be? What is the cost for summarizing things incorrectly? It woud be good to do attention on the image and indeed verify that the model is looking at the image when providing the caption. How often does the model retrieve vs generate? (*)  *Preliminary Evaluation* The paper is quite thorough in its execution, has a number of sensible baselines and proposes a sensible model for a real-world task. Although the model itself is not very novel, it seems like a decent application paper that is well executed and clear.  References: [A]: Bahdanau, Dzmitry, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016. “An Actor-Critic Algorithm for Sequence Prediction.” arXiv [cs.LG]. arXiv. http://arxiv.org/abs/1607.07086.  Final Comments -------------------------------------- After having read the other reviews as well as the author rebuttal, I am unchanged in my opinion about this paper. I think it would be a good application paper at NIPS -- the idea of mixing retrieval with generation in the context of the application makes a lot of sense (although the approach itself is not very novel as pointed out by R4 -- copy mechanisms have been used for sequence to sequence models). In light of the thoroughness of the execution, i still lean towards accept, but it would not be too bad to reject the paper either.