Non-Stationary Spectral Kernels
We propose non-stationary spectral kernels for Gaussian process regression by modelling the spectral density of a non-stationary kernel function as a mixture of input-dependent Gaussian process frequency density surfaces. We solve the generalised Fourier transform with such a model, and present a family of non-stationary and non-monotonic kernels that can learn input-dependent and potentially longrange, non-monotonic covariances between inputs. We derive efficient inference using model whitening and marginalized posterior, and show with case studies that these kernels are necessary when modelling even rather simple time series, image or geospatial data with non-stationary characteristics.

--- Updated Comments Since Rebuttal ---
This seems like very nice work and I hope you do keep exploring in your experiments.  I think your approach could be used to solve some quite exciting problems.
Some of the experiments are conflating aspects of the model (non-stationarity in particular) with what are likely engineering and implementation details, training differences (e.g. MAP), and initialization.  The treadplate pattern, for instance, is clearly mostly stationary. Some of these experiments should be modified or replaced with experiments more enlightening about your model, and less misleading about the alternatives.  It would also be valuable, in other experiments, to look at the the learned GP functions, to see what meaningful non-stationarity is being discovered.
Your paper is also very closely related to "Fast Kernel Learning for Multidimensional Pattern Extrapolation" (Wilson et. al, NIPS 2014), which has all the same real experiments, same approach to scalability, developed GPs specifically for this type of extrapolation, and generally laid the foundation for a lot of this work. This paper should be properly discussed in the text, in addition to [28] and [11], beyond swapping references.  Note also that the computational complexity results and extension to incomplete grids is introduced in this same Wilson et. al (2014) paper and not Flaxman et. al (2015), which simply makes use of these results.  Wilson & Nickisch (2015) generalize these methods from incomplete grids to arbitrary inputs.  Note also that a related proposal for non-stationarity is in Wilson (2014), PhD thesis, sections 4.4.2 - 4.5, involving input dependent mixtures of spectral mixture kernels.
Nonetheless, this is overall very nice work and I look forward to seeing continued experimentation.  I think it would make a good addition to NIPS.  I am happy to trust that the questions raised will be properly addressed in the final version.
---
The authors propose non-stationary spectral mixture kernels, by (i) modelling the spectral density in the generalized Bochner's theorem with a Gaussian mixture, (ii) replacing the resulting RBF component with a Gibbs kernel, and (iii) modelling the weights, length-scales and frequencies with Gaussian processes.
Overall, the paper is nice. It is quite clearly written, well motivated, and technically solid. While interesting, the weakest part of the paper is the experiments and comparisons for texture extrapolation. A stronger experimental validation, clearly showing the benefit of non-stationarity, would greatly improve the paper; hopefully such changes could appear in a camera ready version.  The paper also should make explicit connections to references [28] and [11] clear earlier in the paper, and should cite and discuss "Fast Kernel Learning for Multidimensional Pattern Extrapolation", which has all of the same real experiments, and the same approach to scalability.  Detailed comments for improvement:
- I didn't find there was much of a visual improvement between the SM kernel and the GSM kernel results on the wood texture synthesis, and similar performance from the SM kernel as the GSM kernel could be achieved for extrapolating the treadplate pattern, with a better initialization. It looks like the SM result on treadplate got trapped in a bad local optima, which could happen as easily with the GSM kernel. E.g., in this particular treadplate example, the results seem more likely to be a fluke of initialization than due to profound methodological differences. The comparison on treadplate should be re-evaluated. The paper should also look at some harder non-stationary inpainting problems -- it looks like there is promise to solve some really challenging problems here. Also the advantage of a non-stationary approach for land surface temperature forecasting is not made too clear here -- how does the spectral mixture kernel compare?
- While the methodology can be developed for non-grid data, it appears the implementation developed for this paper has a grid restriction, based on the descriptions, experiments, and the imputation used in the land-surface temperature data. Incidentally, from an implementation perspective it is not hard to alleviate the need for imputation with virtual grid points. The grid restriction of the approach implemented here should be made clear in the text of the paper.
- The logit transform for the frequency functions is very nice, but it should be clarified that the Nyquist approach here only applies to regularly sampled data. It would be easy to generalize this approach to non-grid data. It would strengthen the method overall to generalize the implementation to non-grid data and to consider a wider variety of problems. Then the rather general benefits of the approach could be realized much more easily, and the impact would be greater.
- I suspect it is quite hard to learn Gaussian processes for the length-scale functions, and also somewhat for the frequency functions. Do you have any comments on this?  It would be enlightening to visualize the Gaussian process posteriors on the mixture components so we can see what is learned for each problem and how the solutions differ from stationarity. One of the really promising aspects of your formulation is that the non-stationarity can be made very interpretable through these functions.
- Why not perform inference over the Gaussian processes?
- There should be more discussion of reference [11] on "Generalized spectral kernels", earlier in the paper; Kom Samo and Roberts also developed a non-stationary extension to the spectral mixture kernel leveraging the generalized Bochner's theorem. The difference here appears to be in using Gaussian processes for mixture components, the Gibbs kernel for the RBF part, and Kronecker for scalability. "Fast kernel learning for multidimensional pattern extrapolation" [*] should also be referenced and discussed, since it proposes a multidimensional spectral mixture product kernel in combination with Kronecker inference, and develops the approach for texture extrapolation and land surface temperature forecasting, using the same datasets in this paper. Generally, the connection to [28] should also be made clear much earlier in the paper, and section 4 should appear earlier.  Moreover, the results in [*] "Fast Kernel Learning for Multidimensional Pattern Extrapolation", which have exactly the same real experiments, indicate that some of the results presented in your paper are from a bad initialization rather than a key methodological difference.  This needs to be remedied and there need to be more new experiments.
- Minor: "the non-masked regions in Figure 3a and 3f" in 5.2 is meant to refer to Figure 4.


Nearest-Neighbor Sample Compression: Efficiency, Consistency, Infinite Dimensions
We examine the Bayes-consistency of a recently proposed 1-nearest-neighbor-based multiclass learning algorithm. This algorithm is derived from sample compression bounds and enjoys the statistical advantages of tight, fully empirical generalization bounds, as well as the algorithmic advantages of a faster runtime and memory savings. We prove that this algorithm is strongly Bayes-consistent in metric spaces with finite doubling dimension -the first consistency result for an efficient nearest-neighbor sample compression scheme. Rather surprisingly, we discover that this algorithm continues to be Bayes-consistent even in a certain infinitedimensional setting, in which the basic measure-theoretic conditions on which classic consistency proofs hinge are violated. This is all the more surprising, since it is known that k-NN is not Bayes-consistent in this setting. We pose several challenging open problems for future research.

This work develops a compression-based algorithm for multiclass learning; the authors claim the method is both efficient and strongly Bayes-consistent in spaces of finite doubling dimension. They also provide one example of a space of infinite doubling dimension with a particular measure for which their method is weakly Bayes consistent, whereas the same construction leads to inconsistency of k-NN rules. Overall, I think this paper is technically strong and seems to develop interesting results, but I have a few concerns about the significance of this paper which I will discuss below. If the authors can address these concerns, I would support this paper for acceptance.
Detailed comments:
I did not check the proofs in the appendix in detail but the main ideas appear to be correct. My main comments are on the efficiency of the method (which I expect the authors can easily address) and on the significant of the results in infinite dimensions.
(I) I have a question about the efficiency of the algorithm:
It is claimed that the complexity of Algorithm 1 is $O(n^4)$. I can see that the for loop traverses a set $\Gamma$ of cardinality at most $O(n^2)$, and in line 5 of the algorithm we pay at most $O(n)$ (which globally yields $O(n^3)$). What is the cost of constructing a $\gamma$-net? Is this $O(n^2)$, and if so, what is the algorithm for doing this? I do not yet believe the efficiency claim by the authors until this part is explained.
It is absolutely necessary that Algorithm 1 is efficient, as otherwise one can just turn to standard $k$-NN methods (not based on compression) in order to obtain the same results, although through different methods.
(II) Infinite dimensions:
It is not clear to me why the results/discussion related to finite dimensions is significant.
If I understand correctly, the results of Cérou and Guyader state that, for a particular (realizable) distribution, no $k$-NN learner (where $k$ grows with $n$ in the appropriate way) is Bayes-consistent. Thus, for a particular distribution, $k$-NN fails to be Bayes-consistent. But perhaps it succeeds for other realizable distributions satisfying equation (3). I can see that in the line after Theorem 3, the authors *seem to* claim that in fact the paper of Cérou and Guyader implies that *any* (realizable) distribution satisfying equation (3) will cause $k$-NN to be Bayes inconsistent; am I correct in this interpretation of Line 221? If so, is this easy to see from Cérou and Guyader's paper?
On the other hand, the authors show that for a particular metric space of infinite doubling dimension and for a particular realizable distribution, KSU is Bayes-consistent. This is interesting provided that $k$-NN fails to be Bayes consistent for all constructions satisfying equation (3), but it is not interesting if the point is simply that, for the Preiss's construction, $k$-NN is not Bayes consistent while KSU is.
It is therefore critical if the authors clarify the extent to which $k$-NN fails to be Bayes consistent (is it for all constructions satisfying equation (3) or just for Preiss's construction?).
Also, the authors should mention Biau, Bunea, Wegkamp (2005), ``On the kernel rule for function classification'', because the paper of Cérou and Guyader (near the top of page 13 therein, or page 352 from internal numbering) claims that Biau et al. also show a nearest neighbor-type method that is weakly Bayes consistent in a space of infinite dimension. Mentioning this result (slightly) weakens the force of the authors' results in infinite dimensions.
Minor comments:
Lines 78-80: How do the authors know about the key motivation behind [15, 23, 16]?
Line 277: I think $n(\tilde{\gamma})$ should be replaced by $n(\gamma)$.
In the 3-point proof sketch of Theorem 2, from lines 262-281:
The paper would benefit from a more explicit/longer explanation part 3.
UPDATE AFTER REBUTTAL
I am satisfied with the authors' responses and recommend acceptance. Also, regarding the confusion about the citations, I'm happy the authors were able to straighten out which paper was actually intended, while also pointing out the disadvantages of the approach from the older paper.


Policy Gradient With Value Function Approximation For Collective Multiagent Planning
Decentralized (PO)MDPs provide an expressive framework for sequential decision making in a multiagent system. Given their computational complexity, recent research has focused on tractable yet practical subclasses of Dec-POMDPs. We address such a subclass called CDec-POMDP where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our main contribution is an actor-critic (AC) reinforcement learning method for optimizing CDec-POMDP policies. Vanilla AC has slow convergence for larger problems. To address this, we show how a particular decomposition of the approximate action-value function over agents leads to effective updates, and also derive a new way to train the critic based on local reward signals. Comparisons on a synthetic benchmark and a real world taxi fleet optimization problem show that our new AC approach provides better quality solutions than previous best approaches.

The paper presents a policy gradient algorithm for a multiagent cooperative problem, modeled in a formalism (CDEC-POMDP) whose dynamics, like congestion games, depend on groups of agents rather than individuals.
This paper follows the theme of several similar advances in theis field of complex multiagent planning, using factored models to propose practical/tractable approximations. The novelty here is the use of parameterized policies and training algorithms inspired by reinforcement learning (policy gradients). The work is well-motivated, relevant, and particularly well-presented. The theoretical results are new and important. The experiments (on a large taxi domain) are convincing. Overall, the paper advances the state-of-the-art.
I have several questions:
Q1. What is meant by a "DBN" (... dynamic Bayesian network?) and how does one sample from it? Why is this model shown without even a high-level description? How is it relevant to the rest of the paper?
Q2. What is the architecture of the neural net? What is the representation of the input to the network? How large is it (number of parameters, number of layers, units per layer, non-linearities, etc.) Are there two entirely separate networks, one for the critic and one for the policy, or is it a multiheaded network? What optimizer is used, what learning rates, etc. Without these details, these results would not be reproducible.
Q3. What is the main motivation for using policy gradient? At first, I thought it would reduce memory as opposed to sample-based tabular value methods. But, it seems like the count tables may dominate the space complexity. Is that true? If not, it would be great to show the reduction in memory by using this method. If so, is the answer then that the benefit of parameterized policy + policy gradient is due to generalization across similar states?
Q4. The results show that the algorithm produces better results than the competitor, but it's possible that this setup is particularly more sensitive to parameter values. Even through the authors' claims through the paper, it is implied that the algorithm could be sensitive to the particular choices of training setup. Is the algorithm robust to slight perturbations in values?
Comments to improve the paper (does not affect decision):
C1. Give some background, even if just a few sentences, on the relationship between policy gradient and actor-critic. This seemed to be assumed, but some readers may not be familiar with this and may be confused without any introduction. (Same goes for DBN, or just drop the term if it's unnecessary.)
C2. Somewhere it should be stated that proofs are in the appendix.
C3. The min_w should not be in expression (11), sicne the authors just call it a 'loss function', but this is the entire optimization problem.
C4. The notation using a superscript xi to represent samples is non-standard and slightly awkward. This xi has no meaning as far as I can tell, so why not use something more familiar like \hat{R}, \hat{\mathbf{n}}, or \tilde{...}?


Off-policy evaluation for slate recommendation
This paper studies the evaluation of policies that recommend an ordered set of items (e.g., a ranking) based on some context-a common scenario in web search, ads, and recommendation. We build on techniques from combinatorial bandits to introduce a new practical estimator that uses logged data to estimate a policy's performance. A thorough empirical evaluation on real-world data reveals that our estimator is accurate in a variety of settings, including as a subroutine in a learningto-rank task, where it achieves competitive performance. We derive conditions under which our estimator is unbiased-these conditions are weaker than prior heuristics for slate evaluation-and experimentally demonstrate a smaller bias than parametric approaches, even when these conditions are violated. Finally, our theory and experiments also show exponential savings in the amount of required data compared with general unbiased estimators.

This paper addresses the problem of off-policy evaluation in combinatorial contextual bandits.  A common scenario in many real world contextual bandit problems is when there is a significant amount of logged data (for example logged search queries, results and clicks) but training or evaluating a model online in real time through e.g. an A/B test is expensive or infeasible.  Ideally one would evaluate/train the model off-policy (i.e. on the logged data), but this incurs a significant bias since the data was not generated by the policy being evaluated.  The authors show in their experiments that this bias significantly impacts the quality of models that are directly trained on the logs.  They devise a method called the 'pseudo-inverse estimator' to remove the bias and provide theoretical results showing that the method is unbiased under some strong assumptions (although in experiments it appears that these assumptions can be violated).  They demonstrate empirically in carefully crafted synthetic and real-world experiments that the method outperforms baseline approaches.
This paper is very well written and, although dense, is quite clear.  The careful explanation of notation is appreciated.
In the figures in the appendices, it looks as though the direct method (DM-tree) outperforms or matches the proposed strategy on a variety of the synthetic problems (particularly on ERR vs NDCG).  The authors justify this by noting that ERR violates the assumption of linearity which seems like a reasonable justification (and it's appreciated that both metrics are included).  However, then in Section 4.3 the experiment shows PI doing slightly worse on NDCG and better on ERR.  This seems inconsistent with the linearity hypothesis.  The authors note that the baseline could be stronger for ERR, which suggests that perhaps the proposed method wouldn't outperform SUP on either baseline?
It seems curious that although wIPS is provably unbiased under weaker assumptions than IP, it performs so much more poorly in experiments.  Is this because the variance of the estimator is so high due to the cardinality of S(x)?
Overall, this is a very neat paper proposing a tidy solution with compelling theoretical and empirical results and well thought out experiments.  The empirical results, however, don't suggest to me that this method is absolutely better than the fairly naive (admittedly unsatisfying) approach of directly regressing the logs with e.g. a regression tree.
There's a typo on line 116: "in on".
The assumption on rewards on 113 seems like it could be a strong one.  This line is saying essentially that all the rewards given actions in the slate are independent of all the other actions.  In practice, this must almost certainly be violated in most cases (e.g. people's clicks will vary based on context).  Although I suppose in some cases, e.g. in search where the user is looking for a specific single result, this may be fine.
The notation in the paragraph starting on line 117 is a little confusing to me.  The text refers to vectors, but they're indexed into with pairs (given the notation in the rest of the paper, I assume that they are vectors and the pairs map to a vector index?).


Efficient Optimization for Linear Dynamical Systems with Applications to Clustering and Sparse Coding
Linear Dynamical Systems (LDSs) are fundamental tools for modeling spatiotemporal data in various disciplines. Though rich in modeling, analyzing LDSs is not free of difficulty, mainly because LDSs do not comply with Euclidean geometry and hence conventional learning techniques can not be applied directly. In this paper, we propose an efficient projected gradient descent method to minimize a general form of a loss function and demonstrate how clustering and sparse coding with LDSs can be solved by the proposed method efficiently. To this end, we first derive a novel canonical form for representing the parameters of an LDS, and then show how gradient-descent updates through the projection on the space of LDSs can be achieved dexterously. In contrast to previous studies, our solution avoids any approximation in LDS modeling or during the optimization process. Extensive experiments reveal the superior performance of the proposed method in terms of the convergence and classification accuracy over state-of-the-art techniques.
the system parameters [1], the space of LDSs is non-Euclidean. This in turn makes the direct use of traditional techniques (e.g., conventional sparse solvers) inapplicable. To get around the difficulties induced by the non-Euclidean geometry, previous studies (e.g., [11,12,13,5]) resort to various approximations, either in modeling or during optimization. For instance, the authors in [11] approximated the clustering mean by finding the closest sample under a certain embedding. As we will see in our experiments, involving approximations into the solutions exhibits inevitable limitations to the algorithmic performance.
This paper develops a gradient-based method to solve the clustering and sparse coding tasks efficiently without any approximation involved. To this end, we reformulate the optimization problems for these two different tasks and then unify them into one common problem by making use of the kernel trick. However, there exist several challenges to address this common problem efficiently. The first challenge comes from the aforementioned invariance property on the LDS parameters. To attack this challenge, we introduce a novel canonical form of the system parameters that is insensitive to the equivalent changes. The second challenge comes from the fact that the optimization problem of interest requires solving Discrete Lyapunov Equations (DLEs). At first glance, such a dependency makes backpropagating the gradients through DLEs more complicated. Interestingly, we prove that the gradients can be exactly derived by solving another DLE in the end, which makes our optimization much simpler and more efficient. Finally, as suggested by [14], the LDS parameters, i.e., the transition and measurement matrices require to be stable and orthogonal, respectively. Under our canonical representation, the stability constraint is reduced to the bound constraint. We then make use of the Cayley-transformation [15] to maintain orthogonality and perform the bound-normalization to accomplish stability. Clustering and sparse coding can be combined with high-level pooling frameworks (e.g., bag-of-systems [11] and spatial-temporal-pyramid-matching [16]) for classifying dynamic textures. Our experiments on such kind of data demonstrate that the proposed methods outperform state-of-the-art techniques in terms of the convergence and classification accuracy.

Summary: the paper proposes a clustering method for linear dynamical systems, based on minimizing the kernalized norm between extended observability spaces.  Since the objective function contains terms involving discrete Lypanunov equations (DLEs), the paper  derives how to pass the derivatives through, yielding a gradient descent algorithm.  Experiments are presented on 2 datasets, for LDS clustering and sparse codebook learning.
Originality:
+1) novelty: the paper is moderately novel, but fairly straightforward. The difference with other related works [3,9,10,11,12] in terms of objective functions, assumptions/constraints, canonical forms should be discussed more.
Quality:
There are a few potential problems in the derivations and motivation.
-1) Theorem 1: The canonical form should have an additional ineqaultiy constraint on Lambda, which is:
   lambda_1 > lambda_2 > lambda_3 > ... lambda_n
which comes from Lambda being a matrix of the singular values of A.  Imposing this constraint removes equivalent representations due to permutations of the state vector.  This inequality constraint could also be included in (12) to truly enforce the canonical form.  However, I think it is not necessary (see next point).
-2) It's unclear why we actually need the canonical form to solve (10).  Since gradient descent is used, we only care that we obtain a feasible solution, and the actual solution that we converge to will depend on the initial starting point.  I suppose that using the canonical form allows for an easier way to handle the stability constraint of A, rho(A)<1, but this needs to be discussed more, and compared with the straightforward approach of directly optimizing (A,C).
-3) Theorem 4: I'm not entirely convinced by the proof.  In the supplemental, Line 35 shows an explicit equation for the vectorized R_12.  Then why in (34) does it need to be transformed into a DLE?  There is something not quite correct about the vectorization and differentiation step. Furthermore, it seems that the gradients could be directly computed from (24). For example, taking the derivative wrt A1:
  d/dA1 (A1^T G12 A2 - G12) = d/dA1 (-C1^T C2)
  G12 A2 + A1^T (dG12/dA1) A2 - (dG12/dA1) = 0
Hence dG12/dA1 can be obtained directly as a DLE.
-4) Given that there are previous works on LDS clustering, experiments are somewhat lacking:
 - Since a kernel method is used, it would be interesting to see the results for directly using the linear kernel, or the other kernels in (19,20).
 - In the video classification experiment, there is no comparison to [3,8,11], which also perform LDS clustering.
 - No comparison with probabilistic clustering methods [21,22].

Clarity:
- L210 - Theorem 4 cannot be "directly employed" on new kernels, since the theorem is specific to the kernel in (5). Some extension, or partial re-derivation is needed.
- Other minor typos:
  - Eq 8: it should be mentioned why the k(S_m,S_m) term can be dropped.
  - Eq 9: the minimization is missing z_i.
  - Eq 10: the text is missing how (9) is solved. I'm assuming there are two sub-problems: sparse code estmiation, and codebook estimation with z_i fixed.  (10) corresponds to the second step.
Significance:
Overall, the approach is reasonable and offers a solution to the problem of clustering LDS. However, more could have been done in terms of relating the work with previous similar methods, exploring other kernel functions, and more thorough comparison in the expeirments.  Some steps in the derivation need to be clarified.


Inferring Generative Model Structure with Static Analysis
Obtaining enough labeled data to robustly train complex discriminative models is a major bottleneck in the machine learning pipeline. A popular solution is combining multiple sources of weak supervision using generative models. The structure of these models affects the quality of the training labels, but is difficult to learn without any ground truth labels. We instead rely on weak supervision sources having some structure by virtue of being encoded programmatically. We present Coral, a paradigm that infers generative model structure by statically analyzing the code for these heuristics, thus significantly reducing the amount of data required to learn structure. We prove that Coral's sample complexity scales quasilinearly with the number of heuristics and number of relations identified, improving over the standard sample complexity, which is exponential in n for learning n th degree relations. Empirically, Coral matches or outperforms traditional structure learning approaches by up to 3.81 F1 points. Using Coral to model dependencies instead of assuming independence results in better performance than a fully supervised model by 3.07 accuracy points when heuristics are used to label radiology data without ground truth labels.

The authors propose using static analysis to improve the synthesis of generative models designed to combine multiple heuristic functions. They leverage the fact that expressing heuristics as programs often encodes information about their relationship to other heuristics. In particular, this occurs when multiple heuristics depend (directly or indirectly) on the same value, such as the perimeter of a bounding box in a vision application. The proposed system, Coral, infers and explicitly encodes these relationships in a factor graph model. The authors show empirically how Coral leads to improvements over other related techniques.
I believe the idea is interesting and worth considering for acceptance. However, I have various queries, and points that might strengthen the paper:
* I found it difficult at first to see how Coral relates to other work in this area. It might be useful to position the "Related Work" section earlier in the paper. It might also help to more explicitly describe how Coral relates to [2] (Bach et al.), which seems closest to this work -- in particular, there is quite a lot of similarity between the factor graphs considered by the two models. (Coral seems to be obtained by inserting the phi_i^HF and p_i nodes between the lambda_i and phi^DSP nodes, and by restricting only to pairwise terms in the phi^DSP factor.)
* It isn't stated anywhere I could see that lambda_i \in {-1, 1}, but based on the definition of phi^Acc_i (line 147), this must be the case, right? If so, the "True" and "False" values in Figure 3 (a) should potentially be replaced with 1 and -1 for clarity.
* Why does (i) (line 156) constitute a problem for the model? Surely over-specification is OK? (I can definitely see how under-specification in (ii) is an issue, on the other hand.)
* What is the justification for using the indicator function I[p_i = p_j] for phi_ij^Sim? I would expect this hardly ever to have the value 1 for most primitives of interest. It is mentioned that, for instance, we might expect the area and perimeter of a tumour to be related, but surely we do not expect these values to be identical? (I note that similar correlation dependencies occur in [2] (Bach et al.), but these compare the heuristic functions rather than the primitives as here. Since the heuristic functions return an estimate of the label Y, it seems much more reasonable to expect their values might be equal.)
* It is stated in line 176 that the supplementary materials contains a proof that "if any additional factor would have improved the accuracy of the model, then the provided primitive dependencies would have to be incorrect". However, I couldn't find this -- where exactly can it be found?
Finally, some minor (possible) typos:
* Line 133: "if an edge point" should be "if an edge points"
* Figure 3 (a): The first "return False" has an extra indent
* The double summation in the definition of P between lines 170 and 171 seems to double count certain terms, and is slightly different from the definition of phi^DSP between lines 169 and 170
* Line 227: "structure learning structure learning"
* Line 269: "structure learning performed better than Coral when we looked only used static analysis to infer dependencies"


Training Quantized Nets: A Deeper Understanding
Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding lowprecision model for efficient inference on such systems. However, training models directly with coarsely quantized weights is a key step towards learning on embedded platforms that have limited computing resources, memory capacity, and power consumption. Numerous recent publications have studied methods for training quantized networks, but these studies have mostly been empirical. In this work, we investigate training methods for quantized neural networks from a theoretical viewpoint. We first explore accuracy guarantees for training methods under convexity assumptions. We then look at the behavior of these algorithms for non-convex problems, and show that training algorithms that exploit high-precision representations have an important greedy search phase that purely quantized training methods lack, which explains the difficulty of training using low-precision arithmetic.

Summary
---------
This paper analyzes the performance of quantized networks at a theoretical level. It proofs convergence results for two types of quantization (stochastic rounding and binary connect) in a convex setting and provides theoretical insights into the superiority of binary connect over stochastic rounding in non-convex applications.
General comments
—-----------------
The paper states that the motivation for quantization is training of neural networks on embedded devices with limited power, memory and/or support of floating-point arithmetic. I do not agree with this argumentation, because we can easily train a neural network on a powerful GPU grid and still do the inexpensive forward-pass on a mobile device. Furthermore, the vast majority of devices does support floating-point arithmetic. And finally, binary connect, which the paper finds to be superior to stochastic rounding, still needs floating-point arithmetic.
Presentation
-------------
The paper is very well written and structured. The discussion of related work is comprehensive. The notation is clean and consistent. All proofs can be found in the (extensive) supplemental material, which is nice because the reader can appreciate the main results without getting lost in too much detail.
Technical section
------------------
The paper’s theoretical contributions can be divided into convex and non-convex results: For convex problems, the paper provides ergodic convergence results for quantization through stochastic rounding (SR) and binary connect (BC). In particular, it shows that both methods behave fundamentally different for strongly convex problems. While I appreciate the importance of those results at a theoretical level, they are limited in their applicability because today’s optimization problems, and neural networks in particular, are rarely convex. For non-convex problems, the paper proofs that SR does not enter an exploitation phase for small learning rates (Theorem 5) but just gets slow (Theorem 6), because its behavior is essentially independent of the learning rate below a certain threshold. The paper claims that this is in contrast to BC, although no formal proof supporting this claim is provided for the non-convex case.
Experiments
-------------
The experimental evaluation compares the effect of different types of quantization on the performance of a computer vision task. The main purpose of the experiments is to understand the impact of the theoretical results in practice.
- First of all, the paper should clearly state the task (image classification) and quantization step (binary). Experimental results for more challenging tasks and finer quantization steps would have been interesting.
- Standard SGD with learning rates following a Robbins-Monro sequence should have been used instead of Adam. After all, the theoretical results were not derived for Adam’s adaptive learning rate, which I am concerned might conceal some of the effects of standard SGD.
- Table 1 feels a little incoherent, because different network architectures were used for different datasets.
- The test error on ImageNet is very high. Is this top-1 accuracy?
- Why only quantize convolutional weights (l.222)? The vast majority of weights are in the fully-connected layers, so it is not surprising that the quantized performance is close to the unquantized networks if most weights are the same.
- A second experiment computes the percentage of changed weights to give insights into the exploration-exploitation behaviour of SR and BC. I am not sure this is a meaningful experiment. Due to the redundancy in neural networks, changing a large numbers of weights can still result in a very similar function. Also, it seems that the curves in Fig. 3(c) do saturate, meaning that only very few weights are changed between epochs 80 and 100 and indicating exploitation even with SR.
Minor comments
-----------------
l.76: Citation missing.
Theorem 1: w^* not defined (probably global minimizer).
l.137: F_i not defined.
l.242: The percentage of changed weights is around 40%, not 50%.
Conclusion
-----------
The paper makes important contributions to the theoretical understanding of quantized nets. Although I am not convinced by the experiments and concerned about the practical relevance, this paper should be accepted due to its strong theoretical contributions.


Shape and Material from Sound
Hearing an object falling onto the ground, humans can recover rich information including its rough shape, material, and falling height. In this paper, we build machines to approximate such competency. We first mimic human knowledge of the physical world by building an efficient, physics-based simulation engine. Then, we present an analysis-by-synthesis approach to infer properties of the falling object. We further accelerate the process by learning a mapping from a sound wave to object properties, and using the predicted values to initialize the inference. This mapping can be viewed as an approximation of human commonsense learned from past experience. Our model performs well on both synthetic audio clips and real recordings without requiring any annotated data. We conduct behavior studies to compare human responses with ours on estimating object shape, material, and falling height from sound. Our model achieves near-human performance.

This paper contains a lot of ideas and methods packed into a tightly wound package - less would have been certainly more: The general aim is to build a system that can mimick the human ability to recognise shape and material objects from their sounds. Certainly a nice idea to explore within a NIPS community. To this end a generative model is defined, and a process for inference is proposed ("Physics-Based Audio Engine") that uses a physics simulation, this physics simulation is coupled (somehow) to a sound generation algorithm. This appears to use (somehow) the physics simulation engine representation of the vibrating surfaces of the objects to render the sounds. This engine is then used to create training stimuli for 4 variants of machine learning algorithms to learn appropriate representations. Again the descriptions are rather curt, so it is difficult to understand how the algorithms actually operate and how they perform. Into the paper mix we also have now human experiments that aims to compare the algorithms with that of the human observer. A description of how the experiment was conducted with human users is missing, however human users are almost always outperformed by the model. While it is up to this point unclear why the authors did not use real data when assessing their models, we are treated to real at the end (Figure 7) but without any meaningful information for us to understand how their method performs. In my opinion these are the major issues
1. Too much information therefore too superficially presented.
2. It is unclear how good the generative model is, comparison to real data would have been help here in anchoring it first.
3. The use of 4 different algorithms for classification is a nice effort, but it remains unclear to me what the point is (do the authors have a clear hypothesis?)
4. The human data is in itself interesting, but at this point it appears unclear how it was obtained, and how it informs us further on the work done in the paper.


Hybrid Reward Architecture for Reinforcement Learning
One of the main challenges in reinforcement learning (RL) is generalisation. In typical deep RL methods this is achieved by approximating the optimal value function with a low-dimensional representation using a deep network. While this approach works well in many domains, in domains where the optimal value function cannot easily be reduced to a low-dimensional representation, learning can be very slow and unstable. This paper contributes towards tackling such challenging domains, by proposing a new method, called Hybrid Reward Architecture (HRA). HRA takes as input a decomposed reward function and learns a separate value function for each component reward function. Because each component typically only depends on a subset of all features, the corresponding value function can be approximated more easily by a low-dimensional representation, enabling more effective learning. We demonstrate HRA on a toy-problem and the Atari game Ms. Pac-Man, where HRA achieves above-human performance.

R5:
Summary:
This paper builds on the basic idea of the Horde architecture: learning many value functions in parallel with off-policy reinforcement learning. This paper shows that learning many value functions in parallel improves the performance on a single main task. The novelty here lies in a particular strategy for generating many different reward functions and how to combine them to generate behavior. The results show large improvements in performance in an illustrative grid world and Miss Pac-man.
Decision:
This paper is difficult to access. In terms of positives: (1) the paper lends another piece of evidence to support the hypothesis that learning many things is key to learning in complex domains. (2) The paper is well written, highly polished with excellent figures, graphs and results. (3) The key ideas are clearly illustrated in a small domain and then scaled up to Atari. (4) The proposed approach achieves substantial improvement over several variants of one of the latest high-performance deep learning systems.
My main issue with the paper is that it is not well paced between two related ideas from the literature, namely the UNREAL architecture [1], and Diuk’s [2] object oriented approach successfully used many years ago in pitfall—a domain most Deep RL systems perform very poorly on. It’s unclear to me if these two systems should be used as baselines for evaluating hydra. However, it is clear that a discussion should be added of where the Hydra architecture fits between these two philosophical extremes.
I recommend accept, because the positives outweigh my concerns, and I am interested to here the author’s thoughts.
The two main ideas to discuss are auxiliary tasks and object-oriented RL. Auxiliary tasks are also built on the Horde idea, where the demons might be predicting or controlling pixels and features. Although pixel control works best, it is perhaps too specific to visual domains, but the feature control idea is quiet general. The nice thing about the UNREAL architecture [1] is that it was perhaps one of the first to show that learning many value functions in parallel can improve performance on some main task. The additional tasks somehow regularize the internal state resulting in a better overall representation. The behavior policy did not require weighting over the different value functions it simply shared the representation. This is a major point of departure from Hydra; what is the best design choice here? Much discussion is needed.
Another natural question is how well does pixel control work in Miss Pac-man, this was not reported in [1].
I see hydra as somewhat less general than auxiliary tasks idea. In Hydra a person must designate in the image what pixel groupings correspond to interesting value functions to learn. One could view Hydra as the outcome of UREAL pixel control combined with some automatic way to determine which pixel control GVFs are most useful for sculpting the representation for good main task behavior. In this way Hydra represents a benchmark for automatic GVF construction and curation.
On the other extreme with have Diuk’s [2] object oriented approach in pitfall. Assuming some way to extract the objects in an Pitfall screen, the objects were converted into a first-order logic representation of the game. Using this approach, optimal behavior could be achieved in one episode. This is interesting because SOTA deep-learning systems don't even report results on this domain. Hydra is somewhat like this but is perhaps more general.
Carefully and thoughtfully relating the Hydra approach to these extremes is critical for accurately discussing the precise contribution of hydra and this paper under review here, and I invite the authors to do that.

The final result of the paper (honestly admitted by the authors) exploits the determinist of Atari domain to achieve super-human play in miss packman. This result shows that A3C and others do not exploit this determinism. I am not sure how generally applicable this result is.
Another method not discussed, Model Free Episodic Control [3], performs well in Miss Pac-man—much better than A3C. Episodic control defines a kernel function and memory built from agent states generated by a deep network. This allows the system to quickly the query the values of related previously seen states, resulting in much faster initial learning in many domains.  I would be interested to see the performance of episodic control in miss pac-man with both representations.
It is interesting that DQN does not seem to exploit the additional domain knowledge in the fruit experiment. I am trying to figure out the main point here. Is the suggestion that we will usually have access to such information, and hydra will often exploit it more effectively? Or that prior information will usually be of this particular form that hydra uses so effectively?  Could unreal’s aux tasks make use of manually ignoring irrelevant features? If that is not feasible in UNREAL, this would be yet another major difference between the architectures! This raises an interesting question: how often can this feature irrelevance be automatically observed and exploited without causing major problems in other domains. It seems like UNREAL’s independent GVFs could eventually learn to ignore parts of it’s shared representation.
Low-dim representations: this is a meme that runs throughout the paper, and a claim I am not sure I agree with. Several recent papers have suggested that the capacity of deep learning systems is massive and that they can effetely memorize large data-sets. Do you have evidence to support this seemingly opposite intuition, outside of the fact that Hydra works well? This seems like a post-hoc explanation for why hydra works well. This should be discussed more in the paper
[1] Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z., Silver, D., & Kavukcuoglu, K. (2016). Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv:1611.05397.
[2] Diuk, C., Cohen, A., & Littman, M. L. (2008, July). An object-oriented representation for efficient reinforcement learning. In Proceedings of the 25th international conference on Machine learning (pp. 240-247). ACM.
[3] Blundell, C., Uria, B., Pritzel, A., Li, Y., Ruderman, A., Leibo, J. Z., ... & Hassabis, D. (2016). Model-free episodic control. arXiv preprint arXiv:1606.04460.

Small things that did not effect the final decision:
line 23: performed > > achieved
26: DQN carries out a strong generalisation > > DQN utilizes global generalization—like any neural net—
28: model of the optimal value function. I don’t think you mean that
230: how important are the details of these choices? Could it be -500 or -2000?


Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks
By lifting the ReLU function into a higher dimensional space, we develop a smooth multi-convex formulation for training feed-forward deep neural networks (DNNs). This allows us to develop a block coordinate descent (BCD) training algorithm consisting of a sequence of numerically well-behaved convex optimizations. Using ideas from proximal point methods in convex analysis, we prove that this BCD algorithm will converge globally to a stationary point with R-linear convergence rate of order one. In experiments with the MNIST database, DNNs trained with this BCD algorithm consistently yielded better test-set error rates than identical DNN architectures trained via all the stochastic gradient descent (SGD) variants in the Caffe toolbox.

This manuscript proposes a modification of feed-forward neural network optimization problem when the activation function is the ReLU function.
My major concern is about the convergence proof for the block coordinate descent algorithm. In particular, Theorem 1 is incorrect for the following reason:
A. It assumed that the sequence generated by the algorithm has a limit point.
This may not be the case as the set U is not compact (closed but not bounded). Therefore, the sequence may not converge to a finite point.
B. It stated that the sequence generated by the algorithm has a unique limit point (line 263: "the" limit point).
Even the algorithm has limit points, it may not be unique. Consider the sequence x_i = (-1)^n - 1/n, clearly it has 2 limit points: +1 and -1.
For the rest parts of the paper, my comments are as follows.
1. The new formulation seems interesting, and it can be separately discussed from the block coordinate descent part, and it looks to me the major novelty is the new formulation but not the algorithm.
However, the major concern for the new formulation is that it has much more variables, which can potentially lead to spatial unfeasibility when there are more data instances. This problem hinders people from applying the formulation to large-scale problems and thus its usefulness is limited.
2. I do not see that the problem in line 149 is always convex with respect to W. This requires further elaboration.
3. In the introduction, some efforts were put in describing the issue of saddle points. However, this manuscript does not handle this problem as well. I'd suggest the authors to remove the related discussion.
4. The format of paper reference and equation reference is rather non-standard. Usually [1] is used for paper reference and (1) is for equation reference.
5. The comparison with Caffe solvers in term of time is meaningless as the implementation platforms are totally different. I do not understand  the sentence "using MATLAB still run significantly faster" especially regarding the word "still", as matlab should be expected to be faster than python.
6. The comparison of the objective value in 3(a) is comparing apples to oranges, as the two are solving different problems.
=== After feedback ===
I did notice that in Line 100 the authors mentioned that U,V,W are compact sets, but as we see in line 124, U is set to be the nonnegative half-space, which is clearly unbounded and thus non-compact.
Therefore, indeed when U is compact, this is correct that there will be at least a limit point and thus the convergence analysis may hold true (see details below), it is not the case for the problem (4) or (5).
Regarding the proof, I don't agree with the concept of "the" limit point when t = \infty. I don't think such thing exists by nature.
As stated in my original review, there might be several limit points, and by definition we always have all limit points are associated with t = \infty, just in different ways.
The authors said in the rebuttal that because their algorithm converges so "the" limit point exists, but this is using the result proved under this assumption to prove that the assumption is true. I don't think that is a valid way of proof.
In addition, in the extremest case, consider that the sets U,V,W all consist of one point, say u,v,w, and the gradient at (u,v,w) is not zero. Clearly in this case there is one and only one limit point (u,v,w), but apparently it is not a stationary point as the gradient is non-zero.
Therefore, I will keep the same score.


InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
The goal of imitation learning is to mimic expert behavior without access to an explicit reward signal. Expert demonstrations provided by humans, however, often show significant variability due to latent factors that are typically not explicitly modeled. In this paper, we propose a new algorithm that can infer the latent structure of expert demonstrations in an unsupervised way. Our method, built on top of Generative Adversarial Imitation Learning, can not only imitate complex behaviors, but also learn interpretable and meaningful representations of complex behavioral data, including visual demonstrations. In the driving domain, we show that a model learned from human demonstrations is able to both accurately reproduce a variety of behaviors and accurately anticipate human actions using raw visual inputs. Compared with various baselines, our method can better capture the latent structure underlying expert demonstrations, often recovering semantically meaningful factors of variation in the data.

Paper Summary:
This paper focuses on using GANs for imitation learning using trajectories from
an expert. The authors extend the GAIL (Generative Adversarial Imitation
Learning) framework by including a term in the objective function to incorporate latent structure (similar to InfoGAN). The authors then proceed to show that using their framework,
which they call InfoGAIL, they are able to learn interpretable latent structure
when the expert policy has multiple modes and that in some setting this robustness allows them to outperform current methods.
Paper Overview:
The paper is generally well written. I appreciated that the authors first demon-
started how the mechanism works on a toy 2D plane example before moving
onto more complex driving simulation environment. This helped illustrate the
core concepts of allowing the learned policy to be conditioned on a latent variable
in a minimalistic setting before moving on to a more complex 3D driving simulation.
My main concern with the paper is that, in integrating many different ideas,
it was hard to distil the core contribution of the paper which, as I understood,
was the inclusion of latent variables to learn modes within an overall policy when using GAIL.
Major Comments:
1. In integrating many different ideas, it was difficult to distil the core
contribution of this paper (the inclusion of latent variables to learn modes
within a policy). As an illustration of this problem, In subsection 3.1 the authors refer
to "InfoGAIL" in its simplest form, but the supplementary material they present a more
complex algorithm which includes weight clipping, Wasserstein GAN objective as well as an
optional term for reward augmentation and this is also called "InfoGAIL". I felt the paper
could beneft from concretely defining InfoGAIL in its most simple form, and then when
adding additional enhancements later and making it clear this is InfoGail with these extra enhancements.
2. Linked to my first point, I felt the paper would benefit from an algorithm
box for the simplest form of InfoGAIL in the paper itself (not supplement-
tary). This would add to the readability of the paper and
allow a reader not too familiar with all the details of GANs to quickly
grasp the main ideas of the algorithm. I'd suggest adding this before the
section 4 on optimization. Then keeping the InfoGAIL with extensions in
supplementary material (and renaming it accordingly there).
3. The authors show how InfoGAIL works in an idealized setting where the number of modes are known.
It would be useful to see how InfoGAIL behaves when the number of modes is not known and the number of latent variables
are set incorrectly. For example, in the 2D plane environment there are 3 modes. What happens if we use InfoGail
but set the number of latent variables to 2, 4, 5, 10 etc. Can it still infer the modes of the expert policy?
Is the method robust enough to handle these cases?
Minor Comments:
1. subsection 4.1 refers to reward augmentation. But the experiment in sub-
section 5.2 don't mention if reward augmentation is used to obtain results
(reward augmentation is only mentioned for experiments in subsection
5.3). It's not clear if the authors use reward augmentation in this experiment
and if so how it is defined.
2. The heading of the paper is "Inferring The Latent Structure of Human
Decision-Making from Raw Visual Inputs". Does InfoGAIL only work
in domains where we have raw visual inputs. Also we may assume the method would work
if the expert trajectories do not come from a human but some other noisy source.
The authors may want to revise the title to make the generality of their framework more
clear.


Influence Maximization with ε-Almost Submodular Threshold Functions
Influence maximization is the problem of selecting k nodes in a social network to maximize their influence spread. The problem has been extensively studied but most works focus on the submodular influence diffusion models. In this paper, motivated by empirical evidences, we explore influence maximization in the nonsubmodular regime. In particular, we study the general threshold model in which a fraction of nodes have non-submodular threshold functions, but their threshold functions are closely upper-and lower-bounded by some submodular functions (we call them ε-almost submodular). We first show a strong hardness result: there is no 1/n γ/c approximation for influence maximization (unless P = NP) for all networks with up to n γ ε-almost submodular nodes, where γ is in (0,1) and c is a parameter depending on ε. This indicates that influence maximization is still hard to approximate even though threshold functions are close to submodular. We then provide (1 − ε) (1 − 1/e) approximation algorithms when the number of ε-almost submodular nodes is . Finally, we conduct experiments on a number of real-world datasets, and the results demonstrate that our approximation algorithms outperform other baseline algorithms.

This paper studies influence maximization in the general threshold diffusion model, specifically when node threshold functions are almost submodular (or epsilon-submodular). The key results are: 1) a proof of inapproximability for the influence maximization problem under this model when there are sufficiently many epsilon-submodular threshold nodes; 2) an efficient, simple and principled approximation algorithm for this problem when there is some fixed number of epsilon-submodular threshold nodes.
        This is a strong paper for the following reasons:
        1- Influence maximization under the General Threshold model is a challenging and poorly studied problem;
        2- The epsilon-submodularity condition is very reasonable because it is consistent with empirical evidence in real diffusion processes;
        3- The hardness result and the approximate algorithms are intuitive and non-trivial.
        Barring minor typos and the like (see below for details), the paper is very well-written. In my understanding, the technical proofs are correct. The experimental evaluation is thorough and convincing, and the results are consistent with the theoretical guarantees.
        Minor comments (by line):
        102: shouldn't the equation right-hand side have 1/(1-\epsilon) instead of (1-\epsilon)?
        237: "linear" instead of "liner"
        241: "satisfying" instead of "satisfies"
        262: "PageRank" instead of "PagrRank"
        297: "compared to 2.05" instead of "while 2.05"
        298: "spread" instead of "spreads"
        299: "The more $\epsilon$-AS nodes there are in the network, the more improvement is obtained"


Revisiting Perceptron: Efficient and Label-Optimal Learning of Halfspaces
It has been a long-standing problem to efficiently learn a halfspace using as few labels as possible in the presence of noise. In this work, we propose an efficient Perceptron-based algorithm for actively learning homogeneous halfspaces under the uniform distribution over the unit sphere. Under the bounded noise condition [49], where each label is flipped with probability at most η < 1 2 , our algorithm achieves a near-optimal label complexity ofÕ
the adversarial noise condition [6,45,42], where at most aΩ(�) fraction of labels can be flipped, our algorithm achieves a near-optimal label complexity ofÕ
. Furthermore, we show that our active learning algorithm can be converted to an efficient passive learning algorithm that has near-optimal sample complexities with respect to � and d.

This paper analyzes a variant of the Perceptron algorithm for active learing of
origin-centered halfspaces under the uniform distribution on the unit sphere. It
shows that their algorithm learns using a nearly tight number of samples in the
random independent noise of bounded rate. Previous work had exponentially worse
dependence on the noise rate. In addition, it shows that this algorithm can deal
with adversarial noise of sufficiently low rate. The latter result improves
polynomially on the sample complexity but requires a stronger condtion on the noise
rate.
The assumptions in this setting are very strong and as a result are highly unlikely to
hold in any realistic problem. At the same time there has been a significant amount
of prior theoretical work on this and related settings since it is essentially the only
setting where active learning gives an exponential improvement over passive
learning in high dimensions. This work improves on some aspects of several of
these works (although it is somewhat more restrictive in some other aspects).
Another nice feature of this work is that it relies on a simple variant of the classical
Perceptron algorithm and hence can be seen as analyzing a practically relevant
algorithm in a simplified scenario.
So overall this is solid theoretical contribution to a line of work that aims to better
understand the power of active learning (albeit in a very restricted setting). A
weakness of this work is that beyond the technical analysis it does not have much
of an algorithmic or conceptual message. So while interesting to several experts it's
unlikely to be of broader interest at NIPS.
---response to rebuttal ---
>> "This label query strategy is different from uncertainty sampling methods in practice, which always query for the point that is closest to the boundary. We believe that our novel sampling strategy sheds lights on the design of other practical active learning algorithms as well."
This is an interesting claim, but I do not see any support for this in the paper. The strategy might be just an artifact of the analysis even in your restricted setting (let alone a more practical one). So my evaluation remains positive about the technical achievement but skeptical about this work making a step toward more realistic scenarios.


Learning Populations of Parameters
Consider the following estimation problem: there are n entities, each with an unknown parameter p i ∈ [0, 1], and we observe n independent random variables, X 1 , . . . , X n , with X i ∼ Binomial(t, p i ). How accurately can one recover the "histogram" (i.e. cumulative density function) of the p i 's? While the empirical estimates would recover the histogram to earth mover distance Θ(
1 distance between the CDFs), we show that, provided n is sufficiently large, we can achieve error O( 1 t ) which is information theoretically optimal. We also extend our results to the multi-dimensional parameter case, capturing settings where each member of the population has multiple associated parameters. Beyond the theoretical results, we demonstrate that the recovery algorithm performs well in practice on a variety of datasets, providing illuminating insights into several domains, including politics, sports analytics, and variation in the gender ratio of offspring.

In this paper, authors study the problem of learning a set of Binomial parameters in Wasserstein distance and demonstrate that sorted set of probabilities can be learned at a rate of (1/t) where t is the parameter in the Binomial distribution. The result seems interesting and I had a great time reading it. However, I would like to understand the technical novelty of the paper better.
Can the authors please expand on the differences between this paper and one of the cited papers: “Instance optimal learning of discrete distributions”, by Valiant and Valiant? To be more specific:
In many of the discrete learning problems, Poisson approximation holds i.e., Binomial (t,p_i) ~ Poisson (tp_i). Hence, the proposed problem is very similar to estimating n Poisson random variables with means tp_1, tp_2, \ldots tp_n.  Similarly the problem in the VV paper boils down to estimating many Poisson random variables. Hence, aren't they almost the same problem?
Furthermore, in VV’s paper, they use empirical estimates for large Poisson random variables (mean > log n), and use a moment based approach for small Poisson random variables (mean < log n). In this paper, authors use moment based approach for small Poisson random variables (mean < t). In both the cases, the moment based approach yields an improvement of 1/poly(mean) ~1/poly(t) improvement in earthmovers / Wasserstein’s distance.
Given the above, can you please elaborate the differences between the two papers?
Few other questions to the authors:
1. In Theorem 1: What is the dependence of t in the term O_{delta, t}(1/\sqrt{n}).
2. In Theorem 1: s is not known in advance, it is a parameter to be tuned/ supplied to the algorithm. Hence, how do you obtain min_s in the right hand side of the equation?
3. Why Wasserstein’s distance?
4. Good-Turing approach (McAllester Schapire ‘00) seems to perform well for instance optimality of discrete distributions in KL distance (Orlitsky Suresh ‘15). However, VV showed that Good-Turing does not work well for total variation distance. In your problem, you are looking at Wasserstein’s distance. Can you please comment if you think Good-Turing approaches work well for this distance? Can you given example / compare these approaches at least experimentally?
Edits after the rebuttal:
Changed to Good paper, accept.
> For ANY value of t, there exist dists/sets of p_i's such that the VV and Good-Turing approaches would recover dists that are no better (and might be worse?) than the empirical distribution (i.e. they would get a rate of convergence > O(1/sqrt(t)) rather than the O(1/t) that we get).
Can you please add these examples in the paper, at least in the appendix? It would be also nice if you can can compare with these approaches experimentally.
> Suppose p_i = 0.5 for all i, t=10, and n is large. The analogous "poissonized" setting consists of taking m<--Poi(5n) draws from the uniform distribution over n items, and the expected number of times the ith item is seen is distributed as Poi(5). Poi(5) has variance 5, though Binomial(10,0.5) has variance 10/4 = 2.5, and in general, for constant p, Poi(t*p) and Binomial(t,p) have constant L1 distance and variances that differ by a factor of (1-p).
While I agree that they have a constant L1 distance, I still am not convinced that these problems are 'very different'. Do you believe that your algorithm (or a small variation of it)  would not work for the problem where the goal is to estimate n Poisson random variables? If so / no, please comment. If not, at least say something about it in the paper.
> [A "bad" construction is the following: consider a distribution of p_i's s.t. the corresponding mixture of binomials approximates a Poisson, in which case VV and Good-Turing will conclude wrongly that all the p_i's are identical.]
I do not agree with this comment too. Note that for this argument to work it is not sufficient to approximate each Poisson coordinate with a  mixture of binomials coordinate. One has to approximate a Poisson product distribution over n coordinates, with a mixture of binomial random variables with n coordinates. I believe this task gets harder as n-> infty.
> It depends on the distribution (and is stated in the theorem), but can be as bad as 2^t.
Please mention it in the paper.


Predicting User Activity Level In Point Processes With Mass Transport Equation
Point processes are powerful tools to model user activities and have a plethora of applications in social sciences. Predicting user activities based on point processes is a central problem. However, existing works are mostly problem specific, use heuristics, or simplify the stochastic nature of point processes. In this paper, we propose a framework that provides an efficient estimator of the probability mass function of point processes. In particular, we design a key reformulation of the prediction problem, and further derive a differential-difference equation to compute a conditional probability mass function. Our framework is applicable to general point processes and prediction tasks, and achieves superb predictive and efficiency performance in diverse real-world applications compared to the state of the art.

In my opinion this paper is quite good, although I do not consider myself an expert on point processes, which should be taken into account when reading this review.  I consider it in the top 50% of NIPS papers as a lower bound, admitting that I do not know enough to evaluated it further.  I have asked the committee to ensure that at least one person who can judge the proofs sufficiently be assigned to this paper and they promised me this would be the case.  I do hope if is published as I would like to try using it.  Overall I think the main issue with this paper is that it would more comfortably fit in about 20 pages.
Here are my detailed comments within the scope of my expertise:
Coming from a background of modeling user activity as someone who interprets time series regarding heart rate and accelerators, this paper was not what I expected based on the title.  To some extent I think the paper would have been better titled "HYBRID: a framework that provides an unbiased estimator of the probability mass function of point processes," but I do see why you called it this after reading the paper.
There are many contributions to in this paper that comprise the framework: the reformulation of the prediction problem using a conditional expectation that incorporates history, allowing sample efficiency for prediction due to a lower variance in the new estimator, the derivation of the differential-difference equation to compute the conditional estimator and both a theoretical evaluation of the framework on synthetic data as well as two evaluations on real world data.  Overall, the paper seems quite good and well supported.  It is also well written.
Prior Art:  My one comment on the prior art is that one cluster of author (the authors of 5,7,8,9,11,12,24,25,26,27 and 31) does seem to dominate the references.  These citations are necessary as the authors of this document do need to reference this material, however, for the datasets used it would be more appropriate to cite the original authors who contributed the data rather than simply 12.
Demetris Antoniades and Constantine Dovrolis. Co-evolutionary dynamics in social networks: A case study of twitter. Computational Social Networks, 2(1):14, 2015.
My guess is that since there is so much material here and 12 does need to be cited as well the authors were trying to save space, this would be a more appropriate citation for the dataset.  Similarly, I believe this paper:

Yehia Elkhatib ;  Mu Mu ;  Nicholas Race "Dataset on usage of a live & VoD P2P IPTV service" P2P, London, UK 2014
is a more appropriate citation for one of the datasets data used in Section 6.1.2 cited as "the datasets used in [24] The other dataset "used in [24]" seems to be also the Antoniades and Dovrolis set.  It would be better to reference these directly.  Also, it would be great to know if this area of research is really restricted primarily to this one group or if other groups are actively pursuing these topics.  Again, I feel it is a likely space limitation issue so perhaps this is more a comment for a journal version of the paper.
Solution Overview (Section 3)
With respect to the claim to generalizability, could the authors please give an example of a point process that is not covered by prior art (e.g not represented by a reinforced Poisson process or a Hawkes Process) as a person not overly familiar with point processes, this would help me understand better the impact of this work.  This was done nicely in lines 31 and 31 for why the generalization of the function is important.
In general I think this is well presented, your contribution is a more general framework that can preserves the stochasticity of the function and yet requires fewer samples than MC methods.
With respect to the claim of a new random variable, g(H_t_minus), the idea of a conditional intensity function, conditioned on the history of the intensity function is not new, it seems to be a known concept based on a few quick web searches.  I have to assume that you mean that using it as a random variable for prediction is a new idea.
The closest similar work I found to this is the use of a "mark-specific conditional intensity function" in "Marked Temporal Dynamics Modeling based on Recurrent Neural Network" by
Yongqing Wang, Shenghua Liu, Huawei Shen, Xueqi Cheng on arXiv: (https://arxiv.org/pdf/1701.03918.pdf).  Again unfortunately by the method of google searching as I am not expert in this area.  If you think it is similar you can reference it, if not you can ignore this.  Again, it seems related from my point of view, but my expertise is lacking in this area.
With respect to the proof contained in Appendix C:  It looks good to me.  I had to look up taking the expectation of a conditional random variable, http://www.baskent.edu.tr/~mudogan/eem611/ConditionalExpectation.pdf and the conditions under which a variance could be negative and I believe that you proved your point well.  It can't negative or zero because it right progressing and there is an impulse at t(0) which makes it non-constant and therefore non-zero if I understand you correctly.  I am hoping supplemental material will be included in the proceedings.
With respect to the contribution of the formulation of the novel transport equation, this does seem novel and useful.  I could find nothing similar. The closest match I found was:
"Probability approximation of point processes with Papangelou conditional intensity," by Giovanni Luca Torris  forthcoming in the Bernoulli sociaty (http://www.bernoulli-society.org/index.php/publications/bernoulli-journal/bernoulli-journal-papers) again based on a web search and not my knowledge, so take it as a paper of potential interest for you not a challenge to your novelty.
With respect to the mass transport equation:
This does seem like a very good idea. Intrinsically i am understanding that as intensity process generates events these contribute to that mass and that the accumulated mass is modeled as decreasing over time on absence of new events.  I understand that the authors used tools from numerical methods for approximating the integration of the probability mass function and in the end solved their equation with ode45 from Matlab.  Everything Matlab says about the method is consistent with what is presented here.
"ODE23 and ODE45 are functions for the numerical solution of ordinary differential equations. They employ variable step size Runge-Kutta integration methods. ODE23 uses a simple 2nd and 3rd order pair of formulas for medium accuracy and ODE45 uses a 4th and 5th order pair for higher accuracy."

I cannot comment with much expertise on the proof in Appendix A.  I get the general idea and it seems plausible.  I looked up the fundamental lemma of the calculus of variations and tried to follow the math, but I cannot guarantee that I did not miss something as I am not vary familiar with numerical methods.
I found Figure 2 very helpful for getting an intuitive understanding for what was being described by the equations.
I found the section on event information fairly compelling with respect to understanding why they are able to reduce the samples used for the estimate (increased sample efficiency).
For the experiments int Figure 3.  Did you train on 70% and test on 30% or did you use a train,test and hold-out set (e.g. 70/15/15)?
Overall this looks like quite a promising and solid idea and framework, so I would recommend inclusion at NIPS based on my understanding of the content.

Writing:  here are some type-os I noticed as I was reading the paper.
10: "state-of-arts" -> "the state of the art" is better
36 "adversely influence" -> "adversely influences"
64: "the phenomena of interests" -> "the phenomena of interest"
75: The root mean square error (RMSE) is defines as -> is defined as


Neural Expectation Maximization
Many real world tasks such as reasoning and physical interaction require identification and manipulation of conceptual entities. A first step towards solving these tasks is the automated discovery of distributed symbol-like representations. In this paper, we explicitly formalize this problem as inference in a spatial mixture model where each component is parametrized by a neural network. Based on the Expectation Maximization framework we then derive a differentiable clustering method that simultaneously learns how to group and represent individual entities. We evaluate our method on the (sequential) perceptual grouping task and find that it is able to accurately recover the constituent objects. We demonstrate that the learned representations are useful for next-step prediction.

The paper presents two related deep neural networks for clustering entities together into roughly independant components.  In their implementations the entities are always pixels in an image.  The first technique they brand as neural expectation maximization, and show how the network is performing an unrolled form of generalized EM.  The E-step predicts the probability that a given pixel was generated by each of the components.  The M-step, then updates the parameters of each component using these probabilities.  This is neural, because each of these steps relies on a neural network which maps from the parameters of a component to the pixel values. By unrolling this process they can train this network in an end-to-end basis.  They also proposed a related model which modifies this process by updating the parameters of the components at each step in the unrolling using an RNN-style update.  This allows them to extend the model to sequential/video data where the image is changing at each step.
	The show results only on toy datasets consisting of a small number of shapes or MNIST images (3-5) superimposed on top of each other.  Their neural EM approach performs significantly worse than recent work on Tagger for static data, however the RNN version of their model slightly out performs Tagger.  However Tagger significantly outperforms even the RNN version of their work if Tagger is trained with batch normalization.  In the video setting, their model performs significantly better, which is not surprising given the additional information available in video.  They show both higher scores for predicting the pixel to component mapping, but also much better reconstruction MSE than a standard  (fully entangled) autoencoder model (which is the same a their model with 1 component).
	Overall these seems like a solid paper with solid results on a toy dataset.
	My main concern is that the technique may not work on messier real world data, since they only show results on relatively toy datasets where the components are completely independant, and the component boundaries are relatively clear when they are not overlapping.  Results in a more realistic domain would make the paper much strong, but even in the current state I believe both the technique and the results are interesting.


Asynchronous Coordinate Descent under More Realistic Assumption *
Asynchronous-parallel algorithms have the potential to vastly speed up algorithms by eliminating costly synchronization. However, our understanding of these algorithms is limited because the current convergence theory of asynchronous block coordinate descent algorithms is based on somewhat unrealistic assumptions. In particular, the age of the shared optimization variables being used to update blocks is assumed to be independent of the block being updated. Additionally, it is assumed that the updates are applied to randomly chosen blocks. In this paper, we argue that these assumptions either fail to hold or will imply less efficient implementations. We then prove the convergence of asynchronous-parallel block coordinate descent under more realistic assumptions, in particular, always without the independence assumption. The analysis permits both the deterministic (essentially) cyclic and random rules for block choices. Because a bound on the asynchronous delays may or may not be available, we establish convergence for both bounded delays and unbounded delays. The analysis also covers nonconvex, weakly convex, and strongly convex functions. The convergence theory involves a Lyapunov function that directly incorporates both objective progress and delays. A continuous-time ODE is provided to motivate the construction at a high level.

The paper studies coordinate descent with asynchronous updates. The key contributions include a relaxation of the assumptions on how the block coordinates are chosen and how large could the delays be. Descent lemma and convergence rates are obtained for a variety of settings using the ode analysis and discretization.
I think the paper is well-grounded because existing assumptions in the asynchronous analyses of coordinate descents are highly unsatisfactory; and this work explained the problem well and proposed a somewhat more satisfactory solution. For this reason I vote for accept.
Detailed evaluation:
1. I did not have time to check the proof, but I think the results are believable based on the steps that the authors outlined in the main paper.
2. The conditions for cyclic choice of coordinates and unbounded delays could be used in other settings.
3. In a related problem of asynchronous block coordinate frank wolfe, it was established under weaker tail bound assumptions of the delay random variable that there could be unbounded stochastic delay. The delay random variable also are not required to be independent. Establishing deterministic sequence in AP-BCFW or more heavy-tailed stochastic delay for AP-BCD could both be useful.
- Wang, Y. X., Sadhanala, V., Dai, W., Neiswanger, W., Sra, S., & Xing, E. (2016, June). Parallel and distributed block-coordinate Frank-Wolfe algorithms. In International Conference on Machine Learning (pp. 1548-1557).


Self-Normalizing Neural Networks
Deep Learning has revolutionized vision via convolutional neural networks (CNNs) and natural language processing via recurrent neural networks (RNNs). However, success stories of Deep Learning with standard feed-forward neural networks (FNNs) are rare. FNNs that perform well are typically shallow and, therefore cannot exploit many levels of abstract representations. We introduce self-normalizing neural networks (SNNs) to enable high-level abstract representations. While batch normalization requires explicit normalization, neuron activations of SNNs automatically converge towards zero mean and unit variance. The activation function of SNNs are "scaled exponential linear units" (SELUs), which induce self-normalizing properties. Using the Banach fixed-point theorem, we prove that activations close to zero mean and unit variance that are propagated through many network layers will converge towards zero mean and unit variance -even under the presence of noise and perturbations. This convergence property of SNNs allows to (1) train deep networks with many layers, (2) employ strong regularization schemes, and (3) to make learning highly robust. Furthermore, for activations not close to unit variance, we prove an upper and lower bound on the variance, thus, vanishing and exploding gradients are impossible. We compared SNNs on (a) 121 tasks from the UCI machine learning repository, on (b) drug discovery benchmarks, and on (c) astronomy tasks with standard FNNs, and other machine learning methods such as random forests and support vector machines. For FNNs we considered (i) ReLU networks without normalization, (ii) batch normalization, (iii) layer normalization, (iv) weight normalization, (v) highway networks, and (vi) residual networks. SNNs significantly outperformed all competing FNN methods at 121 UCI tasks, outperformed all competing methods at the Tox21 dataset, and set a new record at an astronomy data set. The winning SNN architectures are often very deep.

The paper proposes a new (class of) activation function f to more efficiently train very deep feed-forward neural networks (networks with f are called SNNs). f is a combination of scaled ReLU (positive x) and an exponential (negative x). The authors argue that 1) SNNs converge towards normalized activation distributions 2) SGD is more stable as the SNN approx preserves variance from layer to layer. In fact, f is part of a family of activation functions, for which theoretical guarantees for fixed-point convergence exist. These functions are contraction mappings and characterized by mean/variance preservation across layers at the fixed point -- solving these constraints allows finding other "self-normalizing" f, in principle.
        Whether f converges to a fixed point, is sensitive to the choice of hyper-parameters: the authors demonstrate certain weight initializations and parameters settings that give the fixed-point behavior. Also, authors analyze the performance of SNNs with a modified version of dropout, which preserves mean/variance while using f.
        In support, authors provide theoretical proofs and experimental results on many datasets, although these vary in size (and quality?).
        As such, I think the paper warrants an accept -- although I have questions / concerns that I hope the authors can address.
        Questions / concerns:
        1. Most exps are run on small datasets. This is understandable since only FNNs are considered, but do we get similar results on networks that also include e.g. convolutional / recurrent layers? How can the theoretical analysis work / apply in that case (e.g. convolutions / pooling / memory cells are not always contraction mappings)?
        2. Ad point 1. What are the results for SNNs on MNIST / Cifar-10?
        3. The authors only considered layers without bias units. What happens when these are included? (This likely make it harder to preserve domains, for instance)
        4. During training, the mean/var of the weights do change, and are not guaranteed to be in a basin of attraction (see Thm 1) anymore. Why is there still convergence to a fixed-point, even though no e.g. explicit weight clipping / normalization is applied (to make Thm 1 apply)?
        5. How does this analysis interact with different optimizers? What is the difference between using momentum-based / Adagrad etc for SNNs?
        6. Hyper-parameters of f need to be set precisely, otherwise f is not domain-preserving / contraction mapping anymore. How sensitive is the model to this choice?
        7. It is unclear how the fixed-point analysis theoretically relates to the generalization performance of the found solution. Is there a theoretical connection?


Limitations on Variance-Reduction and Acceleration Schemes for Finite Sum Optimization
We study the conditions under which one is able to efficiently apply variancereduction and acceleration schemes on finite sum optimization problems. First, we show that, perhaps surprisingly, the finite sum structure by itself, is not sufficient for obtaining a complexity bound ofÕ((n + L/µ) ln(1/ )) for L-smooth and µ-strongly convex individual functions -one must also know which individual function is being referred to by the oracle at each iteration. Next, we show that for a broad class of first-order and coordinate-descent finite sum algorithms (including, e.g., SDCA, SVRG, SAG), it is not possible to get an 'accelerated' complexity bound ofÕ((n+ nL/µ) ln(1/ )), unless the strong convexity parameter is given explicitly. Lastly, we show that when this class of algorithms is used for minimizing L-smooth and convex finite sums, the iteration complexity is bounded from below by Ω(n + L/ ), assuming that (on average) the same update rule is used in any iteration, and Ω(n + nL/ ) otherwise.

This papers covers some theoretical limitations on variance-reduced stochastic algorithms
for finite sum problems (SAG, SDCA, SAGA, SVRG etc.). Such algorithms have had some important
impact in machine learning & convex optimization as they have been shown to reach a linear convergence
for strongly convex problems with lipschitz gradient (log term) while having a factor in $n + \kappa$ (and not as $n\kappa$).
A first result of this paper is that the sequential iterative algorithm needs to know
which individual function has been returned by the oracle to reach the linear convergence
rate. Typically SAG or SAGA need this knowledge to update the gradient memory buffer
so it is not terribly surprising that knowing the function index is necessary.
It is then argued that such variance-reduced methods cannot be used with data augmentation
during learning. This is only true if the data augmentation is stochastic and if there is
not a finite set of augmented samples.
A second result concerns the possibility to accelerate in the "Nesterov sense" such
variance-reduced algorithm (go from $\kappa$ to $\sqrt{\kappa}}$). It is shown that
"oblivious" algorithms whose update rules are (on average) fixed for any iteration
cannot be accelerated.
A practical consequence of the first result in 3.2 is that knowing the value of the
strong convexity parameter is necessary to obtain an accelerated convergence
when the algorithm is stationary.
Such oblivious algorithms are further analysed using
restarts which are well known techniques to alleviate the non-adaptive nature of
accelerated algorithms in the presence of unknown local conditionning.
The paper is overhaul well written with some pedagogy.
Minor:
CLI is first defined in Def 2 but is used before.
Typo: L224 First, let us we describe -> First, let us describe


Continuous DR-submodular Maximization: Structure and Algorithms
DR-submodular continuous functions are important objectives with wide real-world applications spanning MAP inference in determinantal point processes (DPPs), and mean-field inference for probabilistic submodular models, amongst others. DR-submodularity captures a subclass of non-convex functions that enables both exact minimization and approximate maximization in polynomial time. In this work we study the problem of maximizing non-monotone continuous DRsubmodular functions under general down-closed convex constraints. We start by investigating geometric properties that underlie such objectives, e.g., a strong relation between (approximately) stationary points and global optimum is proved. These properties are then used to devise two optimization algorithms with provable guarantees. Concretely, we first devise a "two-phase" algorithm with 1/4 approximation guarantee. This algorithm allows the use of existing methods for finding (approximately) stationary points as a subroutine, thus, harnessing recent progress in non-convex optimization. Then we present a non-monotone FRANK-WOLFE variant with 1/e approximation guarantee and sublinear convergence rate. Finally, we extend our approach to a broader class of generalized DR-submodular continuous functions, which captures a wider spectrum of applications. Our theoretical findings are validated on synthetic and real-world problem instances.

This paper presents a new approach for non-monotone DR sub modular optimisation. The strength of this paper over existing approaches is proposing a solution to a more general problem setting than having an approach subject to constraints such as the box constraints. The proofs are well constructed and presented. However the experimental results are somehow very limited. The parameters used for the simulated data are very limited and not sure if the authors have performed a much more simulations with different parameters to assess more in depth the "performance" of their approach over the existing ones. Also, for the real data set example, it is not very clear if that the proposed two-phase Frank-Wolfe algorithm performs much better than the others. From the figures, it seems performing just slightly better but not very convincing. For real-world problems, it is typically difficult to verify whether the assumptions hold or not, so a comparison with the DoubleGreedy algorithm would have been appropriate to see how much gain we have by considering a more general constraint.


Adversarial Symmetric Variational Autoencoder
A new form of variational autoencoder (VAE) is developed, in which the joint distribution of data and codes is considered in two (symmetric) forms: (i) from observed data fed through the encoder to yield codes, and (ii) from latent codes drawn from a simple prior and propagated through the decoder to manifest data. Lower bounds are learned for marginal log-likelihood fits observed data and latent codes. When learning with the variational bound, one seeks to minimize the symmetric Kullback-Leibler divergence of joint density functions from (i) and (ii), while simultaneously seeking to maximize the two marginal log-likelihoods. To facilitate learning, a new form of adversarial training is developed. An extensive set of experiments is performed, in which we demonstrate state-of-the-art data reconstruction and generation on several image benchmark datasets.

# Summary
Motivated by the failure of maximum-likelihood training objectives to produce models that create good samples, this paper introduces a new formulation of a generative model. This model is based on a symmetric form of the VAE training objective. The goal of this symmetric formulation is to learn a distribution that both assigns high probability to points in the data distribution and low probability to points not in the data distribution (unlike a maximum-likelihood objective which focuses on learning distributions that assign high probability for points in the data distribution). As the resulting model is not tractable, the authors propose to estimate the arising KL-divergences using adversarial training.
# Clarity
This paper is mostly well-written and easy to understand.
# Significance
This paper describes an interesting approach to generative models, that attempts to fix some problems of methods based on maximum-likelihood learning. The proposed method is interesting in that it combines advantages from ALI/BiGans (good samples) with properties of VAEs (faithful reconstructions). However, by using auxiliary discriminators, the approach is only a rough approximation to the true objective, thus making it difficult to interpret the log-likelihoods estimates (see next section). The authors therefore should have evaluated the reliability of their log-likelihood estimates, e.g. by using annealed importance sampling.
# Correctness
=== UPDATE: this concern was succesfully addressed by the authors' feedback ===
While the derivation of the method seems correct, the evaluation seems to be flawed. First of all, the authors evaluate their method with respect to the log-likelihood estimates produced by their method which can be very inaccurate. In fact, the discriminators have to compare the distributions q(x,z) to q(x)q(z) and p(x,z) to p(x)p(z) respectively which are usually very different. This is precisely the reason why adaptive contrast was introduced in the AVB-paper by Mescheder al. In this context, the authors' statement that "AVB does not optimize the original variational lower bound connected to its theory" (ll. 220-221) when using adaptive contrast is misleading, as adaptive contrast is a technique for making the estimate of the variational lower bound more accurate, not an entirely different training objective. It is not clear why the proposed method should produce good estimates of the KL-divergences without such a technique.
For a more careful evaluation, the author's could have run annealed importance sampling (Neal, 1998) to evaluate the correctness of their log-likelihood estimates.
At the very least, the authors should have made clear that the obtained log-likelihood values are only approximations and have to be taken with a grain of salt.
It is also unclear why the author did not use the ELBO from equation (1) (which has an analytic expression as long as p(z), q(z | x) and p(x | z) are tractable) instead of L_VAEx which has to be approximated using an auxiliary discriminator network.
If the authors indeed used L_VAEx only for training and used the ELBO from (1) (without a discriminator) for evaluation, this should be clarified in the paper.
# Overall rating
While the technique introduced in this paper is interesting, the evaluation seems to be flawed. At the very least, the authors should have stated that their method produces only approximate log-likelihood estimates which can therefore not be compared directly to other methods.
# Response to author's feedback
Thank you for your feedback. The authors addressed my major concern about the evaluation. I therefore raised the final rating to "6 - Marginally above acceptance threshold". Note, however, that AR #2 still has major concerns about the correctness of the experimental evaluation that should be addressed by the authors in the final manuscript. In particular, the authors should show the split of the ELBO for cifar-10 into reconstruction and KL-divergence to better understand why the method can achieve good results even though the reconstructions are apparently not that good. The authors should also make sure that they performed the MNIST-experiment according to the standard protocol (binarization etc.).