Position-based Multiple-play Bandit Problem with Unknown Position Bias
Motivated by online advertising, we study a multiple-play multi-armed bandit problem with position bias that involves several slots and the latter slots yield fewer rewards. We characterize the hardness of the problem by deriving an asymptotic regret bound. We propose the Permutation Minimum Empirical Divergence (PMED) algorithm and derive its asymptotically optimal regret bound. Because of the uncertainty of the position bias, the optimal algorithm for such a problem requires non-convex optimizations that are different from usual partial monitoring and semi-bandit problems. We propose a cutting-plane method and related bi-convex relaxation for these optimizations by using auxiliary variables.

#Summary
This submission studies a multi-armed bandit problem, PBMU.
In this problem, there are L slots (to place ads) and K possible ads.
At each round, the player need to choose L from the K possible ads and place them into the L slots.
The reward of placing the i-th ad into the l-th slot is i.i.d. drawn from a Bernoulli distribution Ber(\theta_i \kappa_l), where \theta_i is the (unknown) quality of the i-th ad and \kappa_l is the (also unknown) position bias of the l-th slot.
The authors derived a regret lower bound and proposed two algorithms for PBMU.
The second algorithm has a matching regret bound.
The authors also discussed how to solve the related optimization problems appeared in the algorithms.
The authors give a complete characterization of the hardness of PMMAB in this paper. Their main contributions are:
1.A regret lower bound， which states that the expected regret of a strongly consistent algorithm for PMMAB is lower bounded by ClogT-o(log T). Where C is a parameter determined by every arm's mean profile and slot's position bias.
2.An algorithm for PMMAB. The authors called it Permutation Minimum Empirical Divergence(PMED) which can be compared with the Deterministic Minimum Empirical Divergence (DMED) algorithm for SMAB (see Honda and Takemura COLT2010).  They give an asymptotically upper bound of a modification of PMED  which is Clog T + o(log T).
From the technical point, I think the regret lower bound in this paper is a standard, which follows some techniques from Honda and Takemura COLT2010. In their upper bound analysis, the authors solve some new difficulties. In PMED algorithm, they compute Maximum Likelihood Estimator and use it to compute a K times K matrix which optimizes a corresponding optimization problem in the lower bound analysis. The authors then convert the matrix into a convex combination of permutation matrix (due to  Birkhoff–von Neumann Theorem). Though this idea is natural, the analysis is not straightforward. The authors analyze a modification of PMED in which they add additional terms to modify the optimization problems. They obtain a corresponding upper bound for this modified algorithm.
Experimental results are attached.
#Treatment of related work
In the introduction part, the authors introduced some other related models and compared them with their new model.
They also gave convincing reasons why this new model deserves study.
To the best of my knowledge, their model is novel.
However, the author did not make much effort in comparing their techniques with previous (closely related) results.
This made it difficult to evaluate the technical strength of their results.
#Presentation
The presentation of the paper still needs some work.
In Section 3, the authors defined a set \mathcal{Q}.
However, it is dubious why they need to define this quality when deriving the regret lower bound (the definition of this quality is not furthered used in Section 3 and related parts in the appendix).
On the other hand, the authors did use several properties of this quantity in Section 4 (to run Algorithm 2).
Thus, I advise the authors to place the definition of \mathcal{Q} into Section 4 in order to eliminate unnecessary confusion and change the claim of the lower bound correspondingly.
Line 16-20 of Algorithm 1 (PMED and PMED-Hinge) is difficult to follow.
To my understand, \hat{N}_{i, l} is the number of necessary drawings for the pair (i, l).
The algorithm then decomposes the matrix \hat{N} into a linear combination of permutation matrices and pull the arms according to the permutation matrices and the coefficients.
When will c_v^{aff} be smaller than c_v^{req}?
Shouldn't we pull (v_1, v_2, \ldots, v_L) for c_v^{req} times (instead of once)?
Should c_v^{req} be rounded up to an integer?
There are some other typos/minor issues. See the list below.
1. Line 57-58: Non-convex optimization appears in your algorithm does not necessarily mean that PBMU intrinsically requires non-convex optimization.
2. Line 63-65: Define K, or remove some details.
3. Line 86-87: at round t -> of the first t rounds.
4. Line 102-105: According to the definition, (1), (2), \ldots, (L) is a permutation of {1, 2, 3, \ldots, L}, which is not what you want.
5. Line 115-117: The fact that d_{KL} is non-convex does not necessarily make PBMU intrinsically difficult.
6. Line 118, the definition of \mathcal{Q}: What do you mean by \sum (q_{i, l} = q_{i + 1, l})?
7. Line 122: (1) is not am inequality. Do you want to refer to the Inequality in Lemma 1? "the drawing each pair (i, l)", remove "the".
8. Line 124-127: Please elaborate on the reason why the constraint \theta_i?\kappa_i?= \theta_i \kappa_i is necessary. The argument given here is not easy to follow.
9. Line 138: is based on -> is closed related to
10. Line 141: what do you mean by "clarify"?
11. Algorithm 1, Line 12: Explicitly define \hat{1}(t), \ldots, \hat{L}(t). Currently it is only implicitly defined in Line 160.
12. Algorithm 1, Line 14: "where e_v for each v is a permutation matrix" -> "where for each v, e_v is a permutation matrix".
13. Line 172-174: Explicitly define the naive greedy algorithm.
14. Line 343: native->naive.
15. Algorithm 1, Line 17: \max_{c \ge 0} ... \ge 0 -> \max_{c \ge 0} s.t. ... \ge 0.
-------------------
The rebuttal addressed most of my major questions.
Score changed to 7.


Integration Methods and Optimization Algorithms
We show that accelerated optimization methods can be seen as particular instances of multi-step integration schemes from numerical analysis, applied to the gradient flow equation. Compared with recent advances in this vein, the differential equation considered here is the basic gradient flow, and we derive a class of multi-step schemes which includes accelerated algorithms, using classical conditions from numerical analysis. Multi-step schemes integrate the differential equation using larger step sizes, which intuitively explains the acceleration phenomenon.

The paper provides an interpretation of a number of accelerated gradient algorithms (and other convex optimization algorithms) based on the theory of numerical integration and numerical solution of ODEs. In particular, the paper focuses on the well-studied class of multi-step methods to interpret Nesterov's method and Polyak's heavy ball method as discretizations of the natural gradient flow dynamics. The authors argue that this interpretation is more beneficial than existing interpretations based on different dynamics (e.g. Krichene et al) because of the simplicity of the underlying ODE. Notice that the novelty here lies in the focus on multistep methods, as it was already well-known that accelerated algorithms can be obtained by appropriately applying Euler's discretization to certain dynamics (again, see Krichene et al).
A large part of the paper is devoted to introducing important properties of numerical discretization schemes for ODEs, including consistency and zero-stability, and their instantiation in the case of multistep methods. Given this background, the authors proceed to show that for LINEAR ODES, i.e., for gradient flows deriving from the optimization of quadratic functions, natural choices of parameters for two-step methods yield Nesterov and Polyak's method. The interpretation of Nesterov's method is carried out both in the smooth and strongly convex case and in the smooth-only case.
I found the paper to be an interesting read, as I already believed that the study of different numerical discretization methods is an important research direction to pursue in the design of more efficient optimization algorithms for specific problems. However, I need to make three critical points, two related to the paper in question and one more generally related to the idea of using multistep methods to define faster algorithms:
1) The interpretation proposed by the authors only works in explaining Nesterov's method for functions that are smooth in the l_2-norm and is not directly applicable for different norms. Indeed, it seems to me that different dynamics besides the gradient flow must be required in this case, as the method depends on a suitable choice of prox operator. The authors do not acknowledge this important limitation and make sweeping generalizations that need to be qualified, such as claiming that convergence rates of optimization algorithms are controlled by our ability to discretize the gradient flow equation. I believe that this incorrect when working with norms different from l_2.
2) The interpretation only applies to quadratic functions, which makes it unclear to me whether these techniques can be effectively deployed to produce new algorithms. For comparison, other interpretations of accelerated methods, such as Allen-Zhu and Orecchia's work led to improved approximation algorithms for certain classes of linear programs.
3) Regarding the general direction of research, an important caveat in considering multistep methods is that it is not completely clear how such techniques would help, even for restricted class of problems, for orders larger than 2 (i.e. larger than those that correspond to Nesterov 's AGD). Indeed, it is not hard to argue that both Nesterov's AGD (and also Nemirovski's Mirror Prox) can leverage smoothness to fully cancel the discretization error accrued in converting the continuous dynamics to a discrete-time algorithm. It is then not clear how a further reduction in the discretization error, if not properly related to problem parameters, could yield faster convergence.
In conclusion, I liked the intepretation and I think it is worth disseminating it. However, I am not sure that it i significant enough in its explanatory power, novel enough in the techniques introduced (multistep methods are a classical tool in numerical systems) or promising enough to warrant acceptance in NIPS.


Acceleration and Averaging In Stochastic Descent Dynamics
We formulate and study a general family of (continuous-time) stochastic dynamics for accelerated first-order minimization of smooth convex functions. Building on an averaging formulation of accelerated mirror descent, we propose a stochastic variant in which the gradient is contaminated by noise, and study the resulting stochastic differential equation. We prove a bound on the rate of change of an energy function associated with the problem, then use it to derive estimates of convergence rates of the function values (almost surely and in expectation), both for persistent and asymptotically vanishing noise. We discuss the interaction between the parameters of the dynamics (learning rate and averaging rates) and the covariation of the noise process. In particular, we show how the asymptotic rate of covariation affects the choice of parameters and, ultimately, the convergence rate.

Summary:
The authors consider mirror descent dynamics in continuous time for constrained optimization problems. They work in the context of stochastic differential equations and consider parametrized dynamics which include many of the recently considered parameters for such dynamics. This includes time dependant learning rate, sensitivity and averaging parameters which provide a great flexibility in the analysis. The dynamics also feature acceleration terms which were recently proposed in a deterministic setting.
In a first step, the authors carry out an asymptotic analysis in the deterministic setting. They then introduce a stochastic variant of the dynamics and describe uniqueness properties of its solutions. Combining properties of the deterministic dynamics and stochastic calculus the authors propose three results: Almost sure convergence of the sample trajectories, convergence rates, both in expectation and for sample trajectories.
Main comments:
First of all, I have a limited knowledge of stochastic differential calculus and do not feel competent to validate all the technical details in the manuscript, although none of them felt particularly awkward.
I believe that this work is of very good quality. Technical elements are clearly presented and the authors do not neglect fundamental aspects of the problem such as existence and uniqueness. Furthermore, the overall exposition is clear and of good quality and the text is well balanced between the main proof arguments and further details given in the appendix.
To my understanding, the convergence analysis carried out by the authors extends significantly existing results from the litterature. The model is flexible enough to allow many possible choice of the dynamics. For example the authors are able to show that unit learning rate may robustly maintain good properties of accelerated dynamics in the deterministic case provided that suitable averaging is performed.
Overall, I think that this constitutes valuble work and should be considered for publication.
Additional comments and questions:
1- The authors motivate the introduction of stochastic dynamics by emphasizing on stochastic methods for finite sums. However the additive noise structure is not particularly well suited to model such cases. Indeed, in these settings the variance of the noise usually features a multiplicative term.
2- All the results are presented in abstract form with the addition of an other parameter "r" and assumptions on it. It is not really clear how the assumptions on r restrict the choice of the other parameters. As a result, the convergence rates are not easily read or interpreted. Beyon the proposed corollaries, it is not really clear how the given result provide interesting guaranties in terms of function values. For example would the work of the authors allow to get better rates in the small noise regime?
3- Do the authors have any comment on how the analysis which is carried out in a stochastic, continuous time setting could impact the discrete time setting. Is there a clear path to the conception of efficient algorithms and in particular is it possible to provide a discrete time equivalent for the different time dependent parameters in the ODE?


Tomography of the London Underground: a Scalable Model for Origin-Destination Data
The paper addresses the classical network tomography problem of inferring local traffic given origin-destination observations. Focusing on large complex public transportation systems, we build a scalable model that exploits input-output information to estimate the unobserved link/station loads and the users' path preferences. Based on the reconstruction of the users' travel time distribution, the model is flexible enough to capture possible different path-choice strategies and correlations between users travelling on similar paths at similar times. The corresponding likelihood function is intractable for medium or large-scale networks and we propose two distinct strategies, namely the exact maximum-likelihood inference of an approximate but tractable model and the variational inference of the original intractable model. As an application of our approach, we consider the emblematic case of the London underground network, where a tap-in/tap-out system tracks the starting/exit time and location of all journeys in a day. A set of synthetic simulations and real data provided by Transport For London are used to validate and test the model on the predictions of observable and unobservable quantities.

I thank the authors for the clarification in their rebuttal. It is even more clear that the authors should better contrast their work with aggregate approaches such as Dan Sheldon's collective graphical models (e.g., Sheldon and Dietterich (2011), Kumar et al. 2013, Bernstein and Sheldon 2016).
Part of the confusion came from some of the modeling choices: In equation (1) the travel times added by one station is Poisson distributed?! Poisson is often used for link loads (how many people there are in a given station), not to model time. Is the quantization of time too coarse for a continuous-time model? Treating time as a Poisson r.v. requires some justification. Wouldn't a phase-type distribution(e.g., Erlang)  be a better choice for time? Such modeling choices must be explained.
The main problem, I fear, is not in the technical details. The paper is missing a bigger insight into inferring traffic loads from OD pairs. Something beyond solving this particular problem as a mixture of Poissons with path constraints.
It is important to emphasize that the evaluation is not predictive (of the future) but rather it gives estimates of hidden variables. It is a transductive method as it cannot predict anything beyond the time horizon given in the training data. I am OK with that.
Figure 4 is missing a simple baseline using the shortest paths.
The results of the model in Figure 5 don't seem to match the data in the smaller stations (e.g., Finchley Road).
Minor comments:
What is the difference between the distributions Poisson and poisson in eqs (7) and (9)?
Figure 3: botton => bottom
      A. Kumar, D. Sheldon, and B. Srivastava. 2013. Collective Diffusion over Networks: Models and Inference. In Proceedings of International Conference on Uncertainty in Artificial Intelligence. 351–360.
      Garrett Bernstein, Daniel Sheldon,Consistently Estimating Markov Chains with Noisy Aggregate Data, AISTATS, 2016.
      Du, J., Kumar, A., and Varakantham, P. (2014, May). On understanding diffusion dynamics of patrons at a theme park. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems (pp. 1501-1502). International Foundation for Autonomous Agents and Multiagent Systems.


Dual Discriminator Generative Adversarial Nets
We propose in this paper a novel approach to tackle the problem of mode collapse encountered in generative adversarial network (GAN). Our idea is intuitive but proven to be very effective, especially in addressing some key limitations of GAN. In essence, it combines the Kullback-Leibler (KL) and reverse KL divergences into a unified objective function, thus it exploits the complementary statistical properties from these divergences to effectively diversify the estimated density in capturing multi-modes. We term our method dual discriminator generative adversarial nets (D2GAN) which, unlike GAN, has two discriminators; and together with a generator, it also has the analogy of a minimax game, wherein a discriminator rewards high scores for samples from data distribution whilst another discriminator, conversely, favoring data from the generator, and the generator produces data to fool both two discriminators. We develop theoretical analysis to show that, given the maximal discriminators, optimizing the generator of D2GAN reduces to minimizing both KL and reverse KL divergences between data distribution and the distribution induced from the data generated by the generator, hence effectively avoiding the mode collapsing problem. We conduct extensive experiments on synthetic and real-world large-scale datasets (MNIST, CIFAR-10, STL-10, ImageNet), where we have made our best effort to compare our D2GAN with the latest state-of-the-art GAN's variants in comprehensive qualitative and quantitative evaluations. The experimental results demonstrate the competitive and superior performance of our approach in generating good quality and diverse samples over baselines, and the capability of our method to scale up to ImageNet database.

This paper presents a variant of generative adversarial networks (GANs) that utilizes two discriminators, one tries to assign high scores for data, and the other tries to assign high scores for the samples, both discriminating data from samples, and the generator tries to fool both discriminators.  It has been shown in section 3 that the proposed approach effectively optimizes the sum of KL and reverse KL between generator distribution and data distribution in the idealized non-parametric setup, therefore encouraging more mode coverage than other GAN variants.
The paper is quite well written and the formulation and analysis seems sound and straightforward.  The proposed approach is evaluated on toy 2D points dataset as well as more realistic MNIST, CIFAR-10, STL and ImageNet datasets.
I have one concern about the new formulation, as shown in Proposition 1, the optimal discriminators have the form of density ratios.  Would this cause instability issues?  The density ratios can vary wildly from 0 to infinity when the modes of the model and the data do not match well.  On the other hand, it is very hard for neural nets to learn to predict extremely large values with finite weights.  On the toy dataset, however, the proposed D2GAN seems to be more stable than standard GAN and unrolled GAN, is this in general true though?
I appreciate the authors’ effort in doing an extensive hyperparameter search for both the proposed method and the baselines on the MNIST dataset.  However, for a better comparison, it feels we should train the same generator network using all the methods compared, and tune the other hyperparameters for that network architecture (or a few networks).  Also, learning rate seems to be the most important hyperparameter but it is not searched extensively enough.
The paper claims they can scale up their proposed method to ImageNet, but I feel this is a little bit over-claiming, as they only tried on a subset of ImageNet and downsampled all images to 32x32, while there are already existing methods that can generate ImageNet samples for much higher resolution.
Overall the paper seems like a valid addition to the GAN literature, but doesn’t seem to change state-of-the-art too much compared to other approaches.


Reconstructing perceived faces from brain activations with deep adversarial neural decoding
Here, we present a novel approach to solve the problem of reconstructing perceived stimuli from brain responses by combining probabilistic inference with deep learning. Our approach first inverts the linear transformation from latent features to brain responses with maximum a posteriori estimation and then inverts the nonlinear transformation from perceived stimuli to latent features with adversarial training of convolutional neural networks. We test our approach with a functional magnetic resonance imaging experiment and show that it can generate state-of-the-art reconstructions of perceived faces from brain activations.

The authors propose a brain decoding model tailored to face reconstruction from BOLD fMRI measurements of perceived faces.
There are some promising aspects to this contribution, but overall in its current state there are also a number of concerning issues.
Positive points:
- a GAN decoder was trained on face embeddings coming from a triplet loss or identity-predicting face embedding space to output the original images. Modulo my inability to follow the deluge of GAN papers closely, this is a novel contribution in that it is the application of the existant imagenet reconstruction GAN to faces. This itself may be on the level of a workshop contribution.
- The reconstructed faces often coincide in gender and skin color with the original faces, indicating that the BOLD fMRI data contained this information. Reconstructing from BOLD fMRI is a very hard task and requires good data acquisition. The results indicate that a good job has been done on this front.
- the feature reconstruction numbers indicate that using a GAN for face reconstruction is a step forward with respect to other (albeit much older and inherently weaker) methods.
Points of concern:
No part of the reconstruction method proposed seems novel (as claimed) and validation is shaky in parts.
- The Bayesian framework seems superfluous. The reconstruction is performed using ridge regression with voxel importance weighted by inverse noise level. The subsection of deriving the posterior can be cut. [Notational quirk: If I understand correctly, B is not necessarily square nor necessarily invertible if it were. If the pseudoinverse is meant, it should be written with a different symbol, e.g. +]
- No reason is given as to why PCA is performed on the face embeddings and how the number of components was chosen. (If the full embedding had been chosen, without PCA, would the reconstructions of embeddings look perfect? Is the application of PCA the "crucial aspect that made this work"? If so, please elaborate with experiments as to why this could be - projecting properly into the first two (semantically meaningful) axes does seem like an important thing to be able to do and could add robustness.
- No information is provided as to which voxels were used for reconstruction. Were they selected according to how well they are predicted? If so, was the selection explicit or implicit by using the inverse noise variance weighting? If explicit (e.g. noise level cutoff), how many were selected and according to which cutoff criterion? Were these voxels mostly low-level visual or high-level visual? Could a restriction to different brain areas, e.g. known face-selective vs areas of the low-level visual hierarchy be used and evaluated?
This question is not only neuroscientifically relevant. It would provide insight into how the decoding currently works.
- The evaluation of the reconstruction seems to hang in the void a bit, as well as its comparison with other (incorrigibly worse) methods.
The GAN that was trained outputs faces. It was trained to do this and it seems to do it well. This means that the baseline correlation of *any* output of the GAN will have a relatively high correlation with any face due to alignment of features. The permutation test of significance is appropriate and this outcome should be highlighted more (how about comparing to reconstructions from random vectors also?). A histogram of correlation scores in supplementary material would also provide insight.
The comparison against the other methods seems a bit unfair. These other methods are (in modern view) clearly "all-purpose hacks", because there was not much else available to do at the time. Nobody expects eigenfaces to do well in any measure, nor does anybody expect mean and some restricted form of (diagonal?) covariance to be sufficient to model natural image data. These methods are thus clearly lacking in expressiveness and could never yield as good reconstructions as the neural network model. There are other face representations using active appearance modeling that could have been tried, which perform much better reconstruction. (No fMRI reconstruction has been attempted with these and thus it could be an interesting extra comparison.)
This outcome is good for the vggface/GAN couple in terms of face representation and reconstruction (and as stated, this may be worth writing about separately if it hasn't been done before).
It is however only the increase in feature decoding accuracy that shows that the representation is also slightly better for BOLD data (as opposed to reconstruction accuracy).
However, looking at some of the eigenface/identity reconstructions, they also sometimes get core aspects of the faces right, so they are not doing that badly on certain perceptual axes.
Lastly, why is it that the GAN reconstruction gets to have a "best possible reconstruction from this embedding" column, whereas eigenfaces and identity reconstruction do not get this privilege? Why does Table 1 not have the equivalent ratios for the other methods?
- The title of this contribution is wildly inappropriate and must be changed to reflect the content of the work. It would be less unacceptable if there had also been a reconstruction from general natural images to parallel the face reconstruction, or if there had been several other restricted data types. In the current state the title must contain sufficient information to infer that it revolves around face reconstruction and nothing else.
- Other possible axes of comparison: What about neural image representations trained on imagenet, such as Alexnet convolutional layers? How well does a GAN reconstruction from those do on faces? What about trying the face representation and reconstruction of the original DCGAN paper?
All in all, the contribution is significant but could be much more so, if more experiments had been made.


Multi-Task Learning for Contextual Bandits
Contextual bandits are a form of multi-armed bandit in which the agent has access to predictive side information (known as the context) for each arm at each time step, and have been used to model personalized news recommendation, ad placement, and other applications. In this work, we propose a multi-task learning framework for contextual bandit problems. Like multi-task learning in the batch setting, the goal is to leverage similarities in contexts for different arms so as to improve the agent's ability to predict rewards from contexts. We propose an upper confidence bound-based multi-task learning algorithm for contextual bandits, establish a corresponding regret bound, and interpret this bound to quantify the advantages of learning in the presence of high task (arm) similarity. We also describe an effective scheme for estimating task similarity from data, and demonstrate our algorithm's performance on several data sets.

Summary.
The paper is about contextual bandits with N arms. In each round the learner observes a context x_{ti} for each arm, chooses
an arm to pull and receives reward r_{ti}.
The question is what structure to impose on the rewards. The authors note that
E[r_{ti}] = < x_{ti}, theta > is a common choice, as is
E[r_{ti}] = < x_{ti}, theta_i >
The former allows for faster learning, but has less capacity while the latter has more capacity and slower learning. The natural question
addressed in this paper concerns the middle ground, which is simultaneously generalized by kernelization.
The main idea is to augment the context space so the learner observes (z_{ti}, x_{ti}) where z_{ti} lies in some other space Z.
Then a kernel can be defined on this augmented space that measures similarity between contexts and determines the degree of sharing
between the arms.
Contribution.
The main contribution as far as I can tell is the idea to augment the context space in this way. The regret analysis
employs the usual techniques.
Novelty.
There is something new here, but not much. Unless I am somehow mistaken the analysis of Valko et al. should apply directly to the augmented
contexts with more-or-less the same guarantees. So the new idea is really the augmentation.
Impact.
It's hard to tell what will be the impact of this paper. From a theoretical perspective there is not much new. The practical experiments
are definitely appreciated, but not overwhelming. Eg., what about linear Thompson sampling?
Correctness.
I only skimmed the proofs in the supplementary material, but the bound passes plausibility tests.
Overall.
This seems like a borderline paper to me. I would increase my score if the authors can argue convincingly that the theoretical results are really doing
more than the analysis in the cited paper of Valko et al.
Other comments.
- On L186 you remark that under "further assumption that after time t, n_{a,t} = t/N". But this assumption is completely unjustified, so how meaningful
  are conclusions drawn from it?
- It's a pity that the analysis was not possible for Algorithm 1. I presume the sup-"blah blah" algorithms don't really work? It could be useful to
  show their regret relative to Algorithm 1 in one of the figures. Someone needs to address these issues at some point (but I know this is a tricky problem).
Minors.
- I would capitalize A_t and other random variables.
- L49: "one estimate" -> "one estimates".
- L98: "z_a" -> "z_a \in \mathcal Z".
- L110: The notation t_a as a set that depends on t and a is just odd.
- L114: Why all the primes?
- There must be some noise assumption on the rewards (or bounded, or whatever). Did I miss it?
- L135: augmented context here is also (a, x_a).


Permutation-based Causal Inference Algorithms with Interventions
Learning directed acyclic graphs using both observational and interventional data is now a fundamentally important problem due to recent technological developments in genomics that generate such single-cell gene expression data at a very large scale. In order to utilize this data for learning gene regulatory networks, efficient and reliable causal inference algorithms are needed that can make use of both observational and interventional data. In this paper, we present two algorithms of this type and prove that both are consistent under the faithfulness assumption. These algorithms are interventional adaptations of the Greedy SP algorithm and are the first algorithms using both observational and interventional data with consistency guarantees. Moreover, these algorithms have the advantage that they are nonparametric, which makes them useful also for analyzing non-Gaussian data. In this paper, we present these two algorithms and their consistency guarantees, and we analyze their performance on simulated data, protein signaling data, and single-cell gene expression data.

The paper describes two permutation-based causal inference algorithms that can deal with interventional data. These algorithms extend a recent hybrid causal inference algorithm, Greedy SP, which performs a greedy search on the intervention equivalence classes, similar to GIES [8], but based on permutations. Both of the proposed algorithms are shown to be consistent under faithfulness, while GIES is shown not to be consistent in general.
Algorithm1 is computationally inefficient, so the evaluation is performed only with Algorithm 2 (IGSP) with two different types of independence criteria (linear Gaussian and kernel based). The evaluation on simulated data shows that IGSP is more consistent than GIES. Moreover, the authors apply the algorithm also to two biological datasets, a well-known benchmark dataset (Sachs et al.) and a recent dataset with gene expressions.
Quality
Pros:
- I didn’t check in depth the proofs, but to the best of my knowledge the paper seems to be technically sound.
- It seems to provide a good theoretical analysis in the form of consistency guarantees, as well as an empirical evaluation on both simulated and real-world datasets.
- The evaluation on the perturb-seq seems to be methodologically quite good.
Cons:
- The paper doesn’t seem to discuss a possibly obvious drawback: the potentially higher order conditional independence tests may be very brittle in the finite data case.
- The evaluation compares only with GIES, while there are other related methods, e.g. some of the ones mentioned below.
- The empirical evaluation on real-world data is not very convincing, although reconstructing causal networks in these settings reliably is known to be very difficult.
- For the Sachs data, the paper seems to suggest that the consensus network proposed in the original paper is the true causal graph in the data, while it is not clear if this is the case, so I think Figure 4 is a bit misleading. Many methods have been evaluated on this dataset, so the authors could include some more comparisons, at least in the Supplement.
Clarity
Pros:
- The paper is well-organized and clearly written.
- It is quite clear how the approach is different from previous work.
- From a superficial point of view, the proofs in the Appendix seem to be quite well-written, which seems to be quite rare among the other submissions.
Cons:
- Some parts of the background could be expanded to make it more self-contained, for example the description of Greedy SP.
- It probably sounds obvious to the authors, but maybe it would be good to point out in the beginning of the paper that the paper considers the causal sufficient case, and the case in which the intervention target is known.
- For the Sachs data, I would possibly use a Table of causal relations similar to the one in the Supplement of [13] to compare also with other methods.
Original
Pros:
- The paper seems to provide an interesting extension of Greedy SP to include interventional datasets, which allows it to be applied on some interesting real-world datasets.
Cons:
- The related work section could refer to the several constraint-based (most of them in practice hybrid) methods that have been developed recently to reconstruct causal graphs from observational and interventional datasets (even when we cannot assume causal sufficiency), for example:
-- A. Hyttinen, F. Eberhardt, and M. Järvisalo: Constraint-based Causal Discovery: Conflict Resolution with Answer Set Programming, Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, 2014.
-- Triantafillou, Sofia, and Ioannis Tsamardinos. "Constraint-based causal discovery from multiple interventions over overlapping variable sets." Journal of Machine Learning Research 16 (2015): 2147-2205.
-- Magliacane, Sara, Tom Claassen, and Joris M. Mooij. "Ancestral causal inference." Advances in Neural Information Processing Systems. 2016.
- The last paper also offers some (possibly limited) asymptotic consistency guarantees, maybe the authors could discuss its limitations with respect to their guarantees.
Significance
Pros:
- This paper and other permutation-based causal discovery approaches seem to provide a novel and interesting alternative to previous methods
Cons:
- The applicability of these methods to real-world data is reduced by the assumption that there are no latent confounders.
Typos:
- References: [13] N. Meinshausen, A. Hauser, J. M. Mooijc, -> Mooij


Query Complexity of Clustering with Side Information
Suppose, we are given a set of n elements to be clustered into k (unknown) clusters, and an oracle/expert labeler that can interactively answer pair-wise queries of the form, "do two elements u and v belong to the same cluster?". The goal is to recover the optimum clustering by asking the minimum number of queries. In this paper, we provide a rigorous theoretical study of this basic problem of query complexity of interactive clustering, and give strong information theoretic lower bounds, as well as nearly matching upper bounds. Most clustering problems come with a similarity matrix, which is used by an automated process to cluster similar points together. However, obtaining an ideal similarity function is extremely challenging due to ambiguity in data representation, poor data quality etc., and this is one of the primary reasons that makes clustering hard. To improve accuracy of clustering, a fruitful approach in recent years has been to ask a domain expert or crowd to obtain labeled data interactively. Many heuristics have been proposed, and all of these use a similarity function to come up with a querying strategy. Even so, there is a lack systematic theoretical study. Our main contribution in this paper is to show the dramatic power of side information aka similarity matrix on reducing the query complexity of clustering. A similarity matrix represents noisy pair-wise relationships such as one computed by some function on attributes of the elements. A natural noisy model is where similarity values are drawn independently from some arbitrary probability distribution f + when the underlying pair of elements belong to the same cluster, and from some f − otherwise. We show that given such a similarity matrix, the query complexity reduces drastically from Θ(nk) (no similarity matrix) to O(
where H 2 denotes the squared Hellinger divergence. Moreover, this is also information-theoretic optimal within an O(log n) factor. Our algorithms are all efficient, and parameter free, i.e., they work without any knowledge of k, f + and f − , and only depend logarithmically with n. Our lower bounds could be of independent interest, and provide a general framework for proving lower bounds for classification problems in the interactive setting. Along the way, our work also reveals intriguing connection to popular community detection models such as the stochastic block model and opens up many avenues for interesting future research.

NOTE: I am reviewing two papers that appear to be by the same authors and on the same general topic. The other one is on noisy queries without any side information.
This paper gives upper and lower bounds on the query complexity of clustering based on noiseless pairwise comparisons, along with random side-information associated with every pair. While I am familiar with some of the related literature, my knowledge of it is far from complete, so it's a little hard to fully judge the value of the contributions.  The paper is also so dense that I couldn't spend as much time as ideal checking the correctness -- especially in the long upper bound proof.
My overall impression of both papers is that the contributions seem to be valuable (if correct), but the writing could do with a lot of work. I acknowledge that the contributions are more important than the writing, but there are so many issues that my recommendation is still borderline. It would have been very useful to have another round of reviews where the suggestions are checked, but I understand this is not allowed for NIPS.
Regarding this paper, there are a few very relevant works that were not cited:
- The idea of replacing Bernoulli(p) and Bernoulli(q) by a general distribution in the stochastic block model is not new.  A Google (or Google Scholar) search for "labeled [sometimes spelt labelled] stochastic block model" or the less common "weighted stochastic block model" reveals quite a few papers that have looked at similar generalizations.
- There is also at least one work studying the SBM with an extra querying (active learning) element: "Active Learning for Community Detection in Stochastic Block Models" (Gadde et al, ISIT 2016).  It is with a different querying model, where one only queries a single node and gets its label.  But for fixed k (e.g., k=2 or k=100) and linear-size communities, this is basically the same as your pairwise querying model -- just quickly find a node from each community via random sampling, and then you can get the label of any new node by performing a pairwise query to each of those.  At first glance it looks like your lower bound contradicts their (sublinear-sample) upper bound, but this might be because your lower bound only holds as k -> Infinity (see below)
In any case, in light of these above works, I would ask the authors to remove/re-word all statements along the lines of "we initiate a rigorous theoretical study",  "significantly generalizes them", "This is the first work that...", "generalize them in a significant way", "There is no systematic theoretical study", etc.
Here are some further comments:
- I think the model/problem should be presented more clearly.  It looks like you are looking at minimax bounds, but this doesn't seem to be stated explicitly.  Moreover, is the minimax model without any restriction on the cluster sizes?  For instance, do you allow the case where there are k-1 clusters of size 1 and the other of size n-k+1?
- It looks like Theorem 2 can't be correct as stated.  For the SBM with k=2 (or probably any k=O(1)) we know that *no* queries are required even in certain cases where the Hellinger distance scales as log(n)/n.  It looks like part if the problem is that the proof assumes k -> Infinity, which is not stated in the theorem (it absolutely needs to be).  I'm not sure if that's the only part of the problem -- please check very carefully whether Theorem 2 more generally contradicts cases where clustering is possible even with Q=0.
- The numerical section does not seem to be too valuable, mainly because it does not compare against any existing works, and only against very trivial baselines. Are your methods really the only ones that can be considered/implemented?
- There are many confusing/unclear sentences, like "assume, otherwise the clustering is done" (is there text missing after "assume"?) and "the theorem is already proved from the nk lower bound" (doesn't that lower bound only apply when there's no side information?)
- The paper is FULL of grammar mistakes, typos, and strange wording. Things like capitalization, plurals, spacing, and articles (a/an/the) are often wrong. Commas are consistently mis-used (e.g., they usually should not appear after statements like "let" / "we have" / "this means" / "assume that" / "since" / "while" / "note that", and very rarely need to be used just before an equation). Sentences shouldn't start with And/Or/Plus.  The word "else" is used strangely (often "otherwise" would be better).  Footnotes should start with a capital letter and end with a full stop. This list of issues is far from complete -- regardless of the decision, I strongly urge the authors to proof-read the paper with as much care as possible.
Some less significant comments:
- The abstract seems too long, and more generally, I don't think it's ideal to spend 3 pages before getting to the contributions
- Is your use of terminology "Monte Carlo" and "Las Vegas" standard?  Also, when you first mention Las Vegas on p4, it should be mentioned alongside *AVERAGE* query complexity (the word 'average' is missing)
- Footnote 1: Should the 1- really be there after p= ?
- In the formal problem statement "find Q subset of V x V" makes it sound like the algorithm is non-adaptive.
- I disagree with the statement "clearly exhibiting" in the last sentence of the Simulations section.
- Many brackets are too small throughout the appendix
[POST-AUTHOR FEEDBACK COMMENTS]
The authors clarified a few things in the responses, and I have updated the recommendation to acceptance.  I still hope that a very careful revision is done for the final version considering the above comments.  I can see that I was the only one of the 3 reviewers to have significant comments about the writing, but I am sure that some of these issues would make a difference to a lot more NIPS readers, particularly those coming from different academic backgrounds to the authors.
Some specific comments:
- The authors have the wrong idea if they feel that "issues like capitalization and misplaced commas" were deciding factors in the review.  These were mentioned at the end as something that should be revised, not as a reason for the borderline recommendation.
- Thank you for clarifying the (lack of) contradiction with Gadde's paper.
- It is very important that theorems and lemmas are stated precisely.  e.g., If a result only holds for sufficiently large k, this MUST be stated in the theorem, and not just in the proof.
- Regarding the specific grammar issues addressed in the responses, I still suggest revising the use of Else/And/Or/Plus, the sentence "assume, otherwise...", and so on.  I am convinced that most of these are grammatically incorrect as used in the paper, or at least unsuitable language for an academic paper.  (The authors may disagree)


Dynamic Routing Between Capsules
A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.

Overview :this paper introduces a dynamic routing process for connecting layers in a feedforward neural net, as described in Procedure 1 on p 3. The key idea here is that the coupling coeff c_ij between unit i and unit j is computed dynamically (layerwise), taking into account the agreement between the output v_j of unit j, and the prediction from unit i \hat{u}_{{j|i}. This process is iterates between each layer l and l+1, but does not (as far as I can tell) spread further back.
Another innovation used in the paper is a form of nonlinearity as in eq 1 for units which uses the length of the capsule output v_j to encode strength of activity, and the direction of v_j to encode the values of the capsule parameters.
A shallow CapsNet model is trained on MNIST, and obtains very good performance (a check of the MNIST leaderboard shows best performance of 0.23 obtained with a committee of deep conv nets), cf performance in Table 1.
I regard this paper as very interesting, as it has successfully married the capsules idea with conv nets, and makes use of the dynamic routing capabilities. The results on highly overlapping digits (sec 6) are also impressive. This idea could rally take off and be heavily used.
One major question about the paper is the convergence of the iteration in Proc 1. It would be highly desirable to prove that this converges (if it does so). This issue is as far as I can tell not discussed in the paper.
It would also be very nice to see an example of the dynamic routing process in action, especially to see how a unit whose \hat{u}_{j|i} is out of agreement with the others gets suppressed.
This process of assessing agreement of bottom-up contributions is reminiscent of the work in Rao and Ballard (Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects, Nature Neurosci. 2(1) 79-87, 1999), esp the example where local bar detectors all predict the same longer bar. One could e.g. look at how activations of 3 units in a line --- compare to one that is disoriented, e.g. |-- in your network.
I would encourage the authors to publish their code (this can be following acceptance). This would greatly facilitate uptake by the community.
Quality: As far as I can tell this is technically sound.
Clarity: The writing is clear but my detailed comments give a number of suggestions to make the presentation less idiosyncratic.
Originality: The idea of capsules is not new, but this paper seems to have achieved a step change in their performance and moved beyond the "transforming autoencoder" framework in which they were trained.
Significance: The level of performance that can be achieved with a relatively shallow network is impressive. Given the interest in object recognition etc it is possible that this dynamic routing idea could really take off.
Other points:
p 1 Need to ref Hinton et al (20111) when you first mention "capsules" in the text.
p2 and Procedure 1:I strongly recommend "squashing" to "squidging" as it is a term that has been used in the NN literature (and sounds more mathematical).
p 2 The constraint that the sum_j c_{ij} = 1 is reminiscent of the "single parent constraint" in Hinton et al (2000). It would be good to discuss this more.
p 3. The 0.5 in eq (4) seems like a hack, although it would look better if you introduced a weighting parameter alpha and set it to 0.5. Also I don't understand why you are not just using softmax loss (for a single digit classifier) on the ||v_j||'s (as they lie in 0,1). (For the multiclass case you can use multiple independent sigmoids.)
p 3 I recommend "PrimaryCapsules" rather than "PrimaryCaps" as Caps sounds like CAPS.
p 5 Lee et al [2016] baseline -- justify why this is an appropriate baseline.
p 5 sec 5.1 Tab. 4 -> Fig. 4
p 6 Figure 4 caption: "Each column" -> "Each row".
p 6 Sec 6 first para -- it is confusing to talk about your performance level before you have introduced the task -- leave this for later.
p 6 sec 6: note a heavily overlapping digits task was used in Hinton et al (2000).
p 8. It would be better to put the related work earlier, e.g. after sec 4.
p 9. The ordering of the references (as they appear in the paper) is unusual and makes them hard to find. Suggest alphabetical or numbered in order of use. These are standard bibtex options.


Sparse Approximate Conic Hulls
We consider the problem of computing a restricted nonnegative matrix factorization (NMF) of an m × n matrix X. Specifically, we seek a factorization X ≈ BC, where the k columns of B are a subset of those from X and C ∈ R k×n ≥0 . Equivalently, given the matrix X, consider the problem of finding a small subset, S, of the columns of X such that the conic hull of S ε-approximates the conic hull of the columns of X, i.e., the distance of every column of X to the conic hull of the columns of S should be at most an ε-fraction of the angular diameter of X. If k is the size of the smallest ε-approximation, then we produce an O(k/ε 2/3 ) sized O(ε 1/3 )-approximation, yielding the first provable, polynomial time ε-approximation for this class of NMF problems, where also desirably the approximation is independent of n and m. Furthermore, we prove an approximate conic Carathéodory theorem, a general sparsity result, that shows that any column of X can be ε-approximated with an O(1/ε 2 ) sparse combination from S. Our results are facilitated by a reduction to the problem of approximating convex hulls, and we prove that both the convex and conic hull variants are d-SUM-hard, resolving an open problem. Finally, we provide experimental results for the convex and conic algorithms on a variety of feature selection tasks.

The paper "Sparse approximate conic hulls" develops conic analogues of approximation problems in convex geometry, hardness results for approximate convex and conic hulls, and considers these in the context of non-negative matrix factorization. The paper also presents numerical results comparing the approximate conic hull and convex hull algorithms, a modified approximate conic hull algorithm (obtained by first translating the data), and other existing algorithms for a feature-selection problem. Numerical results are also presented.
The first theoretical contribution is a conic variant on the (constructive) approximate Caratheodory theorem devised for the convex setting. This is obtained by transforming the rays (from the conic problem) into a set of vectors by the "gnomic projection" applying the approximate Caratheodory theorem in the convex setting, and transforming back. The main effort is to ensure the error behavior can be controlled when the gnomic projection/its inverse are applied. This requires the points to have angle to the "center" strictly smaller than pi/2. This kind of condition on the points persists throughout the "conic" results in the paper. The second main contribution is establishing hardness results for approximate conic and convex hull problems. Finally, the main point of the paper seems to be that the epsilon-approximate conic hull algorithm can be used to approximately solve NMF under the column subset restriction.
The paper is quite well written, albeit a little dense, with much of the core technical content relegated to the supplementary material. The results are interesting from a purely convex geometry point of view. Nevertheless, I am somewhat unconvinced about certain issues in the formulation of the problems in the paper (rather than the technical results of the paper, which are nice):
-- the choice of angular metric seems a bit arbitrary (there are other ways to put a distance on the non-negative orthant that is naturally adapted to the conic setting, such as the Hilbert metric. Perhaps the angular metric is best suited to using Frobenius norm error in the matrix factorization problem? If so it would be great if the authors could make this more clear.
-- The gamma-shifted conic version performs well in experiments (and the original conic version does not), which is interesting. How should we choose the shift though? What reason is there not to make the shift very large? Is it possible to "pull back" the gamma shift throughout the paper, and formulate a meaningful version of approximate Caratheodory that has a parameter gamma? Perhaps choosing the best-case over gamma is a more interesting formulation (from a practical point of view) than the vanilla conic version studied in this paper.
In addition to these concerns, I'm not sure how much innovation there is over the existing convex method of Blum et al, and over the existing hardness results for NMF in Arora et al. (I am not expert enough to know for sure, but the paper feels a little incremental).
Minor comments:
-- p3 line 128: this is not the usual definition of "extreme" point/ray in convex geometry (although lines 134-136 in some sense deal with the difference between the usual definition and the definition in this paper, via the notion of "degenerate")
-- p4 line 149: one could also consider using the "base" of the cone defined by intersection with any hyperplane defined by a vector in the interior of the dual cone of the points (not just the all ones vector) instead of using the gnomic projection. This preserves the extreme points,
 so the basic strategy might work, but perhaps doesn't play nicely with Frobenius norm error in the NMF problem? It would be helpful if the authors could briefly explain why this approach is not favorable, and why the gnomic projection approach makes the most sense.
-- p5 line 216, theorem 3.4: It would be useful if the authors describe the dependence on gamma in the parameters of the theorem. It seems this will be very bad as gamma approaches pi/2, and it would be good to be upfront about this (since the authors do a good job of making clear that for the problem setup in this paper the bounded angle assumption is necessary)
-- p6 line 235, there is some strange typesetting with the reference [16] in the Theorem statement.
-- p6 line 239: again the dependence on gamma would be nice to have here (or in a comment afterwards).
-- p6 line 253: it may be useful to add a sentence to say how this "non-standard" version of d-sum differs from the "standard" version, for the non-expert reader.
-- p8 lines 347-348: this is an interesting observation, that the gamma-shifted conic case is a sort of interpolant between the convex and conic cases. It would be interesting to be able to automatically tune gamma for a given scenario.


Aggressive Sampling for Multi-class to Binary Reduction with Applications to Text Classification
We address the problem of multi-class classification in the case where the number of classes is very large. We propose a double sampling strategy on top of a multi-class to binary reduction strategy, which transforms the original multi-class problem into a binary classification problem over pairs of examples. The aim of the sampling strategy is to overcome the curse of long-tailed class distributions exhibited in majority of large-scale multi-class classification problems and to reduce the number of pairs of examples in the expanded data. We show that this strategy does not alter the consistency of the empirical risk minimization principle defined over the double sample reduction. Experiments are carried out on DMOZ and Wikipedia collections with 10,000 to 100,000 classes where we show the efficiency of the proposed approach in terms of training and prediction time, memory consumption, and predictive performance with respect to state-of-the-art approaches.

This paper presents a learning reduction for extreme classification, multiclass classification where the output space ranges up 100k different classes.
Many approaches to extreme classification rely on inferring a tree structure among the output labels, as in the hierarchical softmax, so that only a logarithmic number of binary predictions need to be made to infer the output class, or a label embedding approach that can be learned efficiently through least-squares or sampling. The authors note the problems with these approaches: the inferred latent tree may not be optimal and lead to cascading errors, and the label embedding approach may lead to prediction errors when the true label matrix is not low-rank.
An alternative approach is to reduce extreme classification to pairwise binary classification. The authors operate in the second framework and present a scalable sampling based method with theoretical guarantees as to its consistency with the original multiclass ERM formulation.
Experiments on 5 text datasets comparing a number of competing approaches (including standard one-vs-all classification when that is computationally feasible) show that the method is generally superior in terms of accuracy and F1 once the number of classes goes above 30k, and trains faster while using far less memory, although suffers somewhat in terms of prediction speed.
The Aggressive Double Sampling reduction is controlled by two parameters, one of which controls the frequency of sampling each class, which is inversely proportional to the empirical class frequency in order to avoid entirely missing classes in the long tail. The second parameter sets the number of adversarial examples to draw uniformly. This procedure is repeated to train the final dyadic classifier.
At prediction time, inference is complicated by the fact that generating pairwise features with all possible classes is intractable. A heuristic is used to identify candidate classes by making an input space centroid representation of each class as the average of vectors in the class, with prediction being carried out by a series of pairwise classifications with only those candidates.
Theoretical analysis of the reduction is complicated by the fact that the reduction from multi-class to dyadic binary classification changes a sum of independent random variables into a sum of random variables with a particular dependence structure among the dyadic examples. The other complication is caused by the oversampling of rare classes, shifting the empirical distribution. The proof involves splitting the sum of dependent examples into several sums of independent variables based on the graph structure induced by the example construction, and using the concentration equalities for partially dependent variables introduced by Janson. Although the finite-sample risk bound is biased by the ratios between the true class probabilities and the oversampled ones, these biases decrease linearly with the size of the sampled and re-sampled training sets. The analysis appears sound but I have not verified the proof in detail.
Overall, this paper presents a strong method for extreme classification, with small memory and computational requirements and superior performance on especially the largest datasets. The binary reduction algorithm appears theoretically sound, interesting, and useful to practitioners.


Bayesian Compression for Deep Learning
Compression and computational efficiency in deep learning have become a problem of great significance. In this work, we argue that the most principled and effective way to attack this problem is by adopting a Bayesian point of view, where through sparsity inducing priors we prune large parts of the network. We introduce two novelties in this paper: 1) we use hierarchical priors to prune nodes instead of individual weights, and 2) we use the posterior uncertainties to determine the optimal fixed point precision to encode the weights. Both factors significantly contribute to achieving the state of the art in terms of compression rates, while still staying competitive with methods designed to optimize for speed or energy efficiency.

This paper describes a method for compressing a neural network: pruning its neurons and reducing the floating point precision of its weights to make it smaller and faster at test time. The method works by training the neural net with SVI using a factored variational approximation that shares a scale parameter between sets of weights. The entropy term of the variational objective incentivizes weights to have a large variance, which leads to compression by two separate mechanisms: neurons with high-variance weights relative to their means can be pruned, and layers can have their floating point precision reduced to a level commensurate with the variance of their weights. It's a fairly principled approach in which compression arises naturally as a consequence of Bayesian uncertainty. It also seems effective in practice.
I have three main reasons for not giving a higher score. First, I'm not convinced that the work is hugely original. It leans very heavily on previous work, particularly Kingma et al. on variational dropout and Molchanov et al. on sparsification arising from variational dropout. The two novelties are (1) hierarchical priors and (2) reduced floating point precision, which are certainly useful but they are unsurprising extensions of the previous work. Additionally, the paper does a poor job of placing itself in the context of these previous papers. Despite repeatedly referencing the variational dropout papers for forms of variational approximations or derivations, the concept of 'variational dropout' is never actually introduced! The paper would be much clearer if it at least offered a few-sentence summary of each of these previous papers, and described where it departs from them.
Second, the technical language is quite sloppy in places. The authors regularly use the phrase "The KL divergence between X and Y" instead of "the KL divergence from X to Y". "Between" just shouldn't be used because it implies symmetry. Even worse, in all cases, the order of "X" and "Y" is backwards, so that even "from X to Y" would be incorrect. Another example of sloppy language is the use of the term "posterior" to describe the variational approximation q. This causes genuine confusion, because the actual posterior is a very relevant object. At the very least, the term should be "posterior approximation". In the paper's defense, the actual equations are generally clear and unambiguous.
Finally, the experiments are a bit unimaginative. For example, although the authors argue that Bayesian inference and model compression are well aligned, we know they have quite different goals and you would expect to find some situations where they could be in conflict. Why not explore this tradeoff by looking at performance and compression as a function of hyperparameters like $\tau_0$?  How much compression do you get if you select $\tau_0$ by maximizing the marginal likelihood estimate rather than choosing an arbitrary value?


Scalable Generalized Linear Bandits: Online Computation and Hashing
Generalized Linear Bandits (GLBs), a natural extension of the stochastic linear bandits, has been popular and successful in recent years. However, existing GLBs scale poorly with the number of rounds and the number of arms, limiting their utility in practice. This paper proposes new, scalable solutions to the GLB problem in two respects. First, unlike existing GLBs, whose per-time-step space and time complexity grow at least linearly with time t, we propose a new algorithm that performs online computations to enjoy a constant space and time complexity. At its heart is a novel Generalized Linear extension of the Online-to-confidence-set Conversion (GLOC method) that takes any online learning algorithm and turns it into a GLB algorithm. As a special case, we apply GLOC to the online Newton step algorithm, which results in a low-regret GLB algorithm with much lower time and memory complexity than prior work. Second, for the case where the number N of arms is very large, we propose new algorithms in which each next arm is selected via an inner product search. Such methods can be implemented via hashing algorithms (i.e., "hash-amenable") and result in a time complexity sublinear in N . While a Thompson sampling extension of GLOC is hash-amenable, its regret bound for d-dimensional arm sets scales with d 3/2 , whereas GLOC's regret bound scales with d. Towards closing this gap, we propose a new hashamenable algorithm whose regret bound scales with d 5/4 . Finally, we propose a fast approximate hash-key computation (inner product) with a better accuracy than the state-of-the-art, which can be of independent interest. We conclude the paper with preliminary experimental results confirming the merits of our methods.

The paper identifies two problems with existing Generalized Linear Model Bandit algorithms: their per-step computational complexity is linear in t (time-steps) and N (number of arms). The first problem is solved by the proposed GLOC algorithm (first that achieves constant-in-t runtime and O(d T^½ polylog(T) ) regret) and the second by a faster approximate variant (QGLOC) based on hashing. Minor contributions are a (claimed non-trivial) generalization of linear online-to-confidence bounds to GLM, an exploration-exploitation tradeoff parameter for QGLOC that finally justifies a common heuristic used by practitioners and a novel simpler hashing method for computing QGLOC solutions.
I think the paper is interesting and proposes a novel idea, but the presentation is sometimes confused and makes it hard to evaluate the impact of the contribution w.r.t. the existing literature.
the authors claim that all existing GLMB algorithms have a per-step complexity of t. This is clearly not the case in the linear setting when \mu is the identity, and closed forms for \hat{theta}_t allow the solution to be computed incrementally. This cannot be done in general in the GLM where \hat{\theta}_t is the solution of a convex optimization problem that requires O(t) time to be created and solved. Indeed, GLOC itself uses ONS to estimate \theta, and then obtain a closed form solution for \hat{\theta}, sidestepping the O(t) convex problem. The authors should improve the presentation and comparison with existing methods, highlighting when and where this dependence on t comes from and why other methods (e.g., why hot restarting the convex solver with \hat{\theta}_{t-1}) cannot provide the same guarantees, or other examples in which closed forms can be computed.
The authors should expand the discussion on line 208 to justify the non-triviality: what original technical results were necessary to extend Abbasi-Yadkori [2] to GLMs?
While the road to solving the O(t) problem is somewhat clear, the approach to reduce the O(N) complexity is not well justified. In particular (as i understood):
GLOC-TS is hash-amenable but has d^3/2 regret (when solved exactly)
QGLOC is hash-amenable and has d^5/4 regret, but only when solved exactly
GLOC-TS has no know regret bound when solved inexactly (using hashing)
QGLOC has a fixed-budget regret guarantee (presented only in the appendix, d^5/4 or d^3/2?) that holds when solved inexactly (using hashing), but reintroduces a O(t) space complexity (no O(t) time complexity?)
This raises a number of questions: does GLOC-TS have a regret bound when solved inexactly? Does any of these algorithms (GLOC, QGLOC, GLOC-TS) guarantee O(d^{1,5/4,3/2}\sqrt{T}) regret while using sublinear space/time per step BOTH in t and N at the same time?
The presentation should also be improved (e.g., L1-sampling requires choosing m, but QGLOC's theorem makes no reference to how to choose m)
The experiments report regret vs. t (time), and I take t here is the number of iterations of the algorithm. In this metric, UCB-GLM (as expected) outperforms the proposed method. It is important to report metrics (runtime) that were the reason for the introduction of GLOC, namely per-step time and space complexity, as t and N change.
Minor things:
L230 "certain form" is a vague term
L119 S must be known in advance, it is very hard to estimate, and has a linear impact on the regret. How hard it is to modify the algorithm to adapt to unknown S?
L109+L127 The loss fed to the online learner is \ell(x,y) = -yz +m(z) and m(z) is \kappa strongly convex under Ass. 1. For strongly convex functions, ONS is not necessary and simple gradient descent achieves logarithmic regret. Under appropriate conditions on \kappa and y, the problem might become simpler.
L125 Ass. 1 is common in the literature, but no intuition for a value of \kappa is provided. Since the regret scales linearly with \kappa, it would be useful to show simple examples of known \kappa e.g. in the probit model and 0/1 rewards used in the experiments. It would also be useful to show that it does not satisfy the previous remark.
L44 3 is not included in the "existing algorithm". Typo, or TS does not suffer from both scalability issues?
L158 S should be an input of Alg. 1 as well as Alg. 2
AFTER REBUTTAL
	I have read the author rebuttal, and I confirm my accept. I believe the paper is interesting, but needs to make a significant effort to clarify the message on what it does and what it does not achieve.


Unsupervised Sequence Classification using Sequential Output Statistics
We consider learning a sequence classifier without labeled data by using sequential output statistics. The problem is highly valuable since obtaining labels in training data is often costly, while the sequential output statistics (e.g., language models) could be obtained independently of input data and thus with low or no cost. To address the problem, we propose an unsupervised learning cost function and study its properties. We show that, compared to earlier works, it is less inclined to be stuck in trivial solutions and avoids the need for a strong generative model. Although it is harder to optimize in its functional form, a stochastic primal-dual gradient method is developed to effectively solve the problem. Experiment results on real-world datasets demonstrate that the new unsupervised learning method gives drastically lower errors than other baseline methods. Specifically, it reaches test errors about twice of those obtained by fully supervised learning.

This paper presents a method for predicting a sequence of labels without a labeled training set. This is done by incorporating a prior probability on the possible output sequnces of labels, that is a linear classifier is sought such that the distribution of sequnces it predicts is close (in the KL sense) to a given prior. The authors present a cost function, an algorithm to optimize it, and experimental results on real data of predicting characters in two tasks: OCR and spell correction.
	  The paper is technically sound and generally clear. The idea is novel (some prior work is cited by the authors) and interesting. The main reason for my score is a concern regarding the difficulty of obtaining a useful prior model p_LM. Such models usually suffer from sparseness of data, that is unless the vocabulary is quite small, it is common for some test samples to have zero probability under the prior/LM learned from a training set. It is true that this isuue is well known and solutions exist in the literature, however I would like the authors to adress the influence of it on their method. Another aspect of the same issue is the scaling of the algorithm to large vocabularies and/or long sub-sequences (parameter N in the paper). Indeed, the experiments use quite a small vocabulary (29 characters) and short sub-sequences (Ngrams with N=2,3). In NLP data at word level, the vocabulary is orderes of magnitude larger.
	  Some more comments:
	  - I would also like the authors to adress the application of their method to non-NLP domains.
	  - The inclusion of the supervised solution in the figures of the 2D cost raises a question - it seems the red dot (the supervised solution) is always in a local minimum of the unsupervised cost - I don't understand why this should be the case.
	  - If it is assumed that there is a structure in the output space of sequences, why not incorporate this structure into the classifier, that is why use a point prediction p(y_t|x_t) of a single label independently of its neighbors?
	  Specific comments:
	  (*) lines 117-118, 'negative cross entropy', isn't the following formula the cross entropy (and not the negative of it)?
	  (*) line 119, this sentence is somewhat confusing, if I understand correctly \bar p is the expected frequency of a *given* sequence {i_1,...i_n}, and not of *all* sequences.
	  (*) lines 163-166, here too the sign of the cross entropy seems to be reversed, also when referred to in the text, e.g in line 163, if p_LM goes to 0, then -log(0) is +inf and not -inf as written in the paper, and it indeed suits the sentence in line 167, since you expect that to penalize a minimization problem, some term goes to +inf and not to -inf.
	  Typos:
	  (*) lines 173 and 174, "that p_LM" seems wrong, did the authors mean "for which p_LM" ?
	  (*) line 283, word 'rate' appears twice.


Deep Voice 2: Multi-Speaker Neural Text-to-Speech
We introduce a technique for augmenting neural text-to-speech (TTS) with lowdimensional trainable speaker embeddings to generate different voices from a single model. As a starting point, we show improvements over the two state-ofthe-art approaches for single-speaker neural TTS: Deep Voice 1 and Tacotron. We introduce Deep Voice 2, which is based on a similar pipeline with Deep Voice 1, but constructed with higher performance building blocks and demonstrates a significant audio quality improvement over Deep Voice 1. We improve Tacotron by introducing a post-processing neural vocoder, and demonstrate a significant audio quality improvement. We then demonstrate our technique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron on two multi-speaker TTS datasets. We show that a single neural TTS system can learn hundreds of unique voices from less than half an hour of data per speaker, while achieving high audio quality synthesis and preserving the speaker identities almost perfectly.
3. Using these two single-speaker models as a baseline, we demonstrate multi-speaker neural speech synthesis by introducing trainable speaker embeddings into Deep Voice 2 and Tacotron.
We organize the rest of this paper as follows. Section 2 discusses related work and what makes the contributions of this paper distinct from prior work. Section 3 presents Deep Voice 2 and highlights the differences from Deep Voice 1. Section 4 explains our speaker embedding technique for neural TTS models and shows multi-speaker variants of the Deep Voice 2 and Tacotron architectures. Section 5.1 quantifies the improvement for single speaker TTS through a mean opinion score (MOS) evaluation and Section 5.2 presents the synthesized audio quality of multi-speaker Deep Voice 2 and Tacotron via both MOS evaluation and a multi-speaker discriminator accuracy metric. Section 6 concludes with a discussion of the results and potential future work.

This paper presents a solid piece of work on the speaker-dependent neural TTS system, building on previous works of Deep Voice and Tacotron architecture. The key idea is to learn a speaker-dependent  embedding vector jointly with the neural TTS model. The paper is clearly written, and the experiments are presented well.
My comments are as follows.
-- the speaker embedding vector approach is very similar to the speaker code approach for speaker adaptation studied in ASR, e.g.,
Ossama Abdel-Hamid, Hui Jiang, "Fast speaker adaptation of hybrid NN/HMM model for speech recognition based on discriminative learning of speaker code", ICASSP 2013
This should be mentioned in the paper. ASR researchers later find that using fixed speaker embeddings such i-vectors can work equally well (or even better). It would be interesting if the authors could present some experimental results on that. The benefit is that you may be able to do speaker adaptation fairly easily, without learning the speaker embedding as the presented approach.
--in the second of paragraph of Sec. 3, you mentioned that the phoneme duration and frequency models are separated in this work. Can you explain your motivation?
--the CTC paper from A. Graves should be cited in section 3.1. Also, CTC is not a good alignment model for ASR, as it is trained by marginalizing over all the possible alignments. For ASR, the alignments from CTC may be very biased from the ground truth. I am curious how much segmentation error do you have, and have you tried some other segmentation models, e.g., segmental RNN?
--in Table 1, can you present comparisons between Deep Voice and Tocotron with the same sample frequency?
--is Deep Voice 2 trained end-to-end, or the modules are trained separately? It would be helpful to readers to clarify that.
--the speaker embedding are used as input to several different modules, some ablation experiments in the appendix would be helpful to see how much difference it would make for each module.


Approximation and Convergence Properties of Generative Adversarial Learning
Generative adversarial networks (GAN) approximate a target data distribution by jointly optimizing an objective function through a "two-player game" between a generator and a discriminator. Despite their empirical success, however, two very basic questions on how well they can approximate the target distribution remain unanswered. First, it is not known how restricting the discriminator family affects the approximation quality. Second, while a number of different objective functions have been proposed, we do not understand when convergence to the global minima of the objective function leads to convergence to the target distribution under various notions of distributional convergence. In this paper, we address these questions in a broad and unified setting by defining a notion of adversarial divergences that includes a number of recently proposed objective functions. We show that if the objective function is an adversarial divergence with some additional conditions, then using a restricted discriminator family has a moment-matching effect. Additionally, we show that for objective functions that are strict adversarial divergences, convergence in the objective function implies weak convergence, thus generalizing previous results.

The authors present a formal analysis to characterize general adversarial learning. The analysis shows that under certain conditions on the objective function the adversarial process has a moment-matching effect. They also show results on convergence properties.
The writing is quite dense and may not be accessible to most of the NIPS audience. I did not follow the full details myself. I defer to reviewers closer to the area. One general reservation I have about a formal analysis along the line presented is whether the results actually provide any insights useful in the practice of generative adversarial networks. In particular, do the results suggest properties of domains / distributions that make such domains suitable for a GAN approach, and, conversely, what practical domain properties would suggest a bad fit for GAN? Do the results provide guidelines for the choice of deep net architecture or learning parameters? Ideally, a formal analysis provides such type of feedback to the practitioners. If no such feedback can be provided, the formal results may still be of interest but unfortunately, they will remain largely a theoretical exercise. But at least, I would like to see a brief discussion on the impact on the practice of these results because it's the practical success of the GAN approach that has made it so exciting. (A similar issue arises with any formal framework that "explains" the success of deep learning. Such explanations become much more interesting if they also identify domain properties that predict where deep learning would work well and where it would not, in relation to real-world domains. I.e. what are the lessons for the practice of deep learning?)
I'm happy with the authors' response. It would be great to see the feedback incorporated in the revised paper. I've raised my rating.


Zap Q-Learning
The Zap Q-learning algorithm introduced in this paper is an improvement of Watkins' original algorithm and recent competitors in several respects. It is a matrix-gain algorithm designed so that its asymptotic variance is optimal. Moreover, an ODE analysis suggests that the transient behavior is a close match to a deterministic Newton-Raphson implementation. This is made possible by a two time-scale update equation for the matrix gain sequence. The analysis suggests that the approach will lead to stable and efficient computation even for non-ideal parameterized settings. Numerical experiments confirm the quick convergence, even in such non-ideal cases.

The paper makes the following contributions:
1) It shows that the asymptotic variance of tabular Q-learning decreases slower than the typical 1/n rate even when an exploring policy is used.
2) It suggests a new algorithm, Zap Q(lambda) learning to fix this problem.
3) It shows that in the tabular case the new algorithm can deliver optimal asymptotic rate and even optimal asymptotic variance (i.e., optimal constants).
4) The algorithm is empirically evaluated on both a simple problem and on a finance problem that was used in previous research.
Significance, novelty
---------------------
Q-learning is a popular algorithm, at least in the textbooks. It is an instance of the family of stochastic approximation algorithms which are (re)gaining popularity due to their lightweight per-update computational requirements.
While (1) was in a way known (see the reference in the paper), the present paper complements and strengthens the previous work. The slow asymptotic convergence at minimum must be taken as a warning sign. The experiments show that the asymptotics in a way correctly predicts what happens in finite time, too. Fixing the slow convergence of Q-learning is an important problem. The fix, while relatively straightforward (but not entirely trivial), is important. My only worry is about the computational cost associated with matrix inversions. Even with incremental updates (Sherman-Morrison), the per-update cost is O(d^2) when we have d features. This may be OK in some applications, but it is a limitation (not a show-stopper by any means). The empirical evaluations are of sufficient depth and breadth for a conference publication proposing a new method (I actually like the way that they are conducted). The condition A3 seems strong (as noted by the authors themselves).
I don't have high hopes for the generalization to the full linear feature-based case as in this Q-learning, as many of its relatives, tend not to have any strong guarantees (i.e., performance can be quite bad). However, this is not the problem of *this* paper.
Soundness
---------
I believe that the results are sound. I could not check the long version (yet) as I was asked to write a review just two days ago (as an extra review) and I did not have the time to check the proofs. From what I understand about these type of proofs, I believe that the proof can go through but there are some delicate steps (e.g., nonsmooth right-hand side of the ODE). That the authors discovered that A3 is necessary for their proof to go through at least shows that they are quite careful and attentive to details. (I am siding with them in that I believe it may be possible to remove A3, but this won't be trivial).
The experiments are done with care; demonstrating nicely the effect of learning. Perhaps a sensitivity study could complement the experiments presented, but this is not critical in my opinion for a conference publication.
Quality of writing
------------------
Very well written pose. Some may say that perhaps fewer symbols could be used, but at the end the notation was consistent and systematic and I did not find the paper too hard to read.
Related work
------------
There is some similarity to [1-5], which would be good to cite. While these papers are definitely relevant (they seem to employ the same two-time scale logic in similar, TD/LMS context), the present paper is sufficiently different so that besides citing these as implementing the same idea in a different setting, no other modification is necessary to the paper.
Minor comments
--------------
* Perhaps Theorem 2.1 does not need Q2?
* Line 120: "the gain sequence gamma" should be "the gain sequences gamma and alpha"
* Line 124: The uninitiated reader may find it hard to interpret the phrase "An ODE approximation holds".
* Numerical results: I expect the authors to share their code (github is excellent for this) or otherwise make it clear what parameter settings are used. A few details seem to be missing to replicate the experiments (e.g., what stepsize was used with RPJ?)
* Line 151: "the Watkins'": remove "the"
* Line 152: Which analysis does "this" refer to? (Middle of the page.)
* Figure 3: I am guessing W_n is the error. I am not sure whether this was introduced beforehand. Maybe just remind the reader if it was.
* Fig 4: Why is Speedy-Q not converging? Will it?? Overall vanilla Q-learning is not even that bad here..
* Line 174: "trails" should be "trials"?
* Line 202: How is the condition number defined given that A is not even symmetric, so it may have complex eigenvalues? Magnitudes I assume?
* Line 209: Hmm, I see W_n is defined here.. Move this up?
References
[1] H. Yao and Z-Q. Lie. Preconditioned temporal difference learning. In Proceedings of the
25th International Conference on Machine learning, 2008.
[2] H. Yao, S. Bhatnagar, and Cs. Szepesvari.  Temporal difference learning by direct preconditioning.  In
Multidisciplinary Symposium on Reinforcement Learning (MSRL), 2009.
[3]  Yao, H., Bhatnagar, S., and Szepesvári, Cs., LMS-2: Towards an Algorithm that is as Cheap as LMS and Almost as Efficient as RLS, CDC, pp. 1181--1188, 2009
[4] Pan, Yangchen, Adam M. White, and Martha White. "Accelerated Gradient Temporal Difference Learning." AAAI. 2017.
[5] Givchi, A., & Palhang, M. (2014). Quasi Newton Temporal Difference Learning. ACML.


Fader Networks: Manipulating Images by Sliding Attributes
This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space. As a result, after training, our model can generate different realistic versions of an input image by varying the attribute values. By using continuous attribute values, we can choose how much a specific attribute is perceivable in the generated image. This property could allow for applications where users can modify an image using sliding knobs, like faders on a mixing console, to change the facial expression of a portrait, or to update the color of some objects. Compared to the state-of-the-art which mostly relies on training adversarial networks in pixel space by altering attribute values at train time, our approach results in much simpler training schemes and nicely scales to multiple attributes. We present evidence that our model can significantly change the perceived value of the attributes while preserving the naturalness of images.

*** Summary ***
This paper aims to augment an autoencoder with the ability to tweak certain (known) attributes. These attributes are known at training time and for a dataset of faces include aspects like [old vs young], [smiling vs not smiling], etc.
They hope to be able to tweak these attributes along a continuous spectrum, even when the labels only occur as binary values.
To achieve this they propose an (encoder, decoder) setup where the encoder maps the image  x to a latent vector z and then the decoder produces an image taking z, together with the attributes y as inputs.
When such a network is trained in the ordinary fashion, the decoder learns to ignore y because z already encodes everything that the network needs to know. To compel the decoder network to use y, the authors propose introducing a adversarial learning framework in which a discriminator D is trained to infer the attributes from z. Thus the encoder must produce representations that are invariant to the attributes y.
The writing is clear and any strong researcher should be able to reproduce their results from the presentation here.
*** Conclusions ***
This paper presents a nice idea. Not an inspiring, or world changing idea, but a worthy enough idea. Moreover, this technique offers a solution that large number of people are actually interested in (for good or for malice).
What makes this paper a clear accept is that the authors take this simple useful idea and execute it wonderfully. The presentation is crystal clear. The experiments are compelling (both quantitatively and qualitatively). The live study with Mechanical Turk workers answers some key questions any reasonable reviewer might ask.
I expect this work to actually be used in the real world and happily champion it for acceptance at NIPS 2017.
*** Small Writing Quibble ***
"We found it extremely beneficial to add dropout"
Adverbs of degree (like "very", "extremely", etc) add nothing of value to the sentence. Moreover, it expresses an opinion in a sly fashion. Instead of saying dropout is "extremely" important, say ***why*** it is important. What happens when you don't use dropout? How do the results compare qualitatively and quantitatively?


Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models
This paper studies the numerical computation of integrals, representing estimates or predictions, over the output f (x) of a computational model with respect to a distribution p(dx) over uncertain inputs x to the model. For the functional cardiac models that motivate this work, neither f nor p possess a closed-form expression and evaluation of either requires ≈ 100 CPU hours, precluding standard numerical integration methods. Our proposal is to treat integration as an estimation problem, with a joint model for both the a priori unknown function f and the a priori unknown distribution p. The result is a posterior distribution over the integral that explicitly accounts for dual sources of numerical approximation error due to a severely limited computational budget. This construction is applied to account, in a statistically principled manner, for the impact of numerical errors that (at present) are confounding factors in functional cardiac model assessment.

Summary
The paper presents a method for assessing the uncertainty in the evaluation of an expectation over the output of a complex simulation model given uncertainty in the model parameters. Such simulation models take a long time to solve, given a set of parameters, so the task of averaging over the outputs of the simulation given uncertainty in the parameters is challenging. One cannot simply run the model so many times that error in the estimate of the integral is controlled. The authors approach the problem as an inference task. Given samples from the parameter posterior one must infer the posterior over the integral of interest. The parameter posterior can be modelled as a DP mixture and my matching the base measure of the DP mixture with the kernel used in the Gaussian process based Bayesian Quadrature (BQ) procedure for solving the integral. The method is benchmarked using synthetic and a real-world cardiac model against simply using the mean and standard error of simulations from the model for parameter samples. It is shown to be more conservative - the simple standard error approach is over-confident in the case of small sample size.
Comments:
This is an interesting topic and extends recent developments in probabilistic numerics to the problem of estimating averages over simulations from a model with uncertain parameters. I'm not aware of other principled approaches to solving this problem and I think it is of interest to people at NIPS.
The paper is well written and easy to follow. A lot of technical detail is in the supplementary so really this is a pretty substantial paper if one considers it all together. Putting technical details in the supplement improves the readability.
There is a proof of consistency of the approach in a restricted setting which gives some rigorous foundations to the approach.
The matching of the mixture model components and kernel into conjugate pairs (e.g. Gaussian mixture with squared exponential kernel) seems like a neat trick to make things tractable.
It is presented as an advantage that the method is conservative, e.g. the broad posterior over the quantity of interest is more likely to contain the ground truth. However, I was not sure what was the reason for the conservative behaviour and how general that behaviour is - is it generally the case that the method will be conservative or could there be cases where the procedure also underestimates the uncertainties of interest? Is this simply an empirical observation based on a restricted set of examples? I guess this is difficult to address theoretically as it is not an asymptotic issue.