Parameter Learning for Log-supermodular Distributions
We consider log-supermodular models on binary variables, which are probabilistic models with negative log-densities which are submodular. These models provide probabilistic interpretations of common combinatorial optimization tasks such as image segmentation. In this paper, we focus primarily on parameter estimation in the models from known upper-bounds on the intractable log-partition function. We show that the bound based on separable optimization on the base polytope of the submodular function is always inferior to a bound based on "perturb-and-MAP" ideas. Then, to learn parameters, given that our approximation of the log-partition function is an expectation (over our own randomization), we use a stochastic subgradient technique to maximize a lower-bound on the log-likelihood. This can also be extended to conditional maximum likelihood. We illustrate our new results in a set of experiments in binary image denoising, where we highlight the flexibility of a probabilistic model to learn with missing data.

This work considers the class of probabilistic models whose log-densities are supermodular.  For such models, [5] and [9] give upper-bounds "L-Field" and "Logistic", respectively, on the intractable log-partition normalizer.  This paper first shows that Logistic is always a tighter upper bound than L-Field. It then goes on to show that L-Field is also not useful for parameter learning (gives degenerate solution setting all parameters to 0), but that Logistic gives an expression for log-likelihood that can be optimized via projected stochastic gradient.  Finally, it shows that for the problem of image de-noising, even with just the noisy images available, parameter learning is possible.
Technical quality:    The math seems correct, though in general showing a few more intermediate   steps in derivations would make the work of the reader easier (can push the   proof to the supplement to make room).  In the proof of Proposition 1 for   instance, it would be nice to have more detail on how the final equality is   obtained.    The experiments compare to an SVM baseline, but not to the parameter learning   done in [9].  Specifically, [9] does parameter learning for the spin glass   model and shows that it achieves an error of 1.8% compared to an SVM's 8.2%.   What is different about the learning done in this work (besides the use of a   different probabilistic model)?  Novelty/originality: Section 4.2 is the main contribution of this work, but it does not seem like a very large delta on the work of [9].  Plugging the A_{Logistic} approximation for the partition function into the log likelihood directly yields the learning algorithm.  Potential impact or usefulness: The ability to estimate model parameters even without the original (non-noisy) images is useful.  However, experiments on more realistic data (a variety of more complex images for instance, rather than a single horse silhouette) would be better evidence of this usefulness.  Clarity and presentation: The paper is generally well-written, though, as mentioned above, a few more intermediate steps in some of the derivations would be helpful.  Additional minor notes below.  Minor notes:  Figure 1: It's hard to see the superdifferential lower bound line at all here. Maybe state in the figure caption that it is entirely covered by the logistic upper bound's line.  Experiments: The paper states "using mean marginals (which is optimal given our loss measure) lead to slightly better performance".  This is true or the supervised results (Table 1), but the opposite is true for the unsupervised results (Table 2).  Is there a logical explanation for why max-marginals does better in the unsupervised setting?  Line 92: "The method exploits" -> "This method exploits"  Line 115: "log Z" -> "log Z(f)"  Line 270: "allowing to benefit" -> "allowing us to benefit"


Identification and Overidentification of Linear Structural Equation Models
In this paper, we address the problems of identifying linear structural equation models and discovering the constraints they imply. We first extend the half-trek criterion to cover a broader class of models and apply our extension to finding testable constraints implied by the model. We then show that any semi-Markovian linear model can be recursively decomposed into simpler sub-models, resulting in improved identification and constraint discovery power. Finally, we show that, unlike the existing methods developed for linear models, the resulting method subsumes the identification and constraint discovery algorithms for non-parametric models.

This paper establishes a generalization of the half-trek criterion (HTC) for identifying coefficients in linear structural equation models (LSEMs). The generalized criterion (g-HTC) is designed to identify the coefficients associated with a subset of edges into a variable when the set of all edges into that variable is not identified by the original HTC. It turns out, moreover, that there are models that are identifiable by the g-HTC but not by the HTC. In connection to the g-HTC, this paper also proposes a way to derive constraints implied by a linear structural equation model. Furthermore, it is shown that when combined with decomposition of semi-Markovian models into sub-models over its c-components and marginalization of descendant sets, the proposed method of identification can identify every coefficient in a semi-Markovian LSEM that is identifiable according to non-parametric criteria (such as the do-calculus), and the method of deriving constraints subsumes some non-parametric methods in the literature.
This is a solid contribution to the literature on the identification of linear models. The paper is well organized and for the most part well written. Due to limited space, the technical details (even including the supplementary material) are unfortunately not fully self-contained and not easy to follow for non-experts. It would have been more readable if more background (such as the original HTC and more information on the proof of the theorem on HTC-identifiability) had been included in the supplementary material. Still, most proofs look credible (though I did not have time to check all of them carefully; sorry about that.) I am puzzled, however, by the last part of the proof of Theorem 4; I would appreciate some elaboration of how HT constraints express the conditional independence constraints in question.


Stochastic Gradient MCMC with Stale Gradients
Stochastic gradient MCMC (SG-MCMC) has played an important role in largescale Bayesian learning, with well-developed theoretical convergence properties. In such applications of SG-MCMC, it is becoming increasingly popular to employ distributed systems, where stochastic gradients are computed based on some outdated parameters, yielding what are termed stale gradients. While stale gradients could be directly used in SG-MCMC, their impact on convergence properties has not been well studied. In this paper we develop theory to show that while the bias and MSE of an SG-MCMC algorithm depend on the staleness of stochastic gradients, its estimation variance (relative to the expected estimate, based on a prescribed number of samples) is independent of it. In a simple Bayesian distributed system with SG-MCMC, where stale gradients are computed asynchronously by a set of workers, our theory indicates a linear speedup on the decrease of estimation variance w.r.t. the number of workers. Experiments on synthetic data and deep neural networks validate our theory, demonstrating the effectiveness and scalability of SG-MCMC with stale gradients.

This paper presents theoretical analysis for Stochastic-Gradient MCMC methods using stale (outdated) gradients, which often appear in asynchronous settings. In a nutshell, the paper extends the results of Chen, Ding, Carin, NIPS 2015 to the stale gradient settings. The bias, MSE, and the variance of the SG-MCMC estimator is rigorously analyzed under certain and rather standard assumptions. It has been shown that the bias and the MSE of the estimator depend on the "staleness" of the gradients whereas the variance does not. These results indicate that in a distributed setting where multiple chains are run in parallel, the SG-MCMC estimator would be able to attain lower MSE even if stale gradients are being used. The theory is supported by several experiments (both synthetic and real-data).
Overall, in my opinion this is an excellent paper. The paper is very well written, the objectives and the contributions are stated very clearly. I really enjoyed reading the paper.   Even though SG-MCMC methods have enabled scaling up MCMC to very large-scale settings, distributed and/or asynchronous variants of SG-MCMC have not been rigorously explored yet. In this sense, this paper has very important contributions and would be of great interest to the community. The results of the paper, in a sense, demystify asynchronous SG-MCMC, where for the first time, we obtain a clear understanding of the behaviors of the bias, variance, and the MSE of the SG-MCMC estimator. Besides, certain trade-offs become theoretically more grounded, such as the one explained in line 291 (page 7). Another take-home message is that using an higher-order integrator would not be useful in the stale-gradient settings.   The results presented in the paper can also be used for developing and analyzing new distributed/asynchronous SG-MCMC methods, or even new stochastic optimization methods based on simulated annealing.    I only have some minor comments/corrections.  -- Main Document --   * Line 13: I think it is better to write "with respect to" instead of "w.r.t." since there is enough space.   * It is quite trivial but it might be good to remind the reader about MSE = Bias^2 + Var. I would suggest the authors to check Section D of the supplementary document of (Li et al. 2016). ([28] in the paper)  * Line 168: "M" should be "S".  * Line 241: "indicates" should be "indicated"  * Line 249: I would prefer saying "we average" since all the rest of the paragraph is in the present tense.   * Theorem 6: Should the step length be "h_m" instead of "h"?  * Line 291: I am not very happy with the usage of PSGLD as the baseline SG-MCMC method in the deep learning experiments. Due to the negligence of the "correction term" in PSGLD, the bounds given in Lemma 1 no longer hold for PSGLD: there are additional terms appearing in the bounds of both bias and MSE due to this negligence. Therefore, the theory presented in this paper is not completely appropriate for PSGLD. I would suggest the authors to use another SG-MCMC method which would obey the bounds given in Lemma 1. If this is not possible I would recommend providing a discussion/explanation on this subject.    * Lines 322-325: The full list of authors should be given in [7,8,9].  -- Supplementary Document --   * Line 427, Line 448, Line 455: $X$ should be $x$ (lower-case).   * The first equation after line 458: There should not be an absolute value operator on the right hand side. If there is, then it should be \leq (after applying a triangle inequality)  * The last equation after line 458: I do not really understand how ||E \nabla \psi|| becomes ||\nabla \psi||. \nabla \psi is not deterministic, how come it is equal to its expectation?  * Eq 19: (and the equation after that) h should be h_s. Eq 20: h should be h_m. The same problem appears in the equations after Line 511. This should be also corrected in all the relevant parts of the main text.


A Unified Approach for Learning the Parameters of Sum-Product Networks
We present a unified approach for learning the parameters of Sum-Product networks (SPNs). We prove that any complete and decomposable SPN is equivalent to a mixture of trees where each tree corresponds to a product of univariate distributions. Based on the mixture model perspective, we characterize the objective function when learning SPNs based on the maximum likelihood estimation (MLE) principle and show that the optimization problem can be formulated as a signomial program. We construct two parameter learning algorithms for SPNs by using sequential monomial approximations (SMA) and the concave-convex procedure (CCCP), respectively. The two proposed methods naturally admit multiplicative updates, hence effectively avoiding the projection operation. With the help of the unified framework, we also show that, in the case of SPNs, CCCP leads to the same algorithm as Expectation Maximization (EM) despite the fact that they are different in general.

This paper derives methods to train SPNs starting from a perspective very different from methods established in literature. SPN training is viewed as a signomial program, and different approximate solutions to this problem are proposed. A theoretical and experimental analysis shows that the best method, called concave-convex procedure (CCCP), is equivalent to the already established Expectation Maximization for SPNs.
The single contribution of the paper which is relevant in practice is an alternative derivation of an existing method (Expectation Maximization for learning SPN weights). While this is an interesting result, I think that it does not grant alone a publication in NIPS since it's hard to imagine how this can contribute to better theoretical understanding or practical applications of SPNs.   The interpretation of SPNs as mixtures of tree structured SPNs, which is reported as a novelty by the authors, was actually first derived in  [Dennis and Vantura, Greedy Structure Search for Sum-Product Networks, 2015].  The paper is overall well written, clearly structured and the derivation of the results is really interesting. My main concern, as detailed above, is that in my opinion the potential impact of this paper is low, and the novelty is also somewhat limited due to the fact that the interpretation of SPN as mixture of trees was already given in [Dennis and Vantura, Greedy Structure Search for Sum-Product Networks, 2015] and that this is basically just an alternative derivation of EM. So, I think the paper is (in the current form) a bit under the threshold for NIPS.    The experiment section is also not surprising, since basically it takes a SPN which is not trained, but in which weights are initialized with heuristic methods, and trains them with EM - which naturally improves a bit the log likelihood.   The title "unified approach" is misleading, since not all learning methods for SPNs are covered by this approach - in particular, the hard-gradient and hardEM methods that have been used successfully in the past (see [Discriminative Learning of Sum-Product Networks, Gens and Domingos, NIPS 2012]).   Small remark - in line 311: it is not always true that size is smaller in LearnSPN for 20News and AD (see line 574).  Overall, an interesting paper in terms of original derivation of results which have potential for future development, but the results are not novel enough, in my opinion, to grant publication at NIPS.


Tracking the Best Expert in Non-stationary Stochastic Environments
We study the dynamic regret of multi-armed bandit and experts problem in nonstationary stochastic environments. We introduce a new parameter Λ, which measures the total statistical variance of the loss distributions over T rounds of the process, and study how this amount affects the regret. We investigate the interaction between Λ and Γ, which counts the number of times the distributions change, as well as Λ and V , which measures how far the distributions deviates over time. One striking result we find is that even when Γ, V , and Λ are all restricted to constant, the regret lower bound in the bandit setting still grows with T . The other highlight is that in the full-information setting, a constant regret becomes achievable with constant Γ and Λ, as it can be made independent of T , while with constant V and Λ, the regret still has a T 1/3 dependency. We not only propose algorithms with upper bound guarantee, but prove their matching lower bounds as well.

The paper considers online learning in a non-stationary stochastic environment, and considers the achievable regret as a function of the change in the mean loss vector (measured by the number of switches or the amount of shift) and the cumulative variance of the random losses. Algorithms are given with regret upper bounds, as well as matching lower bounds.
ADDED AFTER THE REBUTTAL:  - Indeed, the proof of Theorem 4.1 is correct, and the constructed environment is oblivious; it was my mistake, sorry about it. - The generalization in the analysis of UCB-V seems quite straightforward, as--if I recall correctly--the applied concentration inequalities do not require that the random variables have the same distribution (neither in your nor in their case). - Comparison to Jadbabaie et al: If in their work one chooses M to be the previous gradient, then their D_T is essentially the same as your Lambda (up to some constants), and the difference is that you take expectation inside the norms in V. While I might be mistaken here, but it seems to me that with a slight modification of their proof you can move the expectation inside. Nevertheless, you have to check this, and describe the connection to related work with care (what follows easily and what not). - You should still argue why the proposed complexity parameters are interesting, e.g., present an example when Lambda is not zero or linear in T.   -----------------------------------------  While the paper might have some interesting contributions, it does not consider the existing literature and does not cite the references adequately. This, in fact, makes it hard to asses the novelty of the results, as several of them seem to easily follow from existing work. In fact, the paper of Jadbabaie, Rakhlin, Shahrampour, and Sridharan, whose arxiv version is "sort of" cited in the paper (included in the references but never discussed properly), which was published in AISTATS 2015, provides an algorithm quite similar to Algorithm 2, which achieves similar bounds, but in the stronger adversarial case.   The main addition of the paper to admitted existing work (Besbes et al., 2014--please correct the order of the authors in the reference) is to consider the variance of the random losses. As far as I can see, this is achieved by combining the techniques to derive second order bounds and some sort of shifting bounds (even blocking, in the simplest case). This would be interesting if it had not been done before by Jadbabaie et al. (2015). However, given the previous work, a thorough discussion of novelty would be required. Also, the writing of the paper has to be improved dramatically: e.g., not even the UCB-V algorithm of Audibert et al. (2009) is referenced (although used almost identically).    Concerning motivation, other than saying that the complexity measures used in the paper appear in the literature, why are they interesting? For example, if there is a clearly suboptimal arm, why does it matter if its mean changes or it has a relatively high variance (assume that even the smallest loss from this arm is bigger than the losses of any other arm)? What are the interesting cases when \Lambda is not constant or linear in time? Also, what is the connection to tracking/shifting bounds (Herbster and Warmuth 1998, 2001)? Note that in the standard online learning literature, tracking and shifting bounds usually correspond to properties of the comparator sequence, not of the loss functions, so this should be clarified to avoid confusion. The switching results should also relate to the large body of work done in information theory on coding piecewise stationary sources (although there usually the log-loss is considered, but the techniques are essentially the same, although instead of the variance the the entropy/entropy-rate of the source is considered). The sleeping expert-type reductions are also available in generic forms, see e.g., Gyorgy et al. (2012) and the references therein.     All in all, I see potential in the paper, but substantial improvements are needed before it can be published.  References:  J-Y. Audibert, R. Munos, and Cs Szepesvari. Exploration-exploitation trade-off using variance estimates in multi-armed bandits. Theoretical Computer Science, 410:1876-1902, 2009.   A. Gyorgy, T. Linder, and G. Lugosi, "Efficient Tracking of Large Classes of Experts," IEEE Transactions on Information Theory, vol. 58, pp. 6709-6725, November 2012.  Herbster, M. and Warmuth, M.K. (1998) “Tracking the Best Expert”, in the special issue on context sensitivity and concept drift of the Journal of Machine Learning, Vol. 32(2), pp. 151-178, August 1998.  Herbster, M. and Warmuth, M.K. (2001) “Tracking the Best Linear Predictor”, Journal of Machine Learning Research, Vol. 1, pp. 281-309, September 2001.


Blind Regression: Nonparametric Regression for Latent Variable Models via Collaborative Filtering
We introduce the framework of blind regression motivated by matrix completion for recommendation systems: given m users, n movies, and a subset of user-movie ratings, the goal is to predict the unobserved user-movie ratings given the data, i.e., to complete the partially observed matrix. Following the framework of nonparametric statistics, we posit that user u and movie i have features x 1 (u) and x 2 (i) respectively, and their corresponding rating y(u, i) is a noisy measurement of f (x 1 (u), x 2 (i)) for some unknown function f . In contrast with classical regression, the features x = (x 1 (u), x 2 (i)) are not observed, making it challenging to apply standard regression methods to predict the unobserved ratings. Inspired by the classical Taylor's expansion for differentiable functions, we provide a prediction algorithm that is consistent for all Lipschitz functions. In fact, the analysis through our framework naturally leads to a variant of collaborative filtering, shedding insight into the widespread success of collaborative filtering in practice. Assuming each entry is sampled independently with probability at least max(m −1+δ , n −1/2+δ ) with δ > 0, we prove that the expected fraction of our estimates with error greater than is less than γ 2 / 2 plus a polynomially decaying term, where γ 2 is the variance of the additive entry-wise noise term. Experiments with the MovieLens and Netflix datasets suggest that our algorithm provides principled improvements over basic collaborative filtering and is competitive with matrix factorization methods.

This paper motivates a method for blind regression, using a nonparametric model for collaborative filtering within the domain of matrix completion. Using some weak assumptions of regularity and smoothness, the paper proposes a model whereby observed entries of the matrix are noisy observations of the output of an unknown Lipschitz function on features relating to a user and an item. In contrast to previous work on collaborative filtering, this paper considers the case where the features on the users and items are themselves latent. Using a multiple Taylor expansions, the paper presents an algorithm and provides a finite same error bound. Intuitively, the bound stems from the accuracy of local Taylor approximations under small amounts of noise and then makes use of the variance between two rows or columns of the unknown matrix as a bound on the squared error. This variance is estimated empirically on the known points. After the discussion of the algorithm and its error bound, the paper extends this matrix completion method to tensor completion in a setting with equivalent assumptions for tensors. This is achieved by partitioning the tensor into two disjoint subsets and flattening in the tensor in each of those subsets to create the rows and columns of a new matrix which satisfies all of the assumptions needed for their earlier blind regression algorithm for matrices. The paper closes by providing some experiments performed on the MovieLens and Netflix datasets, where the algorithm is shown to give competitive results compared to other collaborative filtering and matrix completion methods.
Explanation of ratings:  Technical Aspect: I thought this paper did a good job of motivating a different approach to user-user or item-item collaborative filtering. It is impressive that equivalent performance (empirically) is achievable with far fewer assumptions on the data, and this is definitely something for practitioners to consider in their research. While less is assumed in the model, less is explained however. A more detailed analysis of the algorithm may benefit this paper.   Novelty: This paper takes a very different, and non-restrictive approach to collaborative filtering and shows good results with it. By using successive Taylor approximations on their latent function, however, it seemed like the paper spent time motivating the idea of a latent function and latent variables but then assumed low noise and removed it from the model. In other words, all of the latency of this model appears to get wrapped into local approximations and then the model appears to proceed by working directly on the observed entries and ignoring the latent features and function entirely. I do think that this idea of weighting based on variance is novel as are the extensions to tensor completion and I would like to see more discussion of the later.   Potential Impact: This paper has the potential for impact by motivating the idea of completing a matrix using the variance as a weighting scheme for selecting representative users and columns. Future theoretical work may well stem from this idea.   Clarity: The paper was mostly clear. I appreciated the section intuitively motivating the algorithm. The section motivating the extension to tensors could be clearer, particularly in the decomposition of the tensor into disjoint sets. It is unclear to me how a matrix is formed from that and how this maps back to the original tensor after the matrix is “flattened.” Furthermore, the algorithm is not described as clearly as it could have been. I think that the steps of the algorithm are somewhat lost in their definition.  Suggestions for revision: Clarify your algorithm section. It is not immediately obvious what you are trying to learn, and how you go about doing so. I understand the goal is to make predictions on unknown entries, but it is not clear where the actual regression is happening. For instance, in the user-user or item-item case, is the regression simply a search problem to find the corresponding rows and columns with minimum variance and weight them accordingly?  Clarify where blind regression plays a role in this paper. It seems as though you motivate the concept of blind regression, but then show that it reduces to a weighting scheme on the variance of the difference between rows and columns.  If there is room, a brief mention of the intuition behind your theorem would be useful. This paper begins to address this in the last paragraph of section 5, but given the length of the full proof, some added intuition would be useful here.   In your abstract, you mentioned the algorithm shedding light on the success of collaborative filtering in practice. I am not sure that this question was fully answered. Perhaps you could add a couple of sentences in the intuition or contribution sections to more fully explain how your nearest-neighbor approach generalizes and explains the success of other, more restrictive models collaborative filtering.


Combinatorial Energy Learning for Image Segmentation
We introduce a new machine learning approach for image segmentation that uses a neural network to model the conditional energy of a segmentation given an image. Our approach, combinatorial energy learning for image segmentation (CELIS) places a particular emphasis on modeling the inherent combinatorial nature of dense image segmentation problems. We propose efficient algorithms for learning deep neural networks to model the energy function, and for local optimization of this energy in the space of supervoxel agglomerations. We extensively evaluate our method on a publicly available 3-D microscopy dataset with 25 billion voxels of ground truth data. On an 11 billion voxel test set, we find that our method improves volumetric reconstruction accuracy by more than 20% as compared to two state-of-the-art baseline methods: graph-based segmentation of the output of a 3-D convolutional neural network trained to predict boundaries, as well as a random forest classifier trained to agglomerate supervoxels that were generated by a 3-D convolutional neural network.

This paper proposed a novel method to handle the large-scale segmentation problem in mapping neuroanatomy. The proposed method uses deep neural network to evaluate local energy given a segmentation configuration, as well as an energy minimization approach to greedily guide the (super)voxel agglomeration. To handle the inherent large scale of the targeted problem, the authors also consider the issue of pruning from several aspects.
The paper is well-written, with lots of details provided. The studied problem is definitely of great impact and would significantly benefit the community of neuroscience. The reviewer noticed the large size of the problem and appreciates the considerable amount of works achieved, as described by the paper.  On the other hand, I also want to point out some associated weakness:  1. To my understanding, two aspects which are the keys to the segmentation performance are: (1) The local DNN evaluation of shape descriptors in terms of energy, and (2) The back-end guidance of (super)voxel agglomeration. Although experiment showed gains of the proposed method over GALA, it is yet not clear enough which part is the major contributor of such gain. The original paper of GALA used methods different from this paper (3D-CNN) to generate edge probability maps. Is the edge map extraction framework in this paper + GALA a fair enough baseline? It would be great if such edge map can also be visualized.  2. The proposed method to some extent is not that novel. The idea of generating or evaluating segmentation masks with DNN has been well studied by the vision community in many general image segmentation tasks. In addition, the idea of greedily guiding the agglomeration of (super)voxels by evaluating energies pretty much resembles many bottom-up graph-theoretic merging in early segmentation methods. The authors, however, failed to mention and compare with many related works from the general vision community. Although the problem general image segmentation somewhat differs from the task of this paper, showing certain results on general image segmentation datasets (like the GALA paper) and comparing with state-of-the-art general segmentation methods may give a better view of the performance of the proposed method.  3. Certain parts did not provide enough explanation, or are flooded by too much details and fail to give the big picture clearly enough. For example in Appendix B Definition 2, when explaining the relationship between shape descriptor and connectivity region, some graphical illustrations would have helped the readers understanding the ideas in the paper much easier and better.  ----------------------------------------------------------------------------- Additional Comments after Discussion  Upon carefully reading the rebuttal and further reviewing some of the related literature, I decide to down-grade scores on novelty and impact. Here are some of the reasons:  1. I understand this paper targets a problem which somewhat differs from general segmentation problems. And I do very much appreciate its potential benefit to the neuroscience community. This is indeed a plus for the paper. However, an important question is how much this paper can really improve over the existing solutions. Therefore, to demonstrate that the algorithm is able to correctly find closed contours, and really show stronger robustness against weak boundaries (This is especially important for bottom up methods), the authors do need to refer to more recent trends in the vision community.  2. I noticed one reference "Maximin affinity learning of image segmentation, NIPS 2009" cited in this paper. The paper proposed a very elegant solution to affinity learning. The core ideas proposed in this paper, such as greedy merging, Rand Index like energy function show strong connections to the cited paper. The maximin affinity is basically the weakest edge along a minimum spanning tree (MST), and we know greedy region merging is also based on cutting weakest edges on a MST. The slight difference is the author proposed the energy term at a lot of positions and scales but the previous paper has a single energy term for the global image. In addition cited paper also addressed a similar problem in the experiment. However, the authors not only did not include the citation as an experimental baseline, but also failed to provide detailed discussions on the relation between the two works.  3. The authors argued for greedy strategy, claiming this is better than factorizable energies. "By sacrificing factorization, we are able to achieve the rich combinatorial modeling provided by the proposed 3d shape descriptors." I guess what the authors mean is they are putting more emphasis on local predictions. But this statement is not solidly justified by the paper. In addition, although to some extent I could understand this argument (local prediction indeed seems important because the size of cells vs volume are much smaller than segments vs whole image in general segmentation), there are significant advances on making pretty strong local predictions from the general vision community using deep learning, which the authors fail to mention and compare.  I think overall the paper addressed an interesting problem and indeed showed solid research works. However it will better if the authors could better address contemporary segmentation literature and further improve the experiments.


Wasserstein Training of Restricted Boltzmann Machines
Boltzmann machines are able to learn highly complex, multimodal, structured and multiscale real-world data distributions. Parameters of the model are usually learned by minimizing the Kullback-Leibler (KL) divergence from training samples to the learned model. We propose in this work a novel approach for Boltzmann machine training which assumes that a meaningful metric between observations is known. This metric between observations can then be used to define the Wasserstein distance between the distribution induced by the Boltzmann machine on the one hand, and that given by the training sample on the other hand. We derive a gradient of that distance with respect to the model parameters. Minimization of this new objective leads to generative models with different statistical properties. We demonstrate their practical potential on data completion and denoising, for which the metric between observations plays a crucial role.

The paper proposes training RBMs using the gamma-smoothed Wasserstein distance W_{gamma}(\hat{p}, p_{theta}) (eqs 3 and 4), where  \hat{p} is the data distribution and p_{theta} is the model.  They  obtain derivatives of W_{gamma}(\hat{p}, p_{theta}) and train  with gradient descent.  However, the shrinkage of the Gaussian example in sec 4.3 (and the results in sec 4.2) gives serious concern wrt the consistency of Wasserstein training and this issue MUST be investigated more carefully before the paper is suitable for publication.
The paper refers to [2] and says that those authors proved statistical consistency. However, I am then surprised to see in section 4.3 that non-zero shrinkage is obtained (including for gamma=0) for the  very simple case of modelling a N(0,I) distribution with  N(0, sigma^2 I). What is going on here?? A failure of consistency  would be a serious flaw in the formulation of a statistical learning criterion.  Also in sec 3 (Stability and KL regularization) the authors say that at least for learning based on samples (\hat{p}_{theta}) that some regularization wrt the KL divergence is required. This clearly weakens the "purity" of the smoothed Wasserstein objective fn.  Experiments (sec 4) are carried out on MNIST-small (the 0s from MNIST), a subset of the UCI PLANTS dataset, and 28-d binary codes (MNIST code). The results (as reported in lines 168-172 and Figs 3 and 4) seem to produce "compact and contiguous regions that are prototypical  of real spread, but are less diverse than the data". This reinforces  my belief above (relating to the Gaussian example insec 4.3) that  minimizing the (smoothed) Wasserstein does not lead to a consistent density estimator.  In sec 4.4 the authors try to "spin" these weakness into strenghts wrt  data completion or denoising.   Overall: The authors are to be congratulated on investigating an interesting alternative to standard KL-based training of RBMs. However, the shrinkage of the Gaussian example in sec 4.3 gives serious concern wrt the consistency of Wasserstein training and this issue MUST be investigated more carefully before the paper is suitable for  publication.  Other points:  Title: the title over claims. The paper only discusses training *restricted* BMs.  Intro l 7: RBMs do not need to be restricted to binary x's, see e.g. Exponential Family Harmoniums  https://papers.nips.cc/paper/2672-exponential-family-harmoniums-with-an-application-to-information-retrieval.pdf  Fig 5 would be much more readable using different colors for the different gammas. Also last entry should be gamma=0 (not OpenCV). You can explain in the text that you used OpenCV to compute this.   === COMMENTS POST REBUTTAL =====  I have read the other reviewers comments and the rebuttal.  I will remove my charge of a "fatal flaw" based on the point that the sample size in the experiments in sec 4.3 is small. However, the KL version of the experiment (where we fit a Gaussian with cov matrix theta^2 I to n = 100 samples drawn from a 10-d Gaussian) the values of theta I get from the max likelihood estimator are all pretty much in the range 0.95-1.05 (for about 10 repetitions). This should be compared  with something around theta = 0.65 (estimated by eye from Fig 5(left))  for the Wasserstein (gamma=0) estimator.  The convergence rate given by the authors of O(n^-1/(10+1)) for  this simple problem is truly appalling, and presumably is  in general O(n^-1/(D+1)) for D dimensions. This gives rise to  serious concerns about the practicality of such a method.   It would be helpful to see a study in this simple Gaussian case, showing e.g. how the theta that minimizes W_0 varies with n, although the rate given above suggests convergence will be  painfully slow. Can you understand in such a case why there is  a shrinkage bias?   I note the response to R5 wrt lines 177-178 i.e.  "the entropy regularizer contributes to the clustering effect [...]. It also appears for the true W distance in Fig 5."  One could use say a KL matching criterion KL(\hat{p}||p_theta) along with an entropy penalty (no Wasserstein), and my understanding from the quote above is that this will cause clumping, whose virtues are extolled in sec 4.4 wrt data completion and denoising.  (Note that due to the KL regularization in section 3 this is partly happening anyway in many of the experiments).  I also agree with other reviwers who found the clarity of technical explanation poor, and suggested use of supp mat to help with this.  Overall, I am not really in favour of accepting this paper (although I could live with acceptance). There is quite a lot of interesting work, but the issue where the clumping comes from (Wasserstein or entropic prior) needs to be better articulated and dissected. I believe some of claims wrt Wasserstein training are actually to do with the entropic prior, and this should be teased out by comparing against a KL + entropic prior construction.


RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism
Accuracy and interpretability are two dominant features of successful predictive models. Typically, a choice must be made in favor of complex black box models such as recurrent neural networks (RNN) for accuracy versus less accurate but more interpretable traditional models such as logistic regression. This tradeoff poses challenges in medicine where both accuracy and interpretability are important. We addressed this challenge by developing the REverse Time AttentIoN model (RETAIN) for application to Electronic Health Records (EHR) data. RETAIN achieves high accuracy while remaining clinically interpretable and is based on a two-level neural attention model that detects influential past visits and significant clinical variables within those visits (e.g. key diagnoses). RETAIN mimics physician practice by attending the EHR data in a reverse time order so that recent clinical visits are likely to receive higher attention. RETAIN was tested on a large health system EHR dataset with 14 million visits completed by 263K patients over an 8 year period and demonstrated predictive accuracy and computational scalability comparable to state-of-the-art methods such as RNN, and ease of interpretability comparable to traditional models.

This paper aims to provide more interpretable models from clinical time series data by relying more on recent time points than distant ones for making predictions. Predictive accuracy is compared to the current state of the art on a set of clinical data.
The problem addressed (model interpretability) is an important one, and certainly a model that a clinician can understand would be preferable to a black-box model with similar accuracy. The assumption that focusing on recent timepoints may help create such models is reasonable on the face of it, but is not well justified in the paper (theoretically or empirically), and it is unclear whether the approach would generalize to any other problems. The lack of confidence in the results and lack of clear application to any other problem limited my enthusiasm. Fundamentally, the paper demonstrates that their approach has similar predictive accuracy to the state of the art on one dataset, but fails to demonstrate that it has any advantages that make it superior despite the similar accuracy.  Key questions that need to be addressed: 1) Handling of irregularly sampled time series: It seems the authors have to assume the same sampling rate for all patients (as on pg 2 they refer to the i and i+1th visit as a time step). However, visits are irregularly spaced and will certainly occur at different frequencies for different patients. How does this affect the algorithm? Similarly, there may be a cluster of frequent visits during an acute illness, but these will likely have redundant information and should be collapsed into a single episode.   2) Why is the assumption of more recent events being most important justified? The entire paper rests on this assumption, but it is not obvious that this should be the case. For example, duration of diabetes is often an important variable for predicting complications, but there we need to understand the onset, not the most recent mention of diabetes in a medical record.  3) A lot of detail on the experiments is missing. It's not clear where the data come from or how this definition of heart failure was created or validated. Further many decisions were made in assigning diagnosis dates (and dates for controls) that are never justified.    4) The evaluation of interpretability contains only anecdotes related to the single experiment. Were any tests done to determine if the proposed approach leads to predictions that are more interpretable by humans? I would think that should be a key evaluation for this paper.  Most concerning, though, is that it's stated that "RETAIN properly puts much attention to the HF-related codes that occurred in recent visits." According to the method for finding HF cases, it seems that in most cases the HF codes should be at or after the dx-date.  It seems that the time series was not appropriately segmented, so that data after diagnosis is being used for the prediction task. Finding that HF codes predict heart failure is not a clinically meaningful model.   5) How well does this method do on other datasets? In particular, are there any experiments on data where ground truth is known? EHRs are very noisy and heart failure diagnosis can be ambiguous in these data. The use of only a single experiment in this case is less convincing.


Geometric Dirichlet Means algorithm for topic inference
We propose a geometric algorithm for topic learning and inference that is built on the convex geometry of topics arising from the Latent Dirichlet Allocation (LDA) model and its nonparametric extensions. To this end we study the optimization of a geometric loss function, which is a surrogate to the LDA's likelihood. Our method involves a fast optimization based weighted clustering procedure augmented with geometric corrections, which overcomes the computational and statistical inefficiencies encountered by other techniques based on Gibbs sampling and variational inference, while achieving the accuracy comparable to that of a Gibbs sampler. The topic estimates produced by our method are shown to be statistically consistent under some conditions. The algorithm is evaluated with extensive experiments on simulated and real data.

The paper proposes a geometric view to topic modeling and develops an inference algorithm based on it. This is an interesting point of view and an inference method. The authors provide theoretical guarantees of their method proving a theorem about convergence of their topic estimates to the true ones.  Topic modeling is considered as a geometric problem, where there is a convex hull of topics and documents, represented as vectors of words probabilities, are points inside this polytope. A topics proportions vector for a document can be then represented as barycentric coordinates of this document w.r.t. this polytope.  The learning procedure consists on two steps: finding centroids of K-means clusters, where K is the number of topics, first, and then extending these centroids to the vertices of the polytope along rays that connect centroids and the barycenter of the polytope. Inference of topic proportions for documents is based on projection of documents points, represented as vectors of normalised words frequencies, to this found polytope.    The algorithm is shown to be fast and accurate in terms of hold-out perplexity and minimum-matching Euclidean distance between true and estimated topics. Moreover, it seems to be possible to extend this approach for more complex topic models. The authors present the extension for the case where the number of topics is unknown and discuss possibility of extensions to correlated and dynamic topic models.
The paper reads well, despite the fact that a lot of information is presented in a dense form. The authors refer everywhere to related work.  Although the paper is presented well, there are ways to improve it. There are two most crucial aspects which are currently not very well covered in the paper, everything else is not critical, they are some suggestions for improvement.  These two crucial moments are descriptions of tuning of extension parameters in the proposed tGDM algorithm and the first step in the proposed extension of the algorithm called as the nGDM algorithm. tGDM algorithm: The authors claim that extension of the centroids of the k-means clusters towards vertices of the polytope by the fixed length (as it is in the basic proposed algorithm called as the GDM algorithm) is not always a good choice. They mention that "careful tuning of the extension parameters based on optimizing the geometric objective over a small range of m_k helps to improve the performance considerably". And they call this tGDM algorithm. It is not clear for me how exactly this tuning happens. It would be good to add some more details about it. nGDM algorithm: The authors present an extension of their approach to the case when the number of topics is unknown. They say that the first step of their algorithm "involves solving a penalized and weighted k-means clustering". It is unclear how it can be done. If it refers to some known method it should be cited, otherwise it would be also better to add some more details about this step.  Other suggestions:  1. As the material is dense it would be good to summarise the whole algorithm with specific implementations of the steps    2. It looks strange that in sections 3.1 and 3.2 thetas are treated as parameters and then in section 3.3 they transform to be random variables. It is better, at least, to mention about this transformation  3. To continue the previous remark the likelihood in section 3.1 is for the PLSA model, rather than for the LDA one (because of treatment thetas as parameters rather than random variables). It would be better to state this and to clarify about LDA likelihood as the claim about connection between the geometric objective function and LDA likelihood appears several times in the paper 4. The claim "V is much higher and the cluster centroids are often on the boundary of the vocabulary simplex, therefore we have to threshold the betas at 0" (lines 146-148) requires some explanations 5. It would be better to add some citation to the claim in the sentence in lines 67-68 6. Regarding experiments. In the paper it is claimed that the results of the variational EM algorithm are always worse than the others but according to the results in figure 2 in the supplementary materials (where there are results for individual runs of the algorithms rather than averaged ones presented in the paper) for only 3 out of 20 different runs of the variational EM algorithm its results are worse than the results of Gibbs sampling and these results of both algorithms coincide in other 17 runs. It looks not so bad. Maybe averaging procedure of the results of the individual runs of the algorithms is not appropriate. At least it is worth to mention about this behaviour. It is mentioned in the paper that variational EM is unstable. Even if it regards to this behaviour it is better to explain it in more details 7. Information about Gibbs sampling and the variational EM algorithm settings in the experiments on simulated data would be good 8. In Figure 2 there are results referred as 0.1(log(N_m)/N_m)^0.5. There is no information about this in the text and it is unclear what it is about 9. The claim that tuning extension parameters could be computationally expensive on big datasets (lines 248-249) looks strange as there was no possibility to check it 10. It is not very clear why there are no results for Gibbs sampling and the variational EM algorithm in figures 2d and 3d 11. Regarding results presented in the supplementary materials. It is not clear for me what is weighted GDM. As I understand GDM presented in the paper is already weighted 12. It is not clear what the reference "projection with VEM" in figure 3 in the supplementary materials refers to 13. It is not clear why there is no presented topics inferred by the variational EM algorithm in the supplementary materials  Minor things: 1. Please check style requirements, e.g. a paper title should not be capitalised 2. Acronyms GDM, VEM and MLE are undefined 3. It would be better to define the notation "Dir_V" 4. It would be better to distinguish scalars and vectors in notations 5. There is "for m = 1, ..., M" in line 58 but there is not "for k = 1, ..., K" in line 57 6. "z", "d" and "n_m" are never defined 7. B := ... in line 77 rather than B = ... 8. "lets" in line 80 seems to be "let" 9. Missing articles before "document" in lines 83 and 84 10. Please enumerate all equations 11. In equation between 107 and 108 V is used as a set, while it is introduced as a scalar 12. Missing article before "topic polytope" in line 133 13. "Generative" rather than "generating" in line 142 14. It seems that there is only one tetrahedron in the simple simulation experiment (lines 140-146), then "tetrahedra" in line 145 should be "tetrahedron" 15. It is better to add subcaptures a, b, c and d in figures 2 - 4 as they are referred. 16. Missing closing bracket in line 243 17. It is better to add citation to the used NIPS corpora 18. Please checked references: there are lower cases for proper names, e.g. "Hierarchical dirichlet processes"; there are also caps for articles, e.g. in "Proceedings of The 31st International ..."; papers from NIPS are referred as "NIPS, 2012" in line 283 and "Advances in neural information processing systems, 2006a" in line 286; authors' given names appear both in full forms and as initials; a place of a conference is stated only for one reference in line 307


A Simple Practical Accelerated Method for Finite Sums
We describe a novel optimization method for finite sums (such as empirical risk minimization problems) building on the recently introduced SAGA method. Our method achieves an accelerated convergence rate on strongly convex smooth problems. Our method has only one parameter (a step size), and is radically simpler than other accelerated methods for finite sums. Additionally it can be applied when the terms are non-smooth, yielding a method applicable in many areas where operator splitting methods would traditionally be applied.

This paper presents an accelerated variant of SAGA for finite sums. Accelerated algorithms are part of an important class of optimization methods which have a much better dependency to the condition number. There is recently been a lot of renewed interest in this class of methods and many papers have been published including accelerated SDCA or the Catalyst method. The authors here describe an accelerated version for SAGA in the regime when L/mu > n. They describe their method as “significantly simpler and requires less tuning than existing accelerated methods”.
Technical quality: To the best of my knowledge, the paper is technically sound and the proofs derived in the paper are correct and not hard to follow. There is not much intuition given as to why the modified step leads to an acceleration. I know accelerated methods are not always very intuitive but is there any intuition that could be gained from recent work on accelerated methods? e.g. the work of Allen-Zhu, Zeyuan, and Lorenzo Orecchia. "Linear coupling: An ultimate unification of gradient and mirror descent.", where the acceleration results from a primal-dual point of view on the progress of the algorithm.  Novelty/originality: The main problem with the paper is the novelty aspect of the work. Similar convergence rates for accelerated variance reduced SGD have already been derived in the catalyst paper as well as other references. One missing reference is Nitanda, Atsushi. "Accelerated Stochastic Gradient Descent for Minimizing Finite Sums." arXiv preprint arXiv:1506.03016 (2015). The authors claim that their method is “significantly simpler and requires less tuning than existing accelerated methods” but I think this point is quite controversial as one could claim that other methods also only have one parameter to tune. The authors acknowledge in the experimental section that “SDCA was chosen as the inner algorithm for the catalyst scheme as it doesn’t require a step-size, making it the most practical of the variants”. Furthermore, the only parameter described for the other baselines in the experimental section is the step-size (“The step-size parameter for each method (κ for catalyst-SDCA) was chosen using a grid search of powers of 2.“). I think this point is very ambiguous in the paper and needs to be rewritten with some compelling arguments as to why their method is simpler or this claim should be toned down. One positive contribution though is that the method proposed by the authors handles non-smooth loss which is not the case for most existing variance-reduced methods (expect SDCA). Could the authors elaborate on whether or not Theorem 7 guarantees a better convergence in comparison to SDCA?  Experimental section: In the experiments with the hinge loss (as a non-smooth objective), the convergence seems linear while their Theorem 7 provides a sub-linear convergence. Could the authors comment on a potential improvement for “nearly everywhere smooth loss functions” as done in the SDCA paper?  Minor: Lines 68-9: there is a typo where the sign of g_j^k is negative.


Learning in Games: Robustness of Fast Convergence
We show that learning algorithms satisfying a low approximate regret property experience fast convergence to approximate optimality in a large class of repeated games. Our property, which simply requires that each learner has small regret compared to a (1 + ✏)-multiplicative approximation to the best action in hindsight, is ubiquitous among learning algorithms; it is satisfied even by the vanilla Hedge forecaster. Our results improve upon recent work of Syrgkanis et al. [28] in a number of ways. We require only that players observe payoffs under other players' realized actions, as opposed to expected payoffs. We further show that convergence occurs with high probability, and show convergence under bandit feedback. Finally, we improve upon the speed of convergence by a factor of n, the number of players. Both the scope of settings and the class of algorithms for which our analysis provides fast convergence are considerably broader than in previous work. Our framework applies to dynamic population games via a low approximate regret property for shifting experts. Here we strengthen the results of Lykouris et al. [19] in two ways: We allow players to select learning algorithms from a larger class, which includes a minor variant of the basic Hedge algorithm, and we increase the maximum churn in players for which approximate optimality is achieved. In the bandit setting we present a new algorithm which provides a "small loss"-type bound with improved dependence on the number of actions in utility settings, and is both simple and efficient. This result may be of independent interest.

This paper introduces a property called low approximate regret for the expert algorithm that will ensure faster convergence in games compared to previous work. The authors show that this property can be satisfied by many common algorithms, including the vanilla Hedge algorithm. The feedback model allowed in this paper is also more natural and less restricted: the player only receives a realized loss vector depending on other players’ chosen actions, or even just the chosen coordinate of this loss vector (i.e. bandit feedback).  High probability bounds are also derived. Finally the authors also extend the results to dynamic population games and improve upon previous work by [LST16].
The results in this paper are interesting in general. However, from a technical viewpoint, the main results seem to be a simple generalization of what is discussed in Thm 23 of [SALS15] (on how a small loss bound leads to a faster rate). Indeed, small loss bound implies low approximation regret as discussed in Sec 3 of this paper. Furthermore, although the main results in [SALS15] are about the expectation feedback model, their result in Thm 23 is actually applicable to the same realized feedback model. So the claim that this paper improves upon [SALS15] in terms of the feedback model seems to be an overstatement to me. In fact, the claim about the improved speed is also a bit misleading, at least for the earlier part of the paper. It does convergence faster, but only to an approximation of the PoA. So in some sense these convergence rates not exactly comparable.  In the bandit setting, there is actually a known algorithm that gives a small loss bound against non-oblivious adversary. See Allenberg, Chamy, Peter Auer, László Györfi, and György Ottucsák. "Hannan consistency in on-line learning in case of unbounded losses under partial monitoring." 2006. It looks like this can be directly applied here?  Some detailed comments are listed below: 1. I would suggest explaining why most of the previous work requires the expectation feedback model while it’s not necessary here. 2. As mentioned above, Line 49 (the second bullet point for improvements) seems to be an overstatement. Same for Line 80, “without a constant factor loss” but still not exactly the PoA, right? 3. In Definition 1, maybe \epsilon should be explicitly required to fall in (0,1)? 4. In Definition 2, the right hand side of the inequality is missing the sum over i. 5. In the end of Proposition 2, instead of writing \epsilon in terms of \gamma, maybe it’s more natural to write \gamma in terms of \epsilon? 6. I would suggest putting Example 1-3 (and related paragraphs) to Sec 2, right after giving the definition of low approximate regret. 7. I don’t see how doubling trick can make any algorithm with weak low approximate regret to an algorithm with strong low approximate regret. 8. For Theorem 3, it’s weird to use informal statements like “regret is bounded by \sqrt{Total loss of best action}” in a theorem. I suggest just writing down the exact expression. 9. Line 222, why “within a (1+\epsilon) favor”? It’s \lambda/(1-\mu-\epsilon) compared to \lambda/(1-\mu), right? 10. The proposed noisy Hedge is a bit similar to Fixed Share, see for example Cesa-Bianchi, et al. 2012 "Mirror descent meets fixed share (and feels no regret)."  11. Some typos and small mistakes:  1) Line 3, “a small a multiplicative”  2) Line 34, remove “in”?  3) Line 42, “…more information than is…”? 4) Line 155, “required that is s is…”? 5) Line 207, “…For instance [FS97, YEYS04]…”, maybe should put this into a parenthesis, and make the punctuation right.


Selective inference for group-sparse linear models
We develop tools for selective inference in the setting of group sparsity, including the construction of confidence intervals and p-values for testing selected groups of variables. Our main technical result gives the precise distribution of the magnitude of the projection of the data onto a given subspace, and enables us to develop inference procedures for a broad class of group-sparse selection methods, including the group lasso, iterative hard thresholding, and forward stepwise regression. We give numerical results to illustrate these tools on simulated data and on health record data.

The paper proposes an approach to construct confidence intervals and p-values for the selection of groups of covariates. The approach is applicable to a broad class of methods, such as forward selection, iterative hard-thresholding, and group lasso, which are taken as illustrative examples in the paper. The challenge of the addressed problem is related to the fact that the statistical assessment of the importance of the groups must account for the fact that the same data have been used to select the groups. The novelty of the paper lies in the ability to construct confidence intervals while previous approaches for groups of covariates were limited to p-values. The contribution relies on a key lemma describing the distribution of truncated projections of Gaussian's. Two small-scale experiments are carried out, on synthetic and real-world data.
Clarity/Presentation/Related work/Presentation of contribution(s): The paper is overall clearly and well written. Also, the paper is well structured and makes a good introduction to the problem at hand, exposing what the new challenges and the resulting contributions are (e.g., paragraph lines 103-115). At some few places (see details below), some additional details would be useful.   Technical level: The paper appears as technically sound and Lemma 1/Theorem 1 represent a non-trivial contribution. Some questions related to the proofs (see afterwards) should be clarified.   Experiments: The experimental section may appear as a bit disappointing. For instance, it is unclear what the message of the second experiment (Section 4.2) is? Moreover, some further comparisons/discussions to other approaches may be beneficial (see details below). For example, since the main challenge of this work is to try to work with the same data as those used for the selection of the groups, it would make sense to compare against the more naive strategy based on splitting the data (which is supposed to lead to some loss of accuracy for model selection and power for inference).    Details comments:  * It should be discussed earlier in the paper whether the groups are assumed to form a partition (i.e., no overlaps).  * A discussion about going beyond the quadratic loss (or equivalently, the Gaussian model) would be interesting.  * Condition (1) should be more detailed: Why is this the right quantity to consider?  * In Theorem 1, it would useful to have more details regarding the quantities we condition on and with respect to which random variables the probabilities are considered.  * How is the equation hat{f_Y}(L_alpha) = alpha numerically solved? (line 252)  * A discussion about the computational complexity of the approach in the different scenarios (IHT, group lasso, etc.) would be useful.  * It is unclear to understand what the take-home message of the second experiment (Section 4.2) is?  * Could the proposed methodology be used for techniques like [a, and references therein]? Moreover, how would the proposed approach compare with a simple bootstrap operation (along the lines of [b] but for group-sparse estimators)?  * Supplementary material: In the beginning of the proof of Theorem 1, don't we need as well the continuity of t -> f_y(t) to guarantee the existence of L_alpha? More details should also be provided regarding the proofs of the properties of f_y (e.g., limiting behaviors and monotonicity).  [a] Ndiaye, E.; Fercoq, O.; Gramfort, A. & Salmon, J. GAP Safe screening rules for sparse multi-task and multi-class models Advances in Neural Information Processing Systems, 2015, 811-819 [b] Bach, F. Bolasso: model consistent Lasso estimation through the bootstrap Proceedings of the International Conference on Machine Learning (ICML), 2008


DISCO Nets: DISsimilarity COefficient Networks
We present a new type of probabilistic model which we call DISsimilarity COefficient Networks (DISCO Nets). DISCO Nets allow us to efficiently sample from a posterior distribution parametrised by a neural network. During training, DISCO Nets are learned by minimising the dissimilarity coefficient between the true distribution and the estimated distribution. This allows us to tailor the training to the loss related to the task at hand. We empirically show that (i) by modeling uncertainty on the output value, DISCO Nets outperform equivalent non-probabilistic predictive networks and (ii) DISCO Nets accurately model the uncertainty of the output, outperforming existing probabilistic models based on deep neural networks.

This paper proposes a new way to introduce uncertainty estimation into deep models. It does so by optimising a different cost, the dissimilarity coefficient, which allows for any arbitrary cost to be considered while still capturing the uncertainty of the samples into account.  They apply it on the NYU Hand Pose estimation dataset and compare to GAN and cGAN.
[EDIT] I'd like to thank the authors for their response, which addressed most of my concerns and made me more confident about the paper.  - The connection between using U[0,1] noise and performing Inverse transform sampling makes total sense and is actually quite interesting, it would be really useful to state it in the paper. - The loss restriction point was helpful as well thanks, it is indeed general enough then, no issues. - I completely agree with the assessment of the differences/difficulties in comparing DISCO Net and VAEs. A comparison would be great, but perhaps it would actually be more meaningful and easier to perform it on another dataset entirely, which may prove quite a lot of work just for this paper. I think mentioning the similarities/differences clearly but leaving it for further work would be fine.  [Original review] I'm not sure how I feel about this paper.  I wasn't aware of the dissimilarity coefficient metrics, but they seems to be supersets of Jensen-shannon divergences, so at least they are expressive enough. I'm not sure how useful they would be without actual proofs that P_G(y|x) actually converges to the true one. Hence most of the motivation for the paper is conditioned on the end of Section 3.2 "Proper scoring rule". Hence actually, the distances that can be used are restricted, and they consider Euclidean distance only. I would have appreciated more space to be spent on explaining exactly how well the distributions are properly captured. The authors just inject uniform noise, without any proof that this will be sufficient to capture the correct distribution.  Variational Autoencoders (VAE) are mentioned in the introduction, but are not used for comparison, and this is quite lacking. Indeed at least for VAE the assumptions about the approximations to the posterior are known and well captured. It would have been great to see how a VAE (not really autoencoder, but say a network which takes the depth map as input for the Encoder, and then Decodes back to the poses, while still having a Gaussian prior over the hidden code).  Indeed, I actually think that the particular architecture shown in Figure 2 is really just a slightly different way to represent a VAE (although using a different noise prior and a different optimisation function, which could be either good or bad depending on how much you want to keep the theoretical proofs of VAEs): The Encoder in VAE usually output two estimates: a vector for the mean of the posterior, and another vector for the covariance of the Gaussian posterior (a diagonal covariance actually for efficiency reasons). When training, the reparametrization trick uses samples from a unit N(0,I) gaussian, multiplies it by the covariance posterior estimator and adds it to the mean. This is functionally equivalent to the architecture shown in Figure 2, if we consider the output of the Convolutional layers of DISCO-nets as the mean of the posterior. The concatenated noise sample will be multiplied by a weight matrix, and then added to the next the layers alongside the mean (which will get its own weight matrix as well). Hence the input, as seen from the first dense layer of the decoder, is exactly the same as for VAE: learnt_mean + sampled noise * learnt_variance.  The difference, as said before, is in the form of the sampled noise, what the cost is, and how it's estimated. DISCO-net assume that K samples are used in order to evaluate the cost. This is, again, similar to extensions to VAE which use multiple samples in order to improve the variational lower bound (https://arxiv.org/abs/1509.00519).  So, this is not really a problem, but more an observation, which would be interesting to discuss.   The comparison with the BASE model are good, but I think \sigma should really have been optimized as an hyperparameter to make it fair. At least I'd like to see how a BASE model with \sigma=0 performs on metrics that did not require multiple samples.  Another aspect of my unease with trusting the overall model is the fact that no checks are done on other datasets which require to model the full distribution. The authors could have used more standard datasets (usually MNIST or OMNIGLOT now) for that, or synthetic distributions like done in [http://arxiv.org/abs/1505.05770].  I commend the authors on their attempts to make GAN and cGAN work. They did seem to run into the usual stability issues with them, but as far as I can tell they tried their best and made a fair comparison. Unfortunately, I felt the authors didn't cover the other models as well, in Section 4.4. This section lacks interpretation and discussion, and Figure 4 is neither presented nor commented.   Considering that DISCO Nets do not achieve SOTA compared to these, it would have required a bit more explanation.  The rest of the paper is clear, the supplementary material covers additional training details, and all the information to reproduce their results is easy to find.  In conclusion, I am happy to see another method to construct DNN which capture uncertainty, but given this paper I am not certain of how correct their uncertainty estimate really is.


Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation
Estimators of information theoretic measures such as entropy and mutual information are a basic workhorse for many downstream applications in modern data science. State of the art approaches have been either geometric (nearest neighbor (NN) based) or kernel based (with a globally chosen bandwidth). In this paper, we combine both these approaches to design new estimators of entropy and mutual information that outperform state of the art methods. Our estimator uses local bandwidth choices of k-NN distances with a finite k, independent of the sample size. Such a local and data dependent choice improves performance in practice, but the bandwidth is vanishing at a fast rate, leading to a non-vanishing bias. We show that the asymptotic bias of the proposed estimator is universal; it is independent of the underlying distribution. Hence, it can be precomputed and subtracted from the estimate. As a byproduct, we obtain a unified way of obtaining both kernel and NN estimators. The corresponding theoretical contribution relating the asymptotic geometry of nearest neighbors to order statistics is of independent mathematical interest.

The authors propose a new class of estimators for Shannon entropy (and Mutual Information (MI)). The estimators are density plug-in estimators based on the empirical average of the log of density estimates at points drawn from the underlying distribution. For density estimation, the authors use local likelihood density estimators (LLDE’s), which are higher-order generalizations of KDE and kNN density estimators.
The most potentially novel contribution of the paper is the discussion in section 6 where the authors show how adaptive bandwidths (i.e. k-nn) can be superior to fixed bandwidth (i.e. bandwidth is the same at all points). However, the discussion focuses on Shannon entropy only. If the results in this discussion can be applied to other distributional functionals (e.g. Renyi entropy and others), then it is indeed very novel. However, if it only applies to Shannon entropy, then it is quite limited. This similarly applies to all of the contributions. The authors should comment on the feasibility of extending these results to other distributional functionals. This is critical as all current state of the art estimators focus on more than just Shannon entropy.  Another contribution of the paper is the result that the empirical average converges to the true entropy plus a constant term B(k,p,d, K) that is independent of the underlying distribution for any choice of kernel K, order p, and parameter k. If the term B(k,p,d) were to be distribution-dependent, then typically k is required to go to infinity (but at a slower rate compared to n) in order to ensure convergence, which results in slower rates of convergence. But because the authors show that B(k,p,d) is independent of the underlying density, they can estimate this quantity using Monte-Carlo methods, and subtract this constant term from their empirical average to derive an estimator that converges to the true entropy even with k fixed. Results of this form have been established before for specific choices of kernel K and order p. For example, in [6], the result is established for p = 0 and the choice of a step kernel. This paper shows that [6] (and other estimators) are a special case of this very general result they have derived, which is a nice result.  Of course, this raises the question as to what the advantages are with using a estimator with different choice of kernel and higher values of p are relative to the existing estimators which correspond to p = 0. The authors claim that the advantage is that the finite sample performance of higher-order estimators is better, and mainly because they do better at the edges of the support of the density, where the higher order density estimators have lower bias by virtue of adapting to the contours of the distribution (see figure 1). They present experimental results, (but no theoretical results), and claim that the higher order estimators have better finite sample MSE compared to [6].  These are some of the issues with the paper:  1- In Section 2, the authors claim that higher order LLDE’s automatically correct for boundary bias. Do they mean that boundary bias is completely eliminated by using LLDE’s? Or do they mean that LLDE’s do a better job of accounting for some of the boundary bias but still do not completely eliminate it? If it is the former, then this is a very strong claim and the analysis in Section 2 does not convince me this is true. For instance, for samples drawn from a distribution with compact support, I believe the density estimates for points on the boundary of the support would be underestimated irrespective of the order of the LLDE. If it is the later, the authors should make this more clear in Section 2. 1.1- On a related note, figure 1(b) helps readers visualize a standard k-NN density estimate and understand why it suffers from boundary bias. But there is no illustration of the LLDE density estimator in this figure. Addition of this illustration would greatly hep readers get a sense for why LLDE’s better account for the boundary bias.  2- The flip side to using LLDE’s of higher order is that they are more expensive to implement. The authors attempt to address this by using subsets of samples for evaluating equation (10). It would be interesting to compare the computation time for lower order kernels like the one in [6] against the higher-order kernels to see what the trade-off is between MSE performance and computation time.  3-The simulations have some major flaws. The authors claim that their estimator outperforms the state of the art estimators. This claim is far too strong. First, they only compare to one state of the art estimator [5] and completely neglect the other state of the art estimators (in terms of MSE convergence rate), most especially ensemble methods (see [5R,6R] below).  Another state of the art estimator is included in [11R]. I am also not convinced that the authors implemented the Von mises estimator of [5] correctly as the MSE is nonmonotonic as the sample size increases in Figure 2. How did the authors choose the bandwidth and what kernel did they use? The authors should provide more details on these experiments. Also, in figure 2(b), the rate at which the MSE of the KL estimator is decreasing in sample size is much faster than the rate at which the p=2 LNN estimator’s MSE is decreasing. If the figure were extrapolated beyond n=1500 samples, it seems like the KL estimator would start to outperform the LNN estimator. Figure 3(b) seems to convey a similar story that the small sample performance of the LNN-KSG estimator is much better compared to the standard KSG, but again it is not clear that the rate of convergence of LNN-KSG is better than KSG. This is a critical issue that must be studied more carefully, especially given that the results presented are asymptotic convergence results and not finite sample analysis results. Finally, the authors only focus on the very low dimensional case. I suspect that in the higher dimensional setting, the state of the art estimators would outperform the proposed estimator which limits its applicability.  4- In addition to the KL and von-moses estimators for entropy, the KSG estimator for MI, and the estimators mentioned above, there are several other estimators in literature for these quantities that are important and have been neglected by the author. See the references below. Some if not all of these references should be included, especially those with known convergence rates.  5- Although the paper in its current form makes significant theoretical contributions, it would definitely be of interest to extend the results to obtain theoretical results that describe finite sample performance of these estimators (or equivalently, obtain the rate of convergence of the estimators). For instance, see [3R, 5R, 6R, 7R, 11R], where rates of convergence are obtained for various entropy and MI estimators. See also [12R] where the convergence rates of the KL estimator are derived.  6- In this paper, results are derived only for estimation of Shannon entropy and MI. As mentioned above, estimators for other forms of entropy and MI are also of interest.  6.1- For instance, see [7R, 8R, 9R] for work on Renyi entropy and divergence estimation. It would be useful to investigate if similar results to the ones derived in this paper hold for these Renyi measures. It has already been shown in [9R] that a bias correction term exists for Renyi entropy estimators (the factor C_k in section 3.1 in [9R]) - does a similar term exist for higher order  LLDE’s?  6.2-In addition to Renyi estimators, there has also been work on estimation of general functionals of densities - see [3R, 5R, 10R, 11R]. To the best of my knowledge, there has been no work on establishing that bias correction factors exist for estimators of these general functionals, and as a result, density plug-in estimators require that k also grow to infinity. It would be very interesting to see if the authors could extend their analysis to estimators for general functionals other than log(x)  (Shannon) and x^{a-1} (Renyi), and comment on whether bias correction factors exist for estimators of these general functionals.   6.3- This reviewer would be interested in hearing the authors take on whether they think their analysis is general enough to accommodate these more general functionals, and if not, if they could comment on what makes the log(.) function critical to their analysis.  In summary, I would not expect the authors to address all of these issues in the rebuttal. At a minimum, I would expect the authors to 1) address the issues wrt the boundary bias; 2) Provide more details on the experiments, especially when implementing the Von Mises estimator; 3) Either soften their language with regards to the experimental results (i.e. recognize that their approach only outperforms one state of the art estimator in rather limited circumstances and at lower dimensions) OR compare their estimator to other state of the art estimators and at higher dimensions; 4) Include at least the references where convergence rates are known; 5) Comment on the feasibility of extending this work to more general functionals.   If these issues are addressed, then I would recommend the paper be accepted for publication at NIPS. If any of the other issues are addressed, the paper would be even stronger.    Additional references:  [1R] Pál, Dávid, Barnabás Póczos, and Csaba Szepesvári. "Estimation of Rényi entropy and mutual information based on generalized nearest-neighbor graphs." Advances in Neural Information Processing Systems. 2010. [2R] Beirlant, Jan, et al. "Nonparametric entropy estimation: An overview."International Journal of Mathematical and Statistical Sciences 6.1 (1997): 17-39. [3R] Estimating divergence functionals and the likelihood ratio by convex risk minimization XL Nguyen, MJ Wainwright, MI Jordan - IEEE Transactions on Information Theory, 2010 [4R] Póczos, Barnabás, Liang Xiong, and Jeff Schneider. "Nonparametric divergence estimation with applications to machine learning on distributions."arXiv preprint arXiv:1202.3758 (2012). [5R] Moon, Kevin, and Alfred Hero. "Multivariate f-divergence estimation with confidence." Advances in Neural Information Processing Systems. 2014. [6R] Sricharan, Kumar, Dennis Wei, and Alfred O. Hero. "Ensemble estimators for multivariate entropy estimation." IEEE Transactions on Information Theory59.7 (2013): 4374-4388. [7R] Krishnamurthy, Akshay, et al. "Nonparametric Estimation of Renyi Divergence and Friends." ICML. 2014. [8R] Hero, Alfred O., et al. "Applications of entropic spanning graphs." IEEE signal processing magazine 19.5 (2002): 85-95. [9R] Leonenko, N., L. Prozanto, and V. Savani (2008), A class of r´enyi information estimators for multidimensional densities, Annals of Statistics, 36, 2153–2182. [10R] Sricharan, Kumar, Raviv Raich, and Alfred O. Hero. "Estimation of nonlinear functionals of densities with confidence." IEEE Transactions on Information Theory 58.7 (2012): 4135-4159. [11R] Singh and Poczos, "Exponential Concentration Inequality of a Density Functional Estimator," NIPS 2014. [12R] Biau and Devroye, "Entropy Estimation," In Lectures on the Nearest Neighbor Method," pp 75-91, Springer, 2015.  EDIT: Based on the comments of other reviewers, I have downgraded my score for the novelty of this paper. I think that the novelty of theorems 1 and 2 is sufficient to maintain the score at a "3". However, I think that the authors should give more reference to prior work in the other less novel results as Reviewer 4 discussed. I think that this and toning down their claims on the experiments is critical (as well as referencing more prior work in general).  Some of the other issues brought up by the other reviewers also lead me to leave the technical quality at a "2". These issues further seem to suggest that the authors' approach is mostly limited to Shannon entropy. If the authors had shown broader applicability to other functionals, I would have looked more favorably upon the paper despite these issues.


Learning Treewidth-Bounded Bayesian Networks with Thousands of Variables
We present a method for learning treewidth-bounded Bayesian networks from data sets containing thousands of variables. Bounding the treewidth of a Bayesian network greatly reduces the complexity of inferences. Yet, being a global property of the graph, it considerably increases the difficulty of the learning process. Our novel algorithm accomplishes this task, scaling both to large domains and to large treewidths. Our novel approach consistently outperforms the state of the art on experiments with up to thousands of variables.

This paper uses k-trees to learn bounded tree-width Bayesian networks. The proposed method is built on Nie et al. 2016, which incrementally grows the k-tree and the BN as the subgraph of the k-tree. The parent set of a variable in the BN is restricted by the clique of k nodes that the variable is connected to in the k-tree. To find this clique of k nodes, the authors use A* algorithm similar to Nie et al. 2016 but with a different score function that is consistent with the overall objective function.  They also propose a greedy search approach instead of A* to increase the scalability.
  The proposed method is very similar to previous work by Nie et al. -- both use k-trees to search for low-treewidth Bayesian networks, both start with a randomly chosen initial clique, and both propose using an A* method for finding the best tree.  The differences are that Nie et al. score k-trees using a mutual information score and use BDeu for choosing the final consistent Bayesian network, while this paper proposes using BIC and incrementally building the Bayesian network along with the k-tree, using the BN to score the k-tree.  This paper also includes the additional restriction that the complete variable (partial) order is chosen randomly, while in Nie et al. only the initial clique is chosen randomly.  The main justification for these differences is the ability to scale to large treewidths.  However, in the experiments, the previous S2 algorithm also can scale to large treewidths.  The experimental results look promising, showing that the new method can optimize BIC better (given a treewidth constraint and a particular bound on running time).                Comments: - Why use BIC instead of BDeu, as in previous work?  As the authors note, this makes their results incomparable to previous results; what are the advantages that outweigh this obvious disadvantage? - The score function in g(S) (line 135) is not explicitly defined. Do you mean BIC? - It would be helpful to see the performance of all methods with different time limits (e.g., 1 minute, 1 hour, 10 hours). - Many bounded-treewidth methods exist in the literature, some for learning chain graphs and some for learning BNs; the results would be more convincing with additional baselines. (I would be interested in seeing test set log-likelihood as well, though I understand that the stated goal of these algorithms is to optimize a particular criterion as well as possible, not to evaluate generalization.) - The paper is clear for the most part, but the presentation could be improved.  For example, it would help to show the full table of results for the large datasets, not just a boxplot and counts of which algorithm was better.  Supplementary material could be used for this if it will not fit in the main paper.  Minor: - The sentences at line 45 and 46 are redundant.  Maybe authors want to merge them together. - The space of learnable DAGs is not well explored. It would be interesting to see some experiments that show the edge recovery potential of the proposed algorithm by generating samples from a different set of synthetic graphs.


VIME: Variational Information Maximizing Exploration Rein Houthooft
Scalable and effective exploration remains a key challenge in reinforcement learning (RL). While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high-dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristics such as -greedy exploration or adding Gaussian noise to the controls. This paper introduces Variational Information Maximizing Exploration (VIME), an exploration strategy based on maximization of information gain about the agent's belief of environment dynamics. We propose a practical implementation, using variational inference in Bayesian neural networks which efficiently handles continuous state and action spaces. VIME modifies the MDP reward function, and can be applied with several different underlying RL algorithms. We demonstrate that VIME achieves significantly better performance compared to heuristic exploration methods across a variety of continuous control tasks and algorithms, including tasks with very sparse rewards.

This paper looks at the notion of curiosity in deep reinforcement learning. It returns to the idea of equating exploratory behavior with maximizing information. Key contributions are the formulation of a variational perspective on information gain, an algorithm based on Blundell et al.'s Bayesian  neural networks and the exposition of the relationship between their approach and Schmidhuber's compression improvement. Results on simple domains are given.
The paper shows a pleasant breadth of understanding of the literature. It provides a number of insights into curiosity for RL with neural networks. I think it could be improved by focusing on the development of the variational approach and the immediately resulting algorithm. As is, there are a number of asides that detract from the main contribution.  My main concern is that the proposed algorithm seems relatively brittle. In the case of Eqn 17, computing the Hessian might only be a good idea in the diagonal case. Dividing by the median suggests an underlying instability.  Questions: * The median trick bothers me. Suppose the model & KL have converged. Then at best the intrinsic reward is 1 everywhere and this does not change the value function. In the worst case the KL is close to 0 and you end up with a high variance in your intrinsic rewards. Why isn't this an issue here?  * Eqn 13: updating the posterior at every step is different from updating the posterior given all data from the prior. Do you think there are issues with the resulting "posterior chaining"?  * How good are the learned transition models?  * Can you explain line 230: "very low eta values should reduce VIME to traditional Gaussian control noise"?  * Why do you propose two intrinsic rewards on line 5 of algorithm 1? I'd like to see a clear position.  Suggestions: Eqn 2: P(. | s_t, a_t) Line 116: For another connection, you may want to look at Lopes et al. (2012), "Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress" Line 119: You should specify which description length you mean... the statement is possibly imprecise/incorrect as is Line 128: in expectation propagation (which I know from Bishop (2006)) the KLs end up getting reversed, too... is there a relationship? Eqn 13: log p(s_t | theta) should be p(s_t | s_t-1, a_t-1, theta), no?  It would be good to give empirical evidence showing why the median is needed Graphs: unreadable It might be good to cite more than just [20] as a reference on Bayesian networks. Again, Bishop (2006) (Section 5.7) provides a nice list.


Gaussian Processes for Survival Analysis
We introduce a semi-parametric Bayesian model for survival analysis. The model is centred on a parametric baseline hazard, and uses a Gaussian process to model variations away from it nonparametrically, as well as dependence on covariates. As opposed to many other methods in survival analysis, our framework does not impose unnecessary constraints in the hazard rate or in the survival function. Furthermore, our model handles left, right and interval censoring mechanisms common in survival analysis. We propose a MCMC algorithm to perform inference and an approximation scheme based on random Fourier features to make computations faster. We report experimental results on synthetic and real data, showing that our model performs better than competing models such as Cox proportional hazards, ANOVA-DDP and random survival forests.

In the paper 'Gaussian Processes for Survival Analysis', the authors present a semi-parametric approach to survival time modeling in the presents of covariates.  The proposed method combines expert-knowledge in form of the parametric component, and a Gaussian Process (GP) as a non-parametric part to model differences between the former and the data.  The model, dubbed SGP, is compared to other, standard approaches in this area on synthetic and real world data.  The paper thoroughly introduces the method in section 2 by walking the reader through the layers of the hierarchical model in Equation (1) which is later extended in Equation (3) to account for covariates. In essence, the survival times are modeled as the first jumps of a Poisson process with an intensity assumed to be the product of an hand-crafted base function times a transformed GP.  The non-linear transformation of the GP assures that the model yields a proper survival function.  Section 3 shows that the SGP is tractable by defining a generative process to sample a survival time needed to perform inference. Also the (approximate) treatment of censored data points is discussed. To be able to handle large data sets, Section 4 describes an approximation scheme to the kernel based on a random Fourier series. Section 5 presents the experiments on one synthetic benchmark and a real world example, namely the Veteran data set available in R. For the latter, SGP is compared to three other methods for survival analysis.
The general idea of the paper seems promising, but I have several concerns:  1. The kernel choices in Equation (2) are not motivated and quite surprising to me. While a base kernel in 't' only makes a lot of sense, I don't understand the additive part for the covariates. Especially for the one used in the experiments, the summation of the product the covariates X_j, Y_j, and a kernel K_j(t,s) seems counterintuitive. Why should the covariance between to points grow if one of the covariates increases? Why not use a ARD (automatic relevance detection) kernel for all covariates combined rather than summing over all of them independently?  2. There is a discussion about the noise-parameters and the length-scales of the kernel, but what I missed was the actual method to adjust them given the data. A maximum likelihood estimation is mentioned, but only for the initialization. From experience, I know that this is crucial for 'high quality' GPs.  3. The experiments section is rather week. There is no comparison to the other algorithms on the synthetic benchmark, and the size of the real world benchmark with 137 samples and 5 covariates seems to small for the effort spent on approximations earlier in the paper. This amount of data should be no problem for a proper GP treatment without any approximations to the kernel.  4. Using GPs together with a parametric function to incorporate expert knowledge is a good idea, but the presented method makes a few ad-hoc choices to combine these two, e.g. the positive link function, or the particular kernel choice. An alternative could be a non-zero mean GP, where the mean function incorporates the expert knowledge. By modeling, for example, log(lambda) with this method, one could potentially enforce positiveness of the rate while also ensuring that the Survival function is well defined.   Overall, I think the paper is trying to accomplish too much at the same time: the GP based semi-parametric model, the (approximate) handling of censored data, the fast inference algorithm, and the random fourier kernel approximation. To me it feels that maybe just the first two point above without a focus the scalability, but with a more thorough empirical evaluation would make a more concise paper. The field of approximate GPs offers several possibilities to extend to larger data sets and many covariates.  Besides the technical points, the presentation of the material requires improvement: there are a few typos; citations are frequently used as nouns; and sometimes certain paragraphs feel out of place. For example, in line 55, Gaussian Cox Processes are mentioned for the first (and I think last time) without any context, citation or further explanation.   At this point, I have to recommend rejecting the paper, but I would like to encourage the authors to improve the paper for a future submission.