Sublinear Time Low-Rank Approximation of Distance Matrices
Let P = {p 1 , p 2 , . . . p n } and Q = {q 1 , q 2 . . . q m } be two point sets in an arbitrary metric space. Let A represent the m × n pairwise distance matrix with A i,j = d(p i , q j ). Such distance matrices are commonly computed in software packages and have applications to learning image manifolds, handwriting recognition, and multi-dimensional unfolding, among other things. In an attempt to reduce their description size, we study low rank approximation of such matrices. Our main result is to show that for any underlying distance metric d, it is possible to achieve an additive error low rank approximation in sublinear time. We note that it is provably impossible to achieve such a guarantee in sublinear time for arbitrary matrices A, and our proof exploits special properties of distance matrices. We develop a recursive algorithm based on additive projection-cost preserving sampling. We then show that in general, relative error approximation in sublinear time is impossible for distance matrices, even if one allows for bicriteria solutions. Additionally, we show that if P = Q and d is the squared Euclidean distance, which is not a metric but rather the square of a metric, then a relative error bicriteria solution can be found in sublinear time. Finally, we empirically compare our algorithm with the singular value decomposition (SVD) and input sparsity time algorithms. Our algorithm is several hundred times faster than the SVD, and about 8-20 times faster than input sparsity methods on real-world and and synthetic datasets of size 10 8 . Accuracy-wise, our algorithm is only slightly worse than that of the SVD (optimal) and input-sparsity time algorithms.
and Demaine et al. use them for music and musical rhythms [7]. For an excellent tutorial on Euclidean distance matrices, we refer the reader to [8], which lists applications to nuclear magnetic resonance (NMR), crystallagraphy, visualizing protein structure, and multi-dimensional unfolding.
We consider the most general case for which P and Q are not necessarily the same set of points. For example, one may have two large unordered sets of samples from some distribution, and may want to determine how similar (or dissimilar) the sample sets are to each other. Such problems arise in hierarchical clustering and phylogenetic analysis 1 . Formally, Let P = {p 1 , p 2 , . . . p m } and Q = {q 1 , q 2 . . . q n } be two sets of points in an arbitrary metric space. Let A represent the m × n pairwise distance matrix with A i,j = d(p i , q j ). Since the matrix A may be very large, it is often desirable to reduce the number of parameters needed to describe it. Two standard methods of doing this are via sparsity and low-rank approximation. In the distance matrix setting, if one first filters P and Q to contain only distinct points, then each row and column can contain at most a single zero entry, so typically such matrices A are dense. Low-rank approximation, on the other hand, can be highly beneficial since if the point sets can be clustered into a small number of clusters, then each cluster can be used to define an approximately rank-1 component, and so A is an approximately low rank matrix.
To find a low rank factorization of A, one can compute its singular value decomposition (SVD), though in practice this takes min(mn 2 , m 2 n) time. One can do slightly better with theoretical algorithms for fast matrix multiplication, though not only are they impractical, but there exist much faster randomized approximation algorithms. Indeed, one can use Fast Johnson Lindenstrauss transforms (FJLT) [16], or CountSketch matrices [4,13,15,1,5], which for dense matrices A, run in O(mn) + (m + n)poly(k/ ) time. At first glance the O(mn) time seems like it could be optimal. Indeed, for arbitrary m × n matrices A, outputting a rank-k matrix B for which
can be shown to require Ω(mn) time. Here A k denotes the best rank-k approximation to A in Frobenius norm, and recall for an m × n matrix C, C 2 F = i=1,...,m,j=1,...,n C 2 i,j . The additive error guarantee above is common in low-rank approximation literature and appears in [10]. To see this lower bound, note that if one does not read nearly all the entries of A, then with good probability one may miss an entry of A which is arbitrarily large, and therefore cannot achieve (1.1).
Perhaps surprisingly, [14] show that for positive semidefinite (PSD) n × n matrices A, one can achieve (1.1) in sublinear time, namely, in n · k · poly(1/ ) time. Moreover, they achieve the stronger notion of relative error, that is, they output a rank-k matrix B for which
(1.
2)
The intuition behind their result is that the "large entries" causing the Ω(mn) lower bound cannot hide in a PSD matrix, since they necessarily create large diagonal entries. A natural question is whether it is possible to obtain low-rank approximation algorithms for distance matrices in sublinear time as well. A driving intuition that it may be possible is that no matter which metric the underlying points reside in, they necessarily satisfy the triangle inequality. Therefore, if
is large, and further, all these distances can be found by reading the first row and column of A. Thus, large entries cannot hide in the matrix. Are there sublinear time algorithms achieving (1.1)? Are there sublinear time algorithms achieving (1.2)? These are the questions we put forth and study in this paper.

This paper presents an algorithm for computing a near-optimal low-rank approximation for a distance matrix — i.e. a matrix where the (i,j)th entry equals d(p_i, q_j) for some metric d and sets of points p_1, …, p_m and q_1,… q_n. Distance matrices are common in learning and data analysis tasks and low-rank approximation is a ubiquitous approach to compressing matrices.   In general, the fastest existing low-rank approximation algorithms are randomized and, to compute a k-rank approximation, run in O(nnz(A) + min(n,m)*poly(k/eps)) time for a matrix A with nnz(A) non-zeros. Since the second term is often considered lower order (k is typically << n and eps is moderate), these algorithms run in roughly linear time in the size of the input.   Naturally, this is optimal for general matrices — if I don’t at least read every entry in the matrix, I could miss a very heavy entry that I would need to see to output a good low-rank approximation. However, we could ask if better runtimes are possible for matrices with more structure, like distance matrices. We might hope for algorithms that run in sublinear time. Recent work shows that, surprisingly, sublinear time low-rank approximation algorithms can be obtained for positive semidefinite matrices, another structured class of matrices. This result relies on the fact that it’s not possible to add an arbitrarily heavy element A_i,j to a PSD matrix without increasing the size of either A_i,i or A_j,j. Accordingly, simply reading the diagonal of A (which only takes n time) avoids the trivial lower bounds that apply to general matrices.   The authors begin with the observation that a similar statement is true for distance matrices. By triangle inequality, if A_i,j is very heavy, A_1,i, A_1,j, or A_1,1 must be too. These entries can all be found by reading the first row of A (or really any single row of A_. As in the case of PSD matrices, this property allows us to avoid lower bounds for general matrices.  The authors leverage this observation to motivate their algorithm, which is based on solving the low-rank approximation using a random subsample of the original matrix. In particular, they sample rows/columns by their norms. It is well known that subsampling in this way makes it possible to obtain a low-rank approximation within additive error eps*||A||_F^2 of optimal. For general matrices, computing norms naively takes nnz(A) time. However, the authors show that for distance matrices, norms can be estimated coarsely in sublinear time. They can subsample the matrix using these coarse estimates, and then recurse to shrink the problem size even smaller. In recursing, the authors alternate between row and column sampling.  Overall, this yields an algorithm that runs in (min(N,m)^1+gamma) * poly(k/eps)^1/gamma time for any chosen gamma, which controls the levels of recursion in the algorithm. The authors present a simpler algorithm with just one level of recursion that runs in min(n,m)^1.34 * poly(k/eps) time. The authors implement this algorithm and it turns out to be quite practical, outperforming nnz(A) time random projection methods pretty significantly.   In addition to their main result, the authors include a number of additional results. Of particular interest is a lower bound, which demonstrates that additive ||A||_F^2 error is necessary — achieving full relative error for the low-rank approximation problem isn’t possible even for distance matrices without at least reading the whole matrix. This places the complexity of low-rank approximation for distances matrices somewhere between standard matrices, where even additive error is not possible in sublinear time, and PSD matrices, where full relative error is possible in sublinear time.  Overall, I think the paper is well written and the techniques are new and interesting. It’s also nice to see that the main algorithm presented is already practical.   One concern with the paper is that I find the proof of the faster (min(N,m)^1+gamma) * poly(k/eps)^1/gamma difficult to follow. In particular, recursive sampling cannot be performed naively because, after sampling and reweighting rows from a distance matrix, you end up with a matrix that is no longer a distance matrix. To deal with this challenge, the authors suggest partitioning sampled rows into “buckets” based on how much they were reweighed by. The matrix to be recursed on is a valid distance matrix, at least when restricted to rows in one bucket, which allows column norms to be estimated by estimating the norm in each block of rows separately (estimating column norms is need for the next step of reduction).   This makes perfect sense to me for the two step in(n,m)^1.34 * poly(k/eps) time algorithm (Algorithm 3), but I do not understand how the approach works for the faster Algorithm 2. In particular, I don’t see how the argument carries through multiple levels of recursion without blowing up the number of buckets and, for whatever reason, Algorithm 2 does not include the reweighing step in its pseudocode! I would encourage the authors to add it in explicitly. If this makes the algorithm much more difficult to present, I would encourage the authors to present the simpler, but still interesting Algorithm 3 in the main part of their paper, and to give a full and rigorous development of Algorithm 2 in the appendix.


BourGAN: Generative Networks with Metric Embeddings
This paper addresses the mode collapse for generative adversarial networks (GANs). We view modes as a geometric structure of data distribution in a metric space. Under this geometric lens, we embed subsamples of the dataset from an arbitrary metric space into the`2 space, while preserving their pairwise distance distribution. Not only does this metric embedding determine the dimensionality of the latent space automatically, it also enables us to construct a mixture of Gaussians to draw latent space random vectors. We use the Gaussian mixture model in tandem with a simple augmentation of the objective function to train GANs. Every major step of our method is supported by theoretical analysis, and our experiments on real and synthetic data confirm that the generator is able to produce samples spreading over most of the modes while avoiding unwanted samples, outperforming several recent GAN variants on a number of metrics and offering new features.

Review:   Mode collapse is a well-known problem in the GAN literature and it is an important one. The authors attribute the problem to the fact that GANs generate samples from a single Gaussian distribution, which has only one peak.  And they propose a fix to the problem by using a mixture of multiple Gaussian distributions.  In my view the diagnosis is clearly correct. It is so obvious that I am surprised that no one has thought of it before.  The solution proposed seems reasonable and has solid theoretical justification.  A key characteristic of this work is that it allows arbitrary user specified distance metric. The authors propose to use a classifier distance metric. It has been shown to improve generalization performance in the case of multiple modes as compared to the L2 distance and Earth Mover distance.  I found this very interesting.  The paper is also well-written and easy to follow.  I recommend that the paper be accepted.  While being very positive overall, I do have some concerns.  First, the number m of components in the Gaussian mixture needs to be provided by the user.  This is clearly a drawback. Moreover, I do not see any easy way to fix it as there isn’t any way to calculate model fit in GAN models. Second, the impact of m has not been sufficiently evaluated. This is an issue because a user is unlikely to get the correct value for m every time.    Other questions and comments:   - The proposed classifier distance is obtained from pre-trained classification networks. Does this mean that the propose method is a semi-supervised method rather than an unsupervised method?   - Why isn’t WGAN included in the comparisons?  - Please provide more information on the choices of hyper-parameters on Lines 200 and 204 and Line 512 in Supplementary Materials.  - The readability might be improved if the LPDD part is discussed before Wasserstein-1 distance in Section 3.1.  Confidence: The reviewer is confident but not absolutely certain about the proof of theorems.


Compact Generalized Non-local Network
The non-local module [27] is designed for capturing long-range spatio-temporal dependencies in images and videos. Although having shown excellent performance, it lacks the mechanism to model the interactions between positions across channels, which are of vital importance in recognizing fine-grained objects and actions. To address this limitation, we generalize the non-local module and take the correlations between the positions of any two channels into account. This extension utilizes the compact representation for multiple kernel functions with Taylor expansion that makes the generalized non-local module in a fast and low-complexity computation flow. Moreover, we implement our generalized non-local method within channel groups to ease the optimization. Experimental results illustrate the clear-cut improvements and practical applicability of the generalized non-local module on both fine-grained object recognition and video classification. Code is available at: https://github.com/KaiyuYue/cgnl-network.pytorch.

This paper proposes a novel network module to exploit global (non-local) correlations in the feature map for improving ConvNets. The authors focus on the weakness of the non-local (NL) module [31] that the correlations across channels are less taken into account, and then formulate the compact generalized non-local (CGNL) module to remedy the issue through summarizing the previous methods of NL and bilinear pooling [14] in a unified manner. The CGNL is evaluated on thorough experiments for action and fine-grained classification tasks, exhibiting promising performance competitive to the state-of-the-arts.  Positives: + The paper is well organized and easy to follow. + The generalized formulation (8,9) to unify bilinear pooling and non-local module is theoretically sound. + Good performance.  Negatives: - Less discussion on the linear version of CGNL using dot product for f. - Missing fundamental comparison to the simple ResBlock.  The authors nicely present the generalized formulation toward CGNL by unifying the two previous works of bilinear pooling and non-local module. Though the kernelized (non-linear) correlation function f is well theoretically motivated, the actual form of f that achieves the better empirical performance is a “linear” form (dot product). In this regard, the reviewer has the following concerns.  - Less discussion about the linear form. If the reviewer correctly understands the CGNL formulation, the linear function f of dot product f (line 204) can greatly simplify the CGNL into Y = X * W_theta * tr[(X*W_phi)’ * (X*W_g)] = X * W_theta * tr[(X’X)* W_g* W_phi’]   = s * X * W_theta, where s = tr[(X’X) * W_g * W_phi’]= tr[(X’X)* W] is just a scalar and W = W_g*W_phi’. This reformulation would be beneficial from the following viewpoints. > It reduces the parameters from {W_theta, W_phi, W_g} to {W_theta, W}, which facilitates the implementation. > It is closely related to squeeze-and-excitation (SE) module [9]. The above formulation can be regarded as a bilinear extension of SE from “squeeze” viewpoint since it “squeezes” the feature map X into the bilinear form of X’X while SE simply employs an average-pooling.   Such discussions as above would help the readers to further understand the methods and to further extend the method.  - Missing comparison. Based on the above discussion, one can think that the baseline for the linear CGNL is a simple ResBlock of  Z = BatchNorm( X * W_z ) + X, while the linear CGNL is  Z = BatchNorm( s * X * W_theta * W_z ) + X    = BatchNorm( s * X * W_tz ) + X. The only difference is the scaling factor s that is also build on X. Through batch normalization, such a scaling might be less effective (during the training) and thus by comparing these closely-related methods, the authors have to clarify its effectiveness of CGNL empirically. Due to this concern, the reviewer can not fairly evaluate the impact of the method on classification performance.  [After Rebuttal] The reviewer appreciates the authors’ efforts to perform the comparison experiments in such a short rebuttal period. The comparison with the standard ResBlock clarifies the effectiveness of the proposed method as well as helps us to further understand how it works.


Smoothed analysis of the low-rank approach for smooth semidefinite programs
We consider semidefinite programs (SDPs) of size n with equality constraints. In order to overcome scalability issues, Burer and Monteiro proposed a factorized approach based on optimizing over a matrix Y of size n × k such that X = Y Y * is the SDP variable. The advantages of such formulation are twofold: the dimension of the optimization variable is reduced, and positive semidefiniteness is naturally enforced. However, optimization in Y is non-convex. In prior work, it has been shown that, when the constraints on the factorized variable regularly define a smooth manifold, provided k is large enough, for almost all cost matrices, all second-order stationary points (SOSPs) are optimal. Importantly, in practice, one can only compute points which approximately satisfy necessary optimality conditions, leading to the question: are such points also approximately optimal? To answer it, under similar assumptions, we use smoothed analysis to show that approximate SOSPs for a randomly perturbed objective function are approximate global optima, with k scaling like the square root of the number of constraints (up to log factors). Moreover, we bound the optimality gap at the approximate solution of the perturbed problem with respect to the original problem. We particularize our results to an SDP relaxation of phase retrieval.

This paper concerns the Burer-Monteiro (i.e. low-rank) factorization heuristic for solving SDP problems of the form   min     s.t.  A(X) = b, X \succeq 0.   The BM heuristic relies on the fact when the constraint set is compact, the above SDP has a global solution with rank up to square root of the **number of the linear constraints** (denoted as m). This allows one to perform substitution X = YY^* and then perform a heuristic numerical search in the low-dimensional subspace of Y. In general, there is no guarantee such heuristic search would find the global solution, i.e. Y such that YY^* solves the original SDP. Recent advances (Bounal et al. 2016b, 2018) show that for almost all matrix C, when A(YY^*) defines a smooth manifold, all second-order stationary points (SOSP) of the resulting nonconvex program are global optimal, provided the rank of Y scales as sqrt(m). In practice, numerical methods can at least find approximate SOSP in finite running time. This paper shows that on average (due to the smoothed analysis), all approximate SOSP's produce approximate global optima to the original SDP (measured in terms of the objective value), again so long as the rank of Y scales as sqrt(m).   Solving large-scale SDP is of broad interest to many scientific communities with computational components, including machine learning. The current work, combined with the other recent works, build a solid theoretical foundation for the practical relevance and viability of the BM heuristic.   The result is closely connected to another recent work Bhojanapalli et al 2018. The two papers address the same stability issue and share some similar points at the technical level. One importance difference is that the current work focuses on a feasible (constrained) version, whereas the other work deals with the penalty version in which the feasibility may not be guaranteed. They work under different technical conditions also, e.g., whether or not A(YY^*) defines a smooth manifold.    Additional comments:   * Theorem 1.1: From hindsight, it is clear that the perturbation is to avoid the worst cases so that a result can be stated for all C. But around theorem 1.1, this is not clearly explained. This could be confusing at an initial read.   * Eq (5): After the first (, I think it should be C instead of CY.   * Fig 1: What do you mean by size of the instance for the horizontal axis? It seems confusing if one were to read it against the last paragraph of the numerical experiment section.   * For the phase retrieval experiment, curious to know if the solver always returns approximate rank-one solutions.


Transfer of Value Functions via Variational Methods
We consider the problem of transferring value functions in reinforcement learning. We propose an approach that uses the given source tasks to learn a prior distribution over optimal value functions and provide to an efficient variational approximation of the corresponding posterior in a new target task. We show our approach to be general, in the sense that it can be combined with complex parametric function approximators and distribution models, while providing two practical algorithms based on Gaussians and Gaussian mixtures. We theoretically analyze them by deriving a finite-sample analysis and provide a comprehensive empirical evaluation in four different domains.

Update: ----------- I had a look at the author response: It seems reasonable, contains a lot of additional information / additional experiments which do address my main concerns with the paper. Had these comparisons been part of the paper in the first place I would have voted for accepting the paper.  I am now a bit on the fence about this as the paper could be accepted but will require a major revision, I will engage in discussion with the other reviewers and ultimately the AC has to decide whether such big changes to the experimental section are acceptable within the review process.   Original review: --------------------- The paper presents a method for transfer learning via a variational inference formulation in a reinforcement learning (RL) setting.  The proposed approach is sound, novel and interesting and could be widely applicable (it make no overly restrictive assumptions on the form of the learned (Q-)value function).  The paper is well written and the used terminology is introduced clearly. The paper is set into contextclearly and points out connections to existing theoretical work where appropriate. I quite enjoyed reading the theoretical section.  The proposed solution for the transfer learning approach is simple and appealing, albeit a few open problems remain in my head as to how well the presented approach can work in scenarios where the overlap between target and source domains is small (as the method makes no attempt to directly optimized for learning speed on the target domain).  However, two main concerns remain regarding the empirical evaluation of the proposed approach (see below). Frankly, I am dissapointed how dismissive the authors are of any potential comparison to existing approaches! As a result no clear assesment can be made regarding the usefulness of the presented approach - although the theoretical derivation might be of separate interest (albeit limited in it's use, see comment below). This failure pulls the paper clearly below the threshold for acceptance for me.  Main concerns: 1) The paper does not consider the influence of the initial weights on the convergence speed of the target task (which is e.g. considered in recent meta-learning algorithms). While the approach is sound as presented it begs the question: how would it compare to an approach that is more directly tailored to finding Q-function/policy-parameterizations that are easy to adapt ? The fact that the experimental section makes no attempt at answering this question and, instead, simply dismisses all existing approaches for transfer learning as "not-comparable" without giving any further explanation is concerning! For a NIPS paper this is simply not good enough. At least simple baselines like transferring from a random source task and fine-tuning need to be provided, especially when the considered domains are so simple that running an individual experiment takes on the order of minutes rather than hours. 2) Aside from the main concern described in 1 the considered domains are extremely simple and have huge overlap. State of the art transfer learning approaches nowadays consider more complicated domains (see e.g. [1]). While this is not a major concern given that the approach is interesting enough and novel it does make me wonder whether the method is limited in its applicability.  Other remarks: - Section 3.4 nicely states - and partially addresses - a question I had in my head: "why and how is it valid to use the only sub-differentiable TD-error as the error function for this approach". While this concern is stated precisely - and subsequently addressed by moving towards a mellowmax style objective (which should potentially be mentioned earlier) - the section fails to exactly pinpoint why using the TD-error would not work here, and no experimental or theoretical confirmation is given thereof, we are only presented with a suggestive argument. I would have liked to see more here. - Theorem 1 might be of independent interest and is explained nicely in the main paper. - Theorem 2 is of little practical use as one cannot really assume linearity in the adaptable parameters (unless the distribution over tasks is quite restrictive). Even if the features \phi are expressive (i.e. features extracted from a neural network or the like) transfer via simply adapting the linear weights cannot be expected to work in general (and I believe [1,2] already show examples for failure cases).   Concerns regarding the experiments: 1) I would dispute the claim made in the beginning of the experiments section that: "To the best of our knowledge, no existing transfer algorithm is directly comparable to our approach from an experimental perspective" (l. 254). Sure, no existing algorithms attempts to perform transfer as done in this paper, but clearly you could attempt to transfer via successof features or MAML or any other transfer learning method for RL in the same way: train on whatever data you give to your approach, then evaluate give as much data as collected by your approach on the target domain and see how well transfer occurs. In fact similar domains to your maze navigation and other experiments have been studied in the literature see [1-4]. 2) Similar to point 1: your domains in the transfer experiments overlap heavily! You need to provide a baseline for how well a simple fine-tuning approach (start with parameters from 1 random source domain, fine-tune on target task) would work. I suspect that this would work quite well on the Montain Car task. 3) How can 1-MGVT ever be better than GVT ? Clearly given the parametrization from 3.2 and 3.3 the results should be exactly the same ? The only thing I can explain this result is that you mean 1 component per source task, but then use a mixture over all source tasks for the target task. In which case this should be stated clearly. 4) Are the results averaged over multiple runs or merely wrt. different start positions ? What do the graphs even show ? Mean performance ? Median performance ? 1 std. deviation or 2 or .... ?  Minor errors: l. 24: "and so on" remove this, unecessary. l. 27: "to attempt at" -> "to attempt to characterize" l. 44: "Leveraging on" -> remove on l. 60: is policy deterministic here ? l. 138: "which task *it* is solving* l. 201: proved -> proven l. 203: "such *a* fixed point" l. 236: "in such *a* case" l. 313: "from such a set" Why does the supplementary contain the main paper ? That should be fixed.  Reviewer remark: I have no verified the proof of Theorem 2.  [1] Successor features for transfer in reinforcement learning. André Barreto, Will Dabney, Rémi Munos, Jonathan J Hunt, Tom Schaul, Hado P van Hasselt, and David Silver [2] Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments. Jingwei Zhang, Jost Tobias Springenberg, Joschka Boedecker, Wolfram Burgard [3] Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Chelsea Finn, Pieter Abbeel, Sergey Levine [4] Unsupervised Meta-Learning for Reinforcement Learning. Abhishek Gupta, Benjamin Eysenbach, Chelsea Finn, Sergey Levine


Faithful Inversion of Generative Models for Effective Amortized Inference
Inference amortization methods share information across multiple posteriorinference problems, allowing each to be carried out more efficiently. Generally, they require the inversion of the dependency structure in the generative model, as the modeller must learn a mapping from observations to distributions approximating the posterior. Previous approaches have involved inverting the dependency structure in a heuristic way that fails to capture these dependencies correctly, thereby limiting the achievable accuracy of the resulting approximations. We introduce an algorithm for faithfully, and minimally, inverting the graphical model structure of any generative model. Such inverses have two crucial properties: a) they do not encode any independence assertions that are absent from the model and b) they are local maxima for the number of true independencies encoded. We prove the correctness of our approach and empirically show that the resulting minimally faithful inverses lead to better inference amortization than existing heuristic approaches.

The authors present a novel way to automatically generate the dependency structure for constructing inference network. They frame the problem as a graphical-model-inversion problem and propose to use variable elimination and the min-fill heuristic.  The proposed method generates a faithful and minimal structure for constructing inference network.  The authors show that the generated inference network outperforms the existing heuristically designed network and the fully connected network. This submission is well-written. I enjoying reading this paper.  I have the following questions related to this work.  (1) About the parametric form of variational factors in continuous cases Since the true posterior distribution is intractable, why do we use the same (distribution) parametric form in the inference network as the one in the model? Does the choice of the parametric form play a role in choosing the dependency structure of inference network? If the parametric form is limited, does a fully connected inference network have more expressive power?   (2) For a deep hierarchical Bayesian network, do skip connections play a role in training? Figure 5 shows that skip connections could speed up training at the first epochs.   Why is the minimality important? How about a faithful structure with skip connections from observations to latent variables? For example, we can construct a dependency structure by adding these skip connections into a faithful and minimal structure.


Bayesian Model Selection Approach to Boundary Detection with Non-Local Priors
Based on non-local prior distributions, we propose a Bayesian model selection (BMS) procedure for boundary detection in a sequence of data with multiple systematic mean changes. The BMS method can effectively suppress the nonboundary spike points with large instantaneous changes. We speed up the algorithm by reducing the multiple change points to a series of single change point detection problems. We establish the consistency of the estimated number and locations of the change points under various prior distributions. Extensive simulation studies are conducted to compare the BMS with existing methods, and our approach is illustrated with application to the magnetic resonance imaging guided radiation therapy data.

===Update:======= I downgrade my review to 5. The main concern is  1)  Some more extensive simulations will make the results more convincing, as the numerical experiment is the only way to assess the performance of the proposed priors.   2)  I would recommend more theoretical clarifications on the usage of Bayes factor, which from my understanding will only be selection-consistent under M-closed assumption. It might take a major revision to reflect such comprehensive comparisons.   With that being said, I believe the paper does contain interesting results that are novel and useful to the community. In particular, the theoretical results seem sound, and the paper is fairly readable. But I think there is also room for improvement.  ===Original============ This paper applies Bayesian model selection in the context of boundary detection.  Non-local prior is proved to have a higher order convergence rate of Bayes factor BF(1|0) in hypothesis-testing when the null is true. As a consequence, the proposed algorithm will enjoy a quicker rate towards consistency in boundary detection. Theorem 2 mathematically proves the consistency. The experiments show advantages over all other existing methods.   The structure of the paper is clear and easy to follow. However, I am not extremely convinced why Bayesian model selection is the most valid approach in the context of change point detection. Particularly in the case of multiple change points (the non-trivial situation), the proposed method is more like Bayesian model averaging (BMA), rather than model selection in the sense that multiple models are selected and combined in the end. Why is the marginal evidence a good indicator for the joint?  (Sure, I understand it works in empirical experiments)  Furthermore, I am wondering if it would be worth comparing the proposed methods with other model selection methods other than marginal likelihood (for example, leave-one-out log predictive density, which is not sensitive to the construction of priors, though you probably have to modify LOO a little bit for time series). Or equivalently, others model averaging methods (Bayesian leave-one-out stacking, for instance, has shown advantage over BMA in M-open situation).


Uncertainty-Aware Attention for Reliable Interpretation and Prediction
Attention mechanism is effective in both focusing the deep learning models on relevant features and interpreting them. However, attentions may be unreliable since the networks that generate them are often trained in a weakly-supervised manner. To overcome this limitation, we introduce the notion of input-dependent uncertainty to the attention mechanism, such that it generates attention for each feature with varying degrees of noise based on the given input, to learn larger variance on instances it is uncertain about. We learn this Uncertainty-aware Attention (UA) mechanism using variational inference, and validate it on various risk prediction tasks from electronic health records on which our model significantly outperforms existing attention models. The analysis of the learned attentions shows that our model generates attentions that comply with clinicians' interpretation, and provide richer interpretation via learned variance. Further evaluation of both the accuracy of the uncertainty calibration and the prediction performance with "I don't know" decision show that UA yields networks with high reliability as well.

This paper addresses measuring the uncertainty of model predictions and evaluating the importance of input features. The work proposed to apply dropout and l2 weight decay [7] on the model, and add input dependent noise on stochastic attention model. The evaluation compared the results on electronic health record datasets with conventional approaches that use soft attention and stochastic hard attention (which use input-independent noise). The work also evaluated the quality of feature selection by the attention mechanism through comparing it with human selected features, and shown improvement compared to the soft attention. The authors show that the resulting uncertainty prediction allows their models to filter out more false negative.  Being able to provide uncertainty of model prediction is a critical criterion for medical domain, and the proposed approach provide reasonable improvement over existing approaches. On the other hand, the novel part of this paper is to combine [7] with stochastic attention mechanism and proposed input dependent noise, which has some but limited originality. The paper is clearly written.


Inference Aided Reinforcement Learning for Incentive Mechanism Design in Crowdsourcing
Incentive mechanisms for crowdsourcing are designed to incentivize financially self-interested workers to generate and report high-quality labels. Existing mechanisms are often developed as one-shot static solutions, assuming a certain level of knowledge about worker models (expertise levels, costs of exerting efforts, etc.). In this paper, we propose a novel inference aided reinforcement mechanism that learns to incentivize high-quality data sequentially and requires no such prior assumptions. Specifically, we first design a Gibbs sampling augmented Bayesian inference algorithm to estimate workers' labeling strategies from the collected labels at each step. Then we propose a reinforcement incentive learning (RIL) method, building on top of the above estimates, to uncover how workers respond to different payments. RIL dynamically determines the payment without accessing any ground-truth labels. We theoretically prove that RIL is able to incentivize rational workers to provide high-quality labels. Empirical results show that our mechanism performs consistently well under both rational and non-fully rational (adaptive learning) worker models. Besides, the payments offered by RIL are more robust and have lower variances compared to the existing one-shot mechanisms.

Summary:  In this paper, the authors explore the problem of data collecting using crowdsourcing. In the setting of the paper, each task is a labeling task with binary labels, and workers are strategic in choosing effort levels and reporting strategies that maximize their utility. The true label for each task and workers’ parameters are all unknown to the requester. The requester’s goal is to learn how to decide the payment and how to aggregate the collected labels by learning from workers’ past answers. The authors’ proposed approach is a combination of incentive design, Bayesian inference, and reinforcement learning. At each time step, the authors use Gibbs sampling to help infer the true labels. The estimates are then fed into the reinforcement framework to decide how to choose the payments. Simulated experiments are conducted to evaluate the performance of the proposed approach.  Comments:  On the positive side, while the techniques used in the paper seem pretty standard individually, I like the idea of using Bayesian inference to obtain estimated ground truth for deciding how to set the payments. The overall approach also seems sound and works reasonably well in simulations. On the negative side, I have some concerns about the analysis (more comments below). Using only simulated experiments for evaluations is also not satisfactory (especially a big component of the paper is on learning worker strategies).  Overall, I would still slightly lean towards accepting the paper but won’t argue for it.  I have a few questions about the analysis. In Theorem 1, the authors show that the estimate of P_i (PoBC) will converge. However, the proof seems to treat P_i as a constant while it is actually a strategic choice of agent i (as described in Equation 1).  This is fine if we can show that agents always exert high effort and gives truthful reports, as stated in Theorem 2. However, the proof of Theorem 2 is based on the condition that P_i already converges.  This creates the chicken-and-egg problem (Theorem 1 relies on Theorem 2 to be true, and Theorem 2 relies on Theorem 1 to be true). I might have missed something. It would be helpful if the authors can clarify this.  In Theorem 2 and 3, it is assumed that a_t is larger than some threshold (which is a function of workers’ cost and accuracy). However, in Algorithm 2, there is a probability of \epsilon that a_t will be randomly chosen from A. To satisfy the condition, it seems we need to make all choices in A to be larger than the threshold. Does this mean that the requester needs to know workers’ cost/accuracy in advance?   Overall, the analysis on the three major components seem relatively isolated, while they influence each other. For example, when there is not enough data in the beginning, the Bayesian inference won’t be accurate, could this change workers’ strategies in the beginning and latter influence the follow-up learning (I lean towards believing this won’t happen given the simulation results, however, it is not clear from the current analysis)?  In Figure 2-(a), I am surprised to see the aggregation accuracy of the proposed approach significantly outperforms variational inference even when no noise is added (variational inference has been reported to perform consistently well in the literature). If this result is consistent across many datasets, it could be worth more discussion.  I think the paper would be much stronger if there are some real-world experiments, since real workers are usually not rational (or QR/MWU) and don’t just have two effort levels.


Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization
Despite the success of single-agent reinforcement learning, multi-agent reinforcement learning (MARL) remains challenging due to complex interactions between agents. Motivated by decentralized applications such as sensor networks, swarm robotics, and power grids, we study policy evaluation in MARL, where agents with jointly observed state-action pairs and private local rewards collaborate to learn the value of a given policy. In this paper, we propose a double averaging scheme, where each agent iteratively performs averaging over both space and time to incorporate neighboring gradient information and local reward information, respectively. We prove that the proposed algorithm converges to the optimal solution at a global geometric rate. In particular, such an algorithm is built upon a primal-dual reformulation of the mean squared projected Bellman error minimization problem, which gives rise to a decentralized convex-concave saddle-point problem. To the best of our knowledge, the proposed double averaging primal-dual optimization algorithm is the first to achieve fast finite-time convergence on decentralized convex-concave saddle-point problems.

The article extends previous work of primal-dual optimisation for policy evaluation in RL to the distributed policy evaluation setting, maintaining attractive convergence rates for the extended algorithm. Overall, the article gradually builds its contribution and is reasonably easy to follow. A few exception to this are the start of related work, dropping citations in lists, and the lack of an explanation of the repeatedly mentioned 'convex-concave saddle-point problem'.       The authors equate averaging over 'agents' with averaging over 'space', which is somewhat of an imprecise metaphorical stretch in my view.       The contribution is honestly delineated (collaborative distributed policy evaluation with local rewards), and relevant related work is cited clearly. However, the innovation beyond related work is limited (averaging over agents is added). Empirical results show an improvement over two baselines, but a lag behind SAGA. From my understanding that algorithm has full central information, and is therefore given to indicate a sort of gap between the decentralised solution vs a centralised solver. This is not clear from the text.       Finally, the conclusion is called 'discussions' and has several language mistakes, which together with the shallow comments on the empirical performance puts an unsatisfactory end to this otherwise pleasant to read paper.              Minor remarks:     - 'settings with batch trajectory' -> -ies     - Assumption names like 'A1' are easy to miss, and might better be written out in full. Also, next to the first mention of A1 the citation is empty.     - J(w, \theta) and J(\theta, w) are inconsistently used around Equation 12.     - I could not spot what H1 refers to, when it is used after Equation 13. Maybe A1?


On GANs and GMMs
A longstanding problem in machine learning is to find unsupervised methods that can learn the statistical structure of high dimensional signals. In recent years, GANs have gained much attention as a possible solution to the problem, and in particular have shown the ability to generate remarkably realistic high resolution sampled images. At the same time, many authors have pointed out that GANs may fail to model the full distribution ("mode collapse") and that using the learned models for anything other than generating samples may be very difficult. In this paper, we examine the utility of GANs in learning statistical models of images by comparing them to perhaps the simplest statistical model, the Gaussian Mixture Model. First, we present a simple method to evaluate generative models based on relative proportions of samples that fall into predetermined bins. Unlike previous automatic methods for evaluating models, our method does not rely on an additional neural network nor does it require approximating intractable computations. Second, we compare the performance of GANs to GMMs trained on the same datasets. While GMMs have previously been shown to be successful in modeling small patches of images, we show how to train them on full sized images despite the high dimensionality. Our results show that GMMs can generate realistic samples (although less sharp than those of GANs) but also capture the full distribution, which GANs fail to do. Furthermore, GMMs allow efficient inference and explicit representation of the underlying statistical structure. Finally, we discuss how GMMs can be used to generate sharp images.

Summary  The authors compare the performance of Gaussian mixture models (GMMs) and deep generative models (mainly GANs, but also VAEs) for image generation. They use a mixture of factor analysers (MFA) model and train it on several image data sets, and show that this model can faithfully learn the distribution of images. They also propose a new evaluation scheme for generative models, based on Voronoi cells. They use this scheme to compare MFAs, GANs, and VAEs: MFAs perform pretty well, and avoid mode collapse but produce somewhat blurry images. They propose a scheme based on pix2pix to obtain sharper, "GAN-like" images. Globally, the paper is rather thought-provoking and carries an interesting message.  Quality  Overall, the technical quality of the paper is rather good. Regarding the new evaluation method, I think it makes sense. However, a theoretical assessment (at least superficial) of this method would be welcomed. It is not clear what kind of discrepancy the proposed method measures, while some other popular techniques have clearer interpretations (like the Wasserstein distance, or the test log-likelihood).  The MFA part is rather classical but interesting and natural. More details should be added regarding the choice of the dimension of the latent space and the number of components. For example, would it make sense to use different dimensions for the different components.  Regarding the experiments, I think the authors do a good job at showing that their metric is a good "mode collapse detector". I also really like that they show how to interpret the latent variables of the MFA model. However, I think that a crucial experiment would be welcomed: the comparison of (both training and test) log-likelihoods for their models and the competitors that allow likelihood evaluation . It is possible to evaluate the likelihood of VAEs using importance sampling, and there are other deep generative models of images, like RNVP (Density estimation using Real NVP, Dinh, Sohl-Dickstein, and Bengio, ICLR 2017) that allow direct likelihood evaluation. Since their model is also trained using maximum likelihood, such experiments would be very insightful.  Clarity  I think that the paper reads quite well. A few technical points should be clarified (at least in the supplementary material): - What is precisely meant by "a GMM can approximate any density" (l. 120)? With what metric? A citation should back this claim as well as the one stated just after that ("it is easy to construct densities for which the number of gaussian required grows exponentially"). - How is in-painting done exactly? - How is SGD implemented? The authors say that each component is estimated separately (l. 155), could they explain precisely what this means? What is the learning rate? - The relevance of using L2 Voronoi cells should be discussed more.  Originality  The evaluation method is, to the best of my knowledge, novel. The MFA model is very well known, but using it for image generation is somewhat original (in particular the experiments of Fig. 6b). The pix2pix part introduces also an idea that I've never seen before: first learn roughly the distribution with a technique immune to mode collapse, then refine the samples to produce sharp images.   Significance  I really like the general message of this paper, which, albeit not surprising, is thought provoking. However, I think that such an empirical paper should have much stronger experiments. In particular, I think that some comparisons with the existing metrics (like FID, test log-likelihood, or estimated Wasserstein distance) should be present in the paper. In particular, it is very surprising not to see likelihood comparisons with other likelihood-based models. Moreover, the sharp samples obtained with pix2pix could be compared to GANs samples under the FID or the Wasserstein metric.   Minor comments  - In-painting is sometimes spelled in-painting and sometimes inpainting  - The title is quite misleading, and the analysis should focus more generally on deep generative models (not only GANs)   ------ Post-rebuttal edit ------  I have read the rebuttal, that clarified a few points. My main concern remains the absence of likelihood evaluation for the other techniques. In particular, I think that the likelihood of the VAE trained by the authors should really be compared to the one of the GMM in the final version (for example in the supplementary material). This can be rather easily done using importance sampling, as in many VAE papers. Beyond that, I still think that the overall message is very valuable and that the paper should be accepted.


Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion
Integrating model-free and model-based approaches in reinforcement learning has the potential to achieve the high performance of model-free algorithms with low sample complexity. However, this is difficult because an imperfect dynamics model can degrade the performance of the learning algorithm, and in sufficiently complex environments, the dynamics model will almost always be imperfect. As a result, a key challenge is to combine model-based approaches with model-free learning in such a way that errors in the model do not degrade performance. We propose stochastic ensemble value expansion (STEVE), a novel model-based technique that addresses this issue. By dynamically interpolating between model rollouts of various horizon lengths for each individual example, STEVE ensures that the model is only utilized when doing so does not introduce significant errors. Our approach outperforms model-free baselines on challenging continuous control benchmarks with an order-of-magnitude increase in sample efficiency, and in contrast to previous model-based approaches, performance does not degrade in complex environments.

# Paper ID 5026  Sample-efficient RL with stochastic ensemble value expansion  ## Summary The paper proposes a method to learn a memoryless policy in a sample efficient manner using an ensemble of learned MDP models, policies and Q functions. The main algorithmic idea is a weighted combination of H step temporal differences, estimated on H steps (and rolled out by a learned model of the environment). The underlying idea is to allow the learner to tradeoff between estimation errors in model and Q function in different parts of the state-action space during learning. The updated TD estimator is incorporated into the DDPG algorithm in a straightforward manner. The update is computationally more intensive but the result is improved sample complexity.  The experimental results on a variety of continuous control tasks show significant improvement over the baseline DDPG and a related method (MVE) (which is the precursor to this work).  Overall, the paper is well written. The empirical results are very promising. The analysis and discussion is a bit limited but is not a major drawback. Overall, there is much to like about the paper. Detailed comments follow.  ## Detailed Comments  - The main idea is to replace the vanilla temporal difference update with one that uses simulated trajectories to look H steps ahead (followed by the regular TD update on step H+1). The simulated trajectories are generated from an ensemble of M models. Additional ensembles of reward functions and Q functions are maintained during learning.  - My understanding is that this additional overhead (M models, N reward functions, L Q-functions) is only required during training. At test time, only the final learned Q function (and policy network) is required (right?). The paper could perhaps better describe the difference between training and test.  - In Section 3.1, I'm not sure I completely followed how the mean and variance of the TD estimates at each time step $i$ are computed. I believe the algorithm creates MNL samples on each (s, a) pair visited during training and uses the MNL samples to fit the T_i mean and variance *for that particular (s, a) pair (or (r, s') pair)*. Is this correct? I think Section 3.1 needs to be described much more clearly. A graphical visualization of the N models being rolled out and converted into the final T_i mean and variance estimates may make the learning algorithm easier to visualize.  - Assuming I've understood correctly, the proposed algorithm (STEVE) boils down to using the learned models to generate TD estimates at every step $0 \leq i \leq H$. The TD estimates are noisy (due to model errors) so the final TD estimate is a weighted combination of estimated means of the $T_i$, where the weights are larger when the variance in $T_i$ is small.  - Overall, the approach makes good intuitive sense to me. The authors provide some theoretical justification of the algorithm. A more rigorous analysis in Section 3.2 would strengthen the paper.  - The modified TD update is plugged into the DDPG RL algorithm which uses a pair of neural networks for the Q function and the policy. The paper evaluates the algorithm on a number of continuous control tasks.  - The experimental section shows that the results are quite promising. The results show the STEVE algorithm doing significantly better. The baselines are the vanilla DDPG and the MVE variant of DDPG.  - In Figure 2, I'd be interested in a discussion of the high variance. Perhaps the caption could include more details about the averaging that was performed? Additional baselines (besides DDPG) would also be good to see on the same chart.  - The ablation study description in Section 4.3 was a bit confusing. Why would the "additional parameters of the ensembles" (line 181) matter to the test-time score in Figure 3? This seems to imply that the test time score depends on the ensemble somehow but I don't think that's happening here. What am I missing?  - I found Figure 3 to be very interesting. The performance of STEVE across (H=1/3/5) was particularly interesting as well as the uniform weighting scheme. These results seem somewhat central to the claims of the method. GCP budget permitting, I think the the experimental section would be much improved if these graphs were generated for the other control tasks as well. Any preliminary results on these would be interesting to hear about as well.  - Figure 5 suggests something interesting is happening but I'm not sure I fully understand the discussion in the paper. Why exactly does the model usage go to zero in the least complex AND the most complex environments? I think it might be useful to analyze and describe these three scenarios (going to zero in easy env, going to zero in complex env, hovering around 50%) in more detail. Finally, how are the weights typically distributed across non-zero timesteps?  - Overall, I liked the paper. Additional experiments and baselines, experimental analysis and a clearer description of the algorithm would significantly strengthen the paper.   Minor points:  - I think the ensemble size ought to be mentioned somewhere in the paper and not the appendix.  UPDATE ---------- After reading the other reviews and the author responses, I'm upgrading my evaluation. I congratulate the authors on their fine work.


DeepPINK: reproducible feature selection in deep neural networks
Deep learning has become increasingly popular in both supervised and unsupervised machine learning thanks to its outstanding empirical performance. However, because of their intrinsic complexity, most deep learning methods are largely treated as black box tools with little interpretability. Even though recent attempts have been made to facilitate the interpretability of deep neural networks (DNNs), existing methods are susceptible to noise and lack of robustness. Therefore, scientists are justifiably cautious about the reproducibility of the discoveries, which is often related to the interpretability of the underlying statistical models. In this paper, we describe a method to increase the interpretability and reproducibility of DNNs by incorporating the idea of feature selection with controlled error rate. By designing a new DNN architecture and integrating it with the recently proposed knockoffs framework, we perform feature selection with a controlled error rate, while maintaining high power. This new method, DeepPINK (Deep feature selection using Paired-Input Nonlinear Knockoffs), is applied to both simulated and real data sets to demonstrate its empirical utility.

The paper proposes a method for feature selection in neural networks using a method for a controlled error rate, quantified through the False Discovery Rate. To control FDR the paper is using the model-X knockoffs framework [2, 3, 10]: construct random features that obey distributional properties with respect to the true features, and extract statistics (filters) of pairwise importance measures between true-knockoff dimensions. The choice of the importance function and the knockoff filters is flexible. The novelty of this paper lies in using a neural network (MLP) to get the importance measure through a linear layer that couples the true and knockoff features pairwise. The final statistic depends both on the trainable linear layer weights and the rest of the network weights.   Results are provided on real and synthetic data: for the latter (noisy linear and nonlinear processes with increasing number of dimensions/features), the proposed method performs better (in terms of FDR control and power or feature relevance certainty) compared to MLP, RF and SVMs. Notably, compared to MLP the difference comes from the coupling initial, linear layer. The paper also compares feature selection to other knockoffs-related methods ([2], [4], [10]), on a real dataset for mining interpretable dimensions features (HIV-1) and provide qualitative analysis on the discovered features from gut microbiome data.    General comments: Feature selection is an important topic related to interpretability, transparency and robustness in neural network models (and machine learning more general) and the paper is an effort in that direction. The framework is an extension of the model-X knockoff framework [10] and the novelty lies in adding a hard-wired, linear, pairwise-connected layer on the knockoff features before an MLP. The benefit is higher statistical power for feature relevance. I like the idea of coupling feature weights and algorithm weights in a single importance measure, which ends up ranking a feature j high if both the filter coefficients difference and the NN output dimension j is high. The paper has a clear exposure of ideas and background (e.g. on the knockoff statistics models provided) and the paper is easy to follow from motivation, to background to empirical evaluation. My concerns with the paper are mainly around: a) the amount of novelty (and generality) on the proposed coupling/pairwise layer and b) the broader applicability of the proposed method.   Strengths: - The framework is an application/extension of an existing statistical framework for error control with theoretical guarantees [10]. - The coupling pairs + parametrization idea is generic and could be applied to different types of algorithms (as a learned representation layer).  - The paper is well written, reads smoothly and the ideas are well exposed (supported, explained or cited).  Weaknesses:  - It would help to have some element of theoretical analysis on the chosen definition of the importance measures Z (Sec. 3). For example, deriving this as an explicit regularizer on the layer weights. - The framework (proposed as a general feature selection method?), is restricted to Gaussian design and the use of an estimated precision matrix, for which there might not be any robustness guarantees. Some discussion (or paths for alternative designs?) might be helpful.  - There is some disconnect, in my opinion, between motivating this work as a feature selection method for NN and using a NN as a way to measure importance (given a prediction task). - The empirical validation, even though convincing in some respect: the benefit of the novelty here (i.e. adding a pairwise connected layer vs. having a vanilla MLP) is only explored on the synthetic data.   Specific comments: - The method is motivated by the need for interoperability in NN and error control in NN, though only the latter is explored (in a synthetic scenario).  - Similarly, the title states “reproducible” feature selection and this might something that needs further explaining. In what sense reproducible?  - Did the authors explore/consider different definitions of the importance measures  on the knockoff statistics?  - What is the dependency of some aspects of the performance on the knock-off feature generation (e.g., s for the Gaussian case)? Is there a way to make vanilla MLP + knockoff features better by changing (or tuning) s? - Is there a specific rationale for fixing the number of neurons in the hidden layer to be p (equal to the input dimension)? As long as the output layer is p-dimensional the reasoning should still hold with a different ‘factorization’ of the input to output mapping that defines w. The same holds for using L1 regularization (and with a specific regularization parameter). - How dependent on the specific parametrization of the MLP (or NN more general) is the performance?  - The FDR target error level is fixed/set to q=0.2 everywhere. Would it be insightful to vary this and re-interpret, e.g. the results of Sec. 5.1? - It would be helpful to see some prediction results with real data, based on the selected features. E.g. a two-pass process where the NN is re-trained on the selected, error-control features to predict the target function values or class.   - How much False Discovery errors are relevant for NN performance in practical applications (and in high-dimensions)? An open question that might be worth discussing. - Did the authors try the knockoffs+vanilla MLP approach on the real data in 5.1 (or the qualitative analysis in 5.2)? The pairwise-connected layer is the novelty of this paper and this should be further highlighted.   =========================  Comments after author feedback:  Thank you for the replies to all reviews and most of the comments. I agree with the points of your clarifications and can see the value of the points made in the response. I also agree with Reviewer 1 that this can inspire future work by the authors and others. I am changing my score to a 6 to reflect these. At the same time my main concern on general applicability remains, and in some cases has been strengthened by some of the responses.   - “in the sense that small changes in the data can yield dramatic changes in the selected features. “ If that is the case, perturbations in the data (and thus reproducibility of the framework) are not explored. What type of changes?   - “We consider the primary virtue of the robustness to be improved reproducibility of the selected features;” As above. I don’t feel that this aspect of the authors’ work is highlighted or supported experimentally in the paper.   - “Given the universal approximation property of DNNs, practitioners can now avoid having to handcraft feature importance measures with the help of DeepPINK.” This is a good point, thank you for making this connection explicit. Is the addition of DeepPINK layer however ‘altering’ the network definition, i.e. are you selecting features for the original network or the modified one? How dependent is this selection on initialization, parametrization, and as the authors state, changes in the input data (e.g. transformations, subsampling, noise etc).  - “using interpretability alone as a measurement is far less reliable than using FDR and power with synthetic data as measurements.”  This is also a good point, and provides statistical guarantees (as Reviewer 1 also points out).   - “it is the fundamental architecture (instead of the tuning) of vanilla MLP that prevents the knockoff filter from performing well.” I am not sure what this claim implies. That MLP is not a good choice for being subjected to knockoff filters? This defies the purpose of the paper. Or that knockoffs can only add value coupled in the learning process (like for DeepPINK)? I wonder, and I made the same point in my initial review on which the authors replied, how much those findings are generally applicable to any ‘neural network’ parametrization or just MLP-type. How about basic linear or (shallow) non-linear regression with the PINK coupling and solved through SGD?   - “Since there is no existing method available for robust feature selection in DNNs, any effective methods would be significant.” I agree. However the idea of input features and feature representation (hidden layers) is coupled in NNs. As the authors point out, interpretability and selection is conditioned on the prediction algorithm. So this is not ‘feature selection’, but input dimension selection. One could envision however a PINK layer attached on higher hidden layers for representation dimension selection, interpretability and transferability.   - “The proposed network architecture may seem simple, but our journey to finding it was nontrivial.” So then does the paper involve a single, fixed architecture? Or is it generally applicable for any parametrization that one can add on top of the pairwise linear layer?


Densely Connected Attention Propagation for Reading Comprehension
We propose DECAPROP (Densely Connected Attention Propagation), a new densely connected neural architecture for reading comprehension (RC). There are two distinct characteristics of our model. Firstly, our model densely connects all pairwise layers of the network, modeling relationships between passage and query across all hierarchical levels. Secondly, the dense connectors in our network are learned via attention instead of standard residual skip-connectors. To this end, we propose novel Bidirectional Attention Connectors (BAC) for efficiently forging connections throughout the network. We conduct extensive experiments on four challenging RC benchmarks. Our proposed approach achieves state-of-the-art results on all four, outperforming existing baselines by up to 2.6% − 14.2% in absolute F1 score.

This paper proposes a new neural architecture for reading comprehension.  Compared to many other existing neural architectures, this model 1) densely connects all pairs of passage layers and question layers for encoding; 2) uses a component called Bidirectional Attention Connectors (BAC)  for connecting any P-layer and Q-layer which employs an FM layer on top of commonly used bi-directional attention.  The proposed architecture has been evaluated on four reading comprehension datasets and demonstrates strong empirical results.  Overall, although the proposed ideas could be potentially interesting, I think the presented results (in the current presentation format) are not convincing enough.   - I am not sure how the four datasets were chosen, but this model is not evaluated on the most competitive datasets: SQuAD and (arguably) TriviaQA. This makes the results less convincing.  - In general, I find it is difficult to understand the influence of the proposed components in this paper. According to the ablation study in Table 6, I don’t see any of the components contributed that much to the overall performance and whether the results can generalize to other datasets.  I think it is necessary to have more ablation studies on other datasets and apparently, this model has also improved more on other datasets. Gated attention is not novel for this paper. How useful are these dense connections are indeed? How useful is the FM layer (if we just replace function G with a feedforward network)? I think it is important to highlight the new components of this paper and their contributions to a reading comprehension system (on various datasets).  - I think the paper could have done a better job explaining the intuitions of the proposed solutions. The paper made a few claims in the introduction part such as “densely connected by attention” and “compress the attention outputs so that they can be small enough to propagate” but it is unclear how they are reflected in the model design.  - The notions in this paper are also confusing.      - Line 76 and equation (3): how are b_i and p_i defined?     - How are Z* used/propagated in the following layers?  ========= I have carefully read all other reviews and the authors' rebuttal. Given the authors provided results on SQuAD and comparisons of FM vs feedforward networks, I increased my score from 5 to 6.


Modular Networks: Learning to Decompose Neural Computation
Scaling model capacity has been vital in the success of deep learning. For a typical network, necessary compute resources and training time grow dramatically with model size. Conditional computation is a promising way to increase the number of parameters with a relatively small increase in resources. We propose a training algorithm that flexibly chooses neural modules based on the data to be processed. Both the decomposition and modules are learned end-to-end. In contrast to existing approaches, training does not rely on regularization to enforce diversity in module use. We apply modular networks both to image recognition and language modeling tasks, where we achieve superior performance compared to several baselines. Introspection reveals that modules specialize in interpretable contexts.

The paper is concerned with conditional computation, which is an interesting topic yet at early stages of research, and as such one that requires much research and investigation. The paper proposes a latent-variable approach to constructing modular networks, modeling the choice of processing modules in a layer as a discrete latent variable.  A modular network is composed of L modular layers, each comprised of M modules and a controller. Each module is a function (standard layer) f_i(x; \theta_i). The controller accepts the input, chooses K of the M modules to process the input, and outputs the  as the module output. Modular layers can be stacked, or placed anywhere inside a standard network. The input/output of a modular layer is a tensor, and as such the layer is easily integrated with standard networks. Interestingly, the paper proposes a controller that outputs a distribution over compositions of modules (i.e., assigns a probability to each of the (M choose K) possible compositions). This is different from Shazeer et al., who assign a probability to each module and choose the K most likely.  The key issue is a training/inference procedure, which is complicated due to the nature of the discrete latent variable. The authors propose trianing the model with Viterbi EM, optimizing a lower bound by introducing a q distribution over $a$, the choice of modules. The authors also mention REINFORCE and noisy top-K as options for training the model, though note that these require additional regularization in order to not collapse modules.   Pros: - Key contribution: conditional computation layer that selects K of M modules per layer by assigning probability to the complete set (more general than greedy assignment as in Shazeer, et al., 2017), and training with EM explicitly leverages sparse computation (as opposed to MOE). - Simple, but powerful, modular layer for conditional computation that can be stacked and assembled in a straight-forward manner. - Well justified training procedure using EM to optimize a lower-bound over the marginal likelihood of y. - Experiments on Penn TreeBank and Cifar100 imply that the method is scalable to larger('ish) datasets.  Cons: ----- - Contribution is somewhat incremental in that the the core idea was introduced by Shazeer et al. 2017 - Related work is somewhat sparse. - Biggest issue is the experimental results: the model does not work well on either benchmark at all: test accuracy was very low and perplexity very high compared to standard-compute models, though these are not shown in the paper. As the authors mention, there is yet a lot of work to make conditional-compute models viable.  My overall impression is that this is a good paper that should be accepted at NIPS. The paper is clearly written, and easy to read. It links nicely to existing work in the field, clearly stating its contributions. The model design and derivation of the training procedure are clearly explained and justified. The paper makes a reasonable contribution in the area of conditional-compute by suggesting distributions over K modules per layer rather than softmax with top-K. This enables the use of a reasonable q distribution and generlized EM for training. Despite the weak experimental results, the paper represents a step in the direction for conditional neural computation.


On the Local Minima of the Empirical Risk
Population risk is always of primary interest in machine learning; however, learning algorithms only have access to the empirical risk. Even for applications with nonconvex nonsmooth losses (such as modern deep networks), the population risk is generally significantly more well-behaved from an optimization point of view than the empirical risk. In particular, sampling can create many spurious local minima. We consider a general framework which aims to optimize a smooth nonconvex function F (population risk) given only access to an approximation f (empirical risk) that is pointwise close to F (i.e., F − f ∞ ≤ ν). Our objective is to find the -approximate local minima of the underlying function F while avoiding the shallow local minima-arising because of the tolerance ν-which exist only in f . We propose a simple algorithm based on stochastic gradient descent (SGD) on a smoothed version of f that is guaranteed to achieve our goal as long as ν ≤ O( 1.5 /d). We also provide an almost matching lower bound showing that our algorithm achieves optimal error tolerance ν among all algorithms making a polynomial number of queries of f . As a concrete example, we show that our results can be directly used to give sample complexities for learning a ReLU unit.

This paper considers the problem of minimizing a non-convex smooth population risk function, where one has access to a 0-th order oracle that can evaluate the empirical risk. With this goal, the authors propose an algorithm (ZPSGD) which is essentially an instance of PSGD [Ge COLT 2015] on a Gaussian smoothing of the empirical loss. Under the standard regularity conditions of the population risk (boundedness, gradient, and Hessian Lipschitzness), and additional pointwise closeness of the empirical risk to the population risk, this paper shows the following results: 1. if empirical and population risk are (eps^1.5)/d pointwise close, then ZPSGD returns an eps-approximate second order stationary point of the population risk with a number of 0-th order queries polynomial in the problem parameters. 2. the (eps^1.5)/d pointwise closeness in optimal up to polylogarithmic factors for all algorithms that make a polynomial number of 0-th order queries. 3. if we let an exponential number of 0-th order queries, there is an algorithm for finding eps-approximate second order stationary point if the empirical and population risk are eps^1.5 pointwise, and this dependence is optimal.  General Evaluation: The paper considers an important problem at the intersection of machine learning and non-convex optimization. The paper is well-written, and the contributions are clear. The mathematical derivations and the proofs seem correct (I have not checked all the proofs). My main concern is about the novelty of the paper. ZPSGD is essentially SPGD on a Gaussian smoothing of the empirical loss. The fact that unbiased estimates of the gradient of the Gaussian smoothing of a function can be obtained by function evaluations on a batch is not novel [Duchi ToIT 2015]. Convergence of PSGD when unbiased stochastic gradients have sub-Gaussian tails is also established in [Jin PC 2018]. So the main novelty of the paper is in setting up the smoothing radius such that together with the other parameters, guarantees of PSGD can be carried over to the Gaussian smoothing of the empirical loss, and that the population risk and its gradients and Hessians are close to those of the Gaussian smoothing of the empirical risk. Despite this, I still believe this paper makes important contributions that makes it a good fit for NIPS.   =============================  I have carefully read the author response. It is not clear to me why a zeroth order oracle is interesting. Authors fail to motivate it from a computational perspective, and zeroth order oracles are not typical in machine learning. More importantly, if one does have access to a first-order oracle, then the results presented here are either well known or follow as simple corollaries to the existing results. So what is the significance of the work here? Secondly, somewhat equally important is the fact that the authors analyze the ZPSGD as an inexact gradient method. The proofs are rather straightforward, and again it is not clear what are the contributions from a technical perspective (no new tools/techniques being contributed). I still think it is a good paper and agree that the lower bound results are novel.


Invertibility of Convolutional Generative Networks from Partial Measurements
The problem of inverting generative neural networks (i.e., to recover the input latent code given partial network output), motivated by image inpainting, has recently been studied by a prior work that focused on fully-connected networks. In this work, we present new theoretical results on convolutional networks, which are more widely used in practice. The network inversion problem is highly non-convex, and hence is typically computationally intractable and without optimality guarantees. However, we rigorously prove that, for a 2-layer convolutional generative network with ReLU and Gaussian-distributed random weights, the input latent code can be deduced from the network output efficiently using simple gradient descent. This new theoretical finding implies that the mapping from the low-dimensional latent space to the high-dimensional image space is one-to-one, under our assumptions. In addition, the same conclusion holds even when the network output is only partially observed (i.e., with missing pixels). We further demonstrate, empirically, that the same conclusion extends to networks with multiple layers, other activation functions (leaky ReLU, sigmoid and tanh), and weights trained on real datasets.

This paper presents a new theoretical result on the invertibility of convolutional neural networks. The presented result is demonstrated (thoroughly in supplementary material) for a two-layer network, and claims are made about the empirical validity of the result on deeper networks. Invertibility is here intended as the recovery of latent code given partially observed network output (i.e. missing pixels), in the context of convolutional generative network. I think the direction of this paper's study is very interesting to better understand neural networks.  The theoretical result presented shows that the mapping from the latent code space to the network output (i.e. the image space) is bijective. This result is demonstrated only for a two-layer convolutional generative network with ReLU, while the general case is let as future work. Could you clarify the choice of a two-layer network? Is a two-layer network more tractable/significantly easier to analyze with respect to, for example, a three-layer network? What is the main difficulty in proving a more general result on deeper networks? Line 182 of the paper says: "This proof can be extended to deeper networks with the same machinery."  Could you clarify why you can only provide empirical evidence for the deeper network case? Why was the general result difficult to include in the paper? And if there is a specific difficulty, how would you suggest to address it? The same reasoning applies to the different activation functions: what was the main challenge in proving the results for different activation functions, and how would you address it? I think it would be important to clarify these points in the paper.   The proof of the main result mostly follows arguments in the archival paper “Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk” by Hand and Voroninski (cited), with some extensions to deal with the permutation applied to the weight matrices. While it seems that not much literature tackles this invertibility problem from the same angle, the cited “Towards Understanding the Invertibility of Convolutional Neural Networks” by Gilbert et al. (2017) is a relevant study, as well as other compressive sensing methods (e.g. the two cited works [4,5]).  How does the proposed approach compare with those works? A quantitative evaluation of the proposed solution, and comparative experiments with the other relevant works are missing. I would like to see a quantitative evaluation of the proposed solution on the experiments presented. I would also like to see how the proposed solution compares with other relevant approaches.  The paper is generally well written and pretty readable. Some claims seem generic, e.g. “we exploit the hierarchical nature of images and other natural signals by leveraging the powerful representation capability of deep learning.” (lines 63-65).  Could you clarify how this is supported by the rest of the paper? For example, only a two-layer network is considered for the proof of the main result, and no guarantees are provided for the validity of the result for deeper networks apart from empirical results (which are, in my opinion, not sufficient to support this claim).   Could you clarify how the experiments presented allow to see that the latent code was successfully recovered? I can only see reconstruction results, but there is no direct comparison between the “actual” latent code and the recovered one (which seems to be the point of this study).  One of the assumptions of the proposed approach is the Gaussian weight assumption. How restrictive is this assumption in practice? Could you provide examples where this assumption would not hold?   It is claimed that the proposed result "provides a sufficient condition to avoid mode collapses" (line 195). However this claim is not supported by experimental results. I would like to see, for example, an experiment where the mode collapse happens if the proposed solution is not applied.  Could you clarify how the permutation applied to the weight matrices affects the geometry of the latent space? How does this operation effectively act on the latent code representation? Can the permutation operation be interpreted as a way to disentangle features in the latent code? Are there information losses or computational costs associated to it?  After reading the author rebuttal, I tend to vote for the acceptance of this paper.


Deep Generative Models with Learnable Knowledge Constraints
The broad set of deep generative models (DGMs) has achieved remarkable advances. However, it is often difficult to incorporate rich structured domain knowledge with the end-to-end DGMs. Posterior regularization (PR) offers a principled framework to impose structured constraints on probabilistic models, but has limited applicability to the diverse DGMs that can lack a Bayesian formulation or even explicit density evaluation. PR also requires constraints to be fully specified a priori, which is impractical or suboptimal for complex knowledge with learnable uncertain parts. In this paper, we establish mathematical correspondence between PR and reinforcement learning (RL), and, based on the connection, expand PR to learn constraints as the extrinsic reward in RL. The resulting algorithm is modelagnostic to apply to any DGMs, and is flexible to adapt arbitrary constraints with the model jointly. Experiments on human image generation and templated sentence generation show models with learned knowledge constraints by our algorithm greatly improve over base generative models.

Summary: The paper proposes a way to incorporate constraints into the learning of generative models through posterior regularization. In doing so, the paper draws connections between posterior regularization and policy optimization. One of the key contributions of this paper is that the constraints are modeled as extrinsic rewards and learned through inverse reinforcement learning.  The paper studies an interesting and very practical problem and the contributions are substantial. The writing could definitely be made clearer for Sections 3 and 4, where the overloaded notation is often hard to follow.  I have the following questions: 1. Since generative models can be easily sampled from, why not introduce an additional term E_p[f(x)] and minimize L(theta) - E_p[f(x)]. The only explanation I find for introducing q in the paper is  “for efficient optimization”. I don’t see theoretically or empirically any reason why the optimization will suffer without q.         When f is learned as well, this reduces to energy-based GAN setting discussed in 210-221. I would be curious to see theoretical and/or empirical evidence in support of the  approach proposed in the paper.  2. L169-171 needs more clarification. For a self contained paper, the link between q_phi* and q_pi* seems like a key detail that’s missing from the paper.  In lines 149-156, the paper establishes a link between q* and q_pi*. In Eq. (8) however, they substitute q_phi with the expression for q*. What’s the justification?  3. Bayesian optimization over the latent space of a generative model is another approach to generate objects with desired properties. It has successfully been applied for generating molecules [1, 2].  Would be good to include a discussion on the merits of either approaches.  With regards to the experiments, it’s hard to comment on the significance of the results since the paper claims that the baselines from prior work are not directly comparable (which I agree). I believe energy-based GANs (as mentioned in pt.1 above)  and potentially Bayesian optimization techniques (if applicable) would have been good baselines to include.  [1] R. Gómez-Bombarelli, D. Duvenaud, J. M. Hernández-Lobato, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. arXiv preprint arXiv:1610.02415, 2016.   [2] M. J. Kusner, B. Paige, and J. M. Hernández-Lobato. Grammar variational autoencoder. arXiv preprint arXiv:1703.01925, 2017.   --- --- Thanks for the response. I would highly encourage the authors to include results for the energy based models in the final version, since I believe that to be the closest baseline to this work.


Unsupervised Learning of Shape and Pose with Differentiable Point Clouds
We address the problem of learning accurate 3D shape and camera pose from a collection of unlabeled category-specific images. We train a convolutional network to predict both the shape and the pose from a single image by minimizing the reprojection error: given several views of an object, the projections of the predicted shapes to the predicted camera poses should match the provided views. To deal with pose ambiguity, we introduce an ensemble of pose predictors which we then distill to a single "student" model. To allow for efficient learning of high-fidelity shapes, we represent the shapes by point clouds and devise a formulation allowing for differentiable projection of these. Our experiments show that the distilled ensemble of pose predictors learns to estimate the pose accurately, while the point cloud representation allows to predict detailed shape models.

This paper introduces an approach to learning to predict 3D shape with 2D images only as input. The core idea is to make use of a 3D points-based representation of shape and to additionally predict the pose of the observed object, so as not to require this information during training.  In general, the paper is clearly written and does not suffer from any major flaw. My main concern is that I find the contribution of this paper a bit weak for NIPS. See detailed comments below.  Novelty: - As acknowledged by the authors, there has been an increasing interest in learning to predict 3D shape from 2D information only. In this context, [6] and [11] also represent shape as a 3D point cloud. Furthermore, [13] and [20] also work in the setting where pose is not available during training. I find this to limit the contributions of this submission. - I acknowledge that there can be different approaches to solving the same problem, which could be the focus here. However, the proposed method does not seem to be of great originality, essentially defining a network that directly outputs 3D points and pose. I might be missing something, but currently the paper fails to convince me of the novelty of the method.  Method: - Section 4 discusses the notion of differentiable point clouds, which is achieved by converting the point cloud to a volume. Considering that [6] and [11] have also worked with point cloud representations within deep networks, I do not see why this different strategy is necessary or beneficial. I would appreciate if the authors could clarify this. - When predicting a point cloud, the order of the points in the set is irrelevant. This suggests that there are ambiguities in what the network can predict. I was wondering if the authors have observed problems related to these ambiguities, or if at least they could discuss this issue.  Experiments: - In Section 5.2, it would be interesting to compare with [11], which also uses a point-cloud representation and works with known pose. - In Section 5.3, it would be interesting to compare with [13], which also handles the unknown pose scenario. - For the classification metric, 30 degrees seems like a very large threshold. It would be worth evaluating what happens with smaller thresholds. - In practice, at test time, is there any reason to also predict the pose? It seems it is mostly the shape that is of interest. - In Section 5.3, the authors report results of Ours-basic and Ours, referred to as the full model. Is Ours what was described as "fast" in Section 4.1? If so, considering that it performs better than Ours-basic, is there any value in the basic algorithm? From Section 4.1, it is supposed to be more flexible, but the quantitative experiments don't really reflect this (although Fig. 5 briefly illustrates this).  Related work: - It might be worth citing Cohen & Welling, 2014, "Transformation Properties of Learned Visual Representations", and Worrall et al., ICCV 2017, "Interpretable transformations with encoder-decoder networks". These methods also exploit a point-based representation but in the context of novel view synthesis. - Rhodin et al., CVPR 2018, "Learning Monocular 3D Human Pose Estimation From Multi-View Images" also make use of multiview data with unknown pose 3D human pose estimation.