Evolution-Guided Policy Gradient in Reinforcement Learning
Deep Reinforcement Learning (DRL) algorithms have been successfully applied to a range of challenging control tasks. However, these methods typically suffer from three core difficulties: temporal credit assignment with sparse rewards, lack of effective exploration, and brittle convergence properties that are extremely sensitive to hyperparameters. Collectively, these challenges severely limit the applicability of these approaches to real-world problems. Evolutionary Algorithms (EAs), a class of black box optimization techniques inspired by natural evolution, are well suited to address each of these three challenges. However, EAs typically suffer from high sample complexity and struggle to solve problems that require optimization of a large number of parameters. In this paper, we introduce Evolutionary Reinforcement Learning (ERL), a hybrid algorithm that leverages the population of an EA to provide diversified data to train an RL agent, and reinserts the RL agent into the EA population periodically to inject gradient information into the EA. ERL inherits EA's ability of temporal credit assignment with a fitness metric, effective exploration with a diverse set of policies, and stability of a population-based approach and complements it with off-policy DRL's ability to leverage gradients for higher sample efficiency and faster learning. Experiments in a range of challenging continuous control benchmarks demonstrate that ERL significantly outperforms prior DRL and EA methods.

In this paper, the authors combine an evolutionary algorithm with a Deep RL algorithm so that the combination achieves the best of both worlds. It is successfully applied to 6 standard mujoco benchmarks.  The following part of this review has been edited based on the other reviewers points, the authors rebuttal, an attempt to reproduce the results of the authors,  and investigations in their code.  First, investigations of the results and code revealed several points that are poorly documented in the paper. Here is a summary:  * DDPG:     - The hidden layer sizes in both the Actor and the Critic in the implementation are different from those mentioned in the paper. This is a problem since ddpg's performance can vary a lot with the architecture of the nets.     - The Actor uses tanh non-linearities, and the  Critic elu non-linearties. This is not mentioned.     - The norm of the gradient of the Critic is clipped. This is not mentioned.  * Evolutionary Algorithm:     - Tournament selection is performed with tournament size of 3, and is done with replacement.     - Individuals selected during tournament selection produce offspring with the elite through crossover until the population is filled.     - The way mutations are handled is more complex that what is mentioned in the paper and involves many hyper-parameters. For each non elitist individual there is a fixed probability for his genome to mutate. If the genome      mutates, then each weight of the actor can mutate in 3 different ways. "Normal" mutations involve adding a 10% Gaussian noise; "Super" mutations involve adding 100% Gaussian noise, and "Reset" mutations involve resetting the weight using a normalized center Gaussian noise. It seems that on average, around 10% of the weights of the Actor that undergoes a mutation are changed, but some parts of the code are still obscure. The whole mutation process is really messy and deserves more attention since it is supposed to account for half of the success of the method (the other half being the deep RL part).   The authors should definitely put forward all the above facts in their paper as some of them play an important role in the performance of their system.  About performance itself, in agreement with other reviewers, I consider that the authors should not base their paper on state-of-the-art performance claims, but rather on the simplicity and conceptual efficiency of their approach with respect to alternatives. I also agree with other reviewers that the title is too generic and that the approach should be better positionned with respect to the related literature (Baldwin effect etc.).  For the rest, my opinion has not changed much so I keep the rest of the review mostly as is.  The idea is simple, the execution is efficient, and the results are compelling (but see the remarks about reproducibility above). I am in favor of accepting this paper as it provides a useful contribution. Below I insist on weaknesses to help the authors improve their paper.  A first point is a lack of clarity about deceptive reward signals or gradients. The authors state that the hard inverted pendulum is deceptive, but they don't explain what they mean, though this matters a lot for the main message of the paper. Indeed, ERL is supposed to improve over EA because it incorporates gradient-based information that should speed up convergence. But if this gradient-based information is deceptive, there should be no speed up, in contradiction with results of ERL in "deceptive reward" tasks. I would be glad to see a closer examination of ERL's behaviour in the context of a truly deceptive gradient task: does it reduce to the corresponding EA, i.e. does it consistently reject the deep RL policy until a policy that is close enough to the optimum has been found? In that respect, the authors should have a look at the work of Colas et al. at ICML 2018, which is closely related to theirs, and where the effect of deceptive gradients on deep RL is discussed in more details.  Related to the above, details are missing about the tasks. In the standard inverted pendulum, is the action discrete in {-1,0,1} or continous in [-1,1] or something else? What is the immediate reward signal? Does it incorporate something to favor smaller actions? Is there a "success" state that may stop a trial before 1000 steps? All these details may make a difference in the results. The same kind of basic facts should also be given about the other benchmarks, and the corresponding details could be rejected into the appendices.  The methodology for reported metrics has nice features, but the study could be made more rigorous with additional information: how many seeds did you use? How do you report variance? Can you say something about the statistical significance of your results?  The third weakness is in the experimental results when reading Fig. 3. For instance, results of DDPG on half-cheetah seem to be lower than results published in the literature (see e.g. Henderson et al. "Deep RL that matters"). Could the authors investigate why (different implementation? Insufficient hyper-parameter tuning... ?)  The fourth weakness is the related work section. First, the authors should read more about Learning Classifier Systems (LCSs), which indeed combined EAs and RL from the very start (Holland's 1975 work). The authors may read Lanzi, Butz or Wilson's papers about LCSs to take more distance about that. They should also take more distance about using EAs to obtain RL algorithms, which is now an important trend in Meta-RL at the moment. More generally, in a NIPS paper, we expect the "related work" section to provide a good overview of the many facets of the domain, which is not the case here. Finally, the last sentence "These approaches are complementary to the ERL framework and can be readily combined for potential further improved performance." is very vague and weak. What approach do the authors want to combine theirs with and how can it be "readily combined"? The authors must be much more specific here.  In Section 2.2 it should be stated more clearly which EA is used exactly. There are many families, most algorithms have names, from the section we just get that the authors use an EA. Later on, more details about mutation etc. are missing. For instance, how do they initialize the networks? How many weights are perturbed during mutation?  The authors may also explain why they are not using an ES, as used in most neuroevolution papers involved in the competition to deep RL (see numerous "Uber labs deep-neuroevolution papers"), why not NEAT, etc. Actually, it took me a while to realize that an ES cannot be used here, because there is no way to guarantee that the deep RL policy will comply with the probability distribution corresponding to the covariance matrix of the ES current population. Besides, ESs also perform a form of approximate gradient descent, making the advantage of using deep RL in addition less obvious. All this could be discussed.  I would also be glad to see a "curriculum learning" aspect: does ERL start using EAs a lot, and accepts the deep RL policy more and more along time once the gradient is properly set?  Note also that the authors may include policies obtained from imitation within their framework without harm. This would make it even richer.  Finally, the paper would be stronger if performance was compared to recent state-of-the-art algorithms such as SAC (Haarnoja), TD3 (Fujimoto) or D4PG (Horgan), but I do not consider this as mandatory...  More local points.  l. 52: ref [19] has nothing to do with what is said, please pick a better ref.  l. 195-201: this is a mere repetition of lines 76-81. Probably with a better view of related work, you can get a richer intro and avoid this repetition.  p.2: "However, each of these techniques either rely on complex supplementary structures or introduce sensitive parameters that are task-specific": actually, the author's method also introduce supplementary structures (the EA/the deep RL part), and they also have task-specific parameters...  The authors keep a standard replay buffer of 1e^6 samples. But the more actors, the faster the former content is washed out. Did the authors consider increasing the size of the replay buffer with the number of agents? Any thoughts or results about this would be welcome.  l.202 sq. (diverse exploration): the fact that combining parameter noise exploration with action exploration "collectively lead to an effective exploration strategy" is not obvious and needs to be empirically supported, for instance using ablative studies about both forms of exploration.  typos:  I would always write "(whatever)-based" rather than "(whatever) based" (e;g. gradient-based). Google seems to agree with me.  l. 57: suffer with => from l. 108: These network(s) l. 141: gradients that enable(s) l. 202: is used (to) generate l. 217: minimas => minima (latin plural of minimum) l. 258: favorable => favorably? l.266: a strong local minima => minimum (see above ;))  In ref [55] baldwin => Baldwin.


Representation Learning of Compositional Data
We consider the problem of learning a low dimensional representation for compositional data. Compositional data consists of a collection of nonnegative data that sum to a constant value. Since the parts of the collection are statistically dependent, many standard tools cannot be directly applied. Instead, compositional data must be first transformed before analysis. Focusing on principal component analysis (PCA), we propose an approach that allows low dimensional representation learning directly from the original data. Our approach combines the benefits of the log-ratio transformation from compositional data analysis and exponential family PCA. A key tool in its derivation is a generalization of the scaled Bregman theorem, that relates the perspective transform of a Bregman divergence to the Bregman divergence of a perspective transform and a remainder conformal divergence. Our proposed approach includes a convenient surrogate (upper bound) loss of the exponential family PCA which has an easy to optimize form. We also derive the corresponding form for nonlinear autoencoders. Experiments on simulated data and microbiome data show the promise of our method.

This paper suggests a generalization of PCA that is applicable to compositional data (non-negative data where each sample sums to 1 or to 100%). It builds up on exponential family PCA developed in the early 2000s. There have been multiple more or less ad hoc approaches of transforming compositional data to make it amenable to PCA, and a more principled method of compositional PCA would certainly be an interesting development.  The paper is reasonably well written and uses interesting mathematics. However, it (a) lacks convincing examples, (b) is somewhat confusing in places. I put it above the acceptance threshold, hoping that the authors would be able to improve the presentation.  Caveat: I am not familiar with Bregman divergences and do not know the [22] paper that this one heavily uses. I cannot therefore judge on the details of the math.  MAJOR ISSUES  1. When reading the paper, I was waiting for some cool visualizations of real-life data. I was disappointed to find *no* visualizations in Section 5.2. Actually the text says "visualizations of the two main axes..." (line 233 and 246), but they are not shown!! Please show! Also, you seem to be saying that all methods gave similar results (e.g. line 236). If so, what's the point of using your method over more standard/simple ones? I would like to see some example where you can show that your method outperforms vanilla PCA and alr/clr/ilr+PCA in terms of visualization. Your metrics in Figure 2 are not very convincing because it's hard to judge which of them should be most relevant (maybe you should explain the metrics better).  2. The toy example (Figure 1) is good, but I would like to see alr+PCA and ilr+PCA in addition to clr+PCA (I know that different people recommend different choices; does it matter here?), and more importantly, I'd like to see vanilla PCA. It seems to me that at least for 3-arms vanilla PCA should give an ideal "Mercedes" shape without any log-transforms. Is it true for 10-arms? If it _is_ true, then this is a poor example because vanilla PCA performs optimally. If so, the authors should modify the toy example such that alr/clr/ilr+PCA performed better than PCA (this would also help motivating clr-transform) and that CodaPCA performed even better than that.  3. I think a better motivation is needed for log-transforms in Section 2. Line 52 mentions "nonlinear structure due to the simplex constraint" but simplex constraint is clearly linear. So if the simplex constraint were the only issue, I wouldn't see *any* problem with using standard vanilla PCA. Instead, there arguably *is* some nonlinearity in the nature of compositional data (otherwise why would anybody use log-transforms), and I remember Aitchinson showing some curved scatterplots on the 2D simplex. This needs to be explained/motivated here.  4. Line 91 says that count data can be described with Poisson distribution. Line 260 again mentions "raw count data". But compositional data does not need to be count, it can come as continuous fractions (e.g. concentrations). Does the method suggested here only apply to counts? If so, this should be made very clear in the title/abstract/etc. If not, please clarify why Poisson distribution still makes sense.  MINOR ISSUES  1. Box after line 81. Shouldn't Eq 3 be part of the box? Without it the box looks incomplete. The same for the box after line 107.  2. Eq 6. "\phi-PCA" should be written with a hyphen (not minus) and capitalized PCA. Please use hyphens also below in similar expressions.  3. The end of Section 3 -- I suggest to insert some comments on how exactly exp family PCA reduces to standard PCA in the Gaussian case. What is \phi function, what is Bregman divergence, etc.   3. Line 113, last sentence of the section. This is unclear. Why is it a hole? Why is not easy?  4. Line 116, first sentence of the section: again, not clear how exactly clr is a workaround.  5. Line 139: period missing.  6. Line 144, where does this \phi() come from? Is it Poisson? Please explain.  7. Eq (11): what is "gauged-KL", the same as "gauged-phi" above?  8. Box after line 150: now you have loss function in the box, compare with minor issue #1.  9. Lines 201 and 214: two hidden layers or one hidden layer?  10. Line 206: space missing.  11. Line 211: isn't CoDA-PCA and SCoDA-PCA the same thing? Got confused here.  12. Line 257: I don't see any overfitting in Figure 2. This would be test-set curve going upwards, no? Please explain why do you conclude there is overfitting.  13. Shouldn't you cite Aitchinson's "PCA of compositional data" 1983?  -----------------  **Post-rebuttal update:** I still think the paper is worth accepting, but my main issue remains the lack of an obviously convincing application. I see that reviewer #3 voiced the same concern. Unfortunately, I cannot say that the scatter plots shown in the author response (panels A and B) settled this issue. The authors write that panel A, unlike B, shows three clusters but to me it looks like wishful thinking.  I am not familiar with the microbiome data but I do agree with the point of reviewer #3 that with 100+ features the compositional nature of the data can simply be ignored without losing much. Would the authors agree with that? I think it would be worth discussing.  Overall, it looks like a good attempt at setting up composional PCA in a mathematically principled way, but in the end I don't see examples where it performs noticeably better than standard naive approaches.


Semi-crowdsourced Clustering with Deep Generative Models
We consider the semi-supervised clustering problem where crowdsourcing provides noisy information about the pairwise comparisons on a small subset of data, i.e., whether a sample pair is in the same cluster. We propose a new approach that includes a deep generative model (DGM) to characterize low-level features of the data, and a statistical relational model for noisy pairwise annotations on its subset. The two parts share the latent variables. To make the model automatically trade-off between its complexity and fitting data, we also develop its fully Bayesian variant. The challenge of inference is addressed by fast (natural-gradient) stochastic variational inference algorithms, where we effectively combine variational message passing for the relational part and amortized learning of the DGM under a unified framework. Empirical results on synthetic and real-world datasets show that our model outperforms previous crowdsourced clustering methods.

Post Rebuttal Summary -- Thanks to the authors for a careful rebuttal that corrected several notation issues and provided an attempt at more realistic experiment using CIFAR-10 with real (not simulated) crowd labels. I found this rebuttal satisfactory and I am willing to vote for acceptance. I won't push *too* hard for acceptance because I still wish both the new experiment and the revised presentation of variational method details could go through more careful review.  My chief remaining concerns are that it is still difficult to distinguish the reasons for gains from BayesSCDC over plain old SCDC because multiple things change (factorization of q(z,x), estimation of global parameters). I do hope the authors keep their promise to compare to a "middle of the road" 3rd version and also offer insight about how amortization makes the q(z|o)q(x|z,o) approach still less flexible than mean field methods.  Review Summary -- Overall, I think the core ideas here are interesting, but I'm rating this paper as borderline accept because it doesn't do any experiments on real crowd-labeled datasets, misses opportunities to provide take-home insights about the various variational approximations attempted here, suffers from some minor bugs in the math (nothing showstopping), and leaves open several procedural questions about experiments. My final decision will depend on a satisfactory rebuttal.   Paper Summary -- This paper considers the problem where a given observed dataset consists of low-level feature vectors for many examples as well as pair-wise annotations for some example pairs indicating if they should belong to the same cluster or not. The goal is to determine a hard/soft clustering of the provided examples. The annotations may be *noisy*, e.g. from a not-totally-reliable set of crowd workers.  The paper assumes hierarchical model for both observed feature vectors O and observed pair-wise annotations L. Both rely on a shared K-component mixture model, which assigns each example into one of K clusters. Given this cluster assignment, the O and L are then generated by separate paths: observed vectors O come from a Gaussian-then-deep-Gaussian, and pairwise annotations L are produced by a two-coin Dawid-Skene model, originally proposed in [16]. Many entries of L may not be observed.  Two methods for training the proposed model are developed: * "SCDC" estimates an approximate posterior for each example's cluster assignment z and latent vector x  which conditions on observed features O -- q(z, x | O) -- but point estimates the "global" parameters (e.g. GMM parameters \pi, \mu, \Sigma) * "BayesSCDC" estimates an approximate posterior for all parameters that aren't neural net weights, using a natural-gradient formulation.  Notably, for "SCDC" the posterior over z,x has conditional structure:     q(z,x|o) = q(z|o) q(x|z,o). In contrast, the same joint posterior under the "BayesSCDC" model has mean-field structure with no conditioning on o: q(z,x) = q(z)q(x).  Experiments focus on demonstrating several benefits: * 5.1 toy data experiments show benefits of including pairwise annotations in the model. * 5.2 compares on a Faces dataset the presented approach to several baselines that can also incorporate pairwise annotations to discover clusters * 5.3 compares the two presented methods (full BayesSCDC vs SCDC) on MNIST data   Strengths -- * The pair-wise annotation model nicely captures individual worker reliability  into a probabilistic model.  * Experiments seem to explore a variety of reasonable baselines.   Weaknesses --  * Although the method is intended for noise crowd-labeling, none of the experiments actually includes truly crowd-labeled annotations. Instead, all labels are simulated as if from the true two-coin model, so it is difficult to understand how the model might perform on data actually generated by human labelers.  * The claimed difference between the fully-Bayesian inference of BayesSCDC and the point-estimation of global parameters in SCDC seems questionable to me... without showing the results of multiple independent runs and breaking down the differences more finely to separate q(z,x) issues from global parameter issues, its tough to be sure that this difference isn't due to poor random initialization, the q(z,x) difference, or other confounding issues.    Originality --  The key novelty claimed in this paper seems to be the shared mixture model architecture used to explain both observed features (via a deep Gaussian model) and observed noisy pairwise annotations (via a principled hierarchical model from [16]). While I have not seen this exact modeling combination before, the components themselves are relatively well understood.  The inference techniques used, while cutting edge, are used more or less in an "out-of-the-box" fashion by intelligently combining ideas from recent papers. For example, the recognition networks for non-conjugate potentials in BayesSCDC come from Johnson et al. NIPS 2016 [12], or the amortized inference approach to SCDC with marginalization over discrete local variables from Kingma, Mohamed, Rezende, and Welling [13].  Overall, I'm willing to rate this as just original enough for NIPS, because of the technical effort required to make all these work in harmony and the potential for applications. However, I felt like the paper had a chance to offer more compelling insight about why some approaches to variational methods work better than others, and that would have really made it feel more original.  Significance -- The usage of noisy annotations to guide unsupervised modeling is of significant interest to many in the NIPS community, so I expect this paper will be reasonably well-received, at least by folks interested in clustering with side information. I think the biggest barriers to widespread understanding and adoption of this work would be the lack of real crowd-sourced data (all experiments use simulated noisy pairwise annotations) and helping readers understand exactly why the BayesSCDC approach is better than SCDC alone when so much changes between the two methods.    Quality Issues --  ## Q1) Correctness issues in pair-wise likelihood in Eq. 2  In the pair-wise model definition in Sec. 2.2, a few things are unclear, potentially wrong: * The 1/2 exponent is just wrong as a poor post-hoc correction to the symmetry issue. It doesn't result in a valid distribution over L (e.g. that integrates to unity over the support of all binary matrices). A better correction in Eq. 2 would be to restrict the sum to those pairs (i,j) that satisify i < j (assuming no self edges allowed).  * Are self-edges allowed? That is, is L_11 or L_22 a valid entry? The sum over pairs i,j in Eq. 2 suggests so, but I think logically self-edges should maybe be forbidden.  ## Q2) Correctness issue in formula for mini-batch unbiased estimator of pair-wise likelihood  In lines 120-122, given a minibatch of S annotations, the L_rel term is computed by reweighting a minibatch-specific sum by a scale factor N_a / |S|, so that the term has similar magnitude as the full dataset. However, the N_a term as given is incorrect. It should count the total number of non-null observations in L. Instead, as written it counts the total number of positive entries in L.  ## Q3) Differences between SCDC and BayesSCDC are confusing, perhaps useful to breakdown more finely  The presented two approximation approaches, SCDC and BayesSCDC, seem to differ on several axes, so any performance difference is hard to attribute to one change. First, SCDC assumes a more flexible q(x, z) distribution, while BayesSCDC assumes a mean-field q(x)q(z) with a recognition network for a surrogate potential for p(o|x). Second, SCDC treats the global GMM parameters as point estimates, while BayesSCDC infers a q(\mu, \Sigma) and q(\pi). I think these two axes should be explored independently. In particular, I suggest presenting 3 versions of the method: * the current "SCDC" method * the current "BayesSCDC" method * a version which does q(x)q(z) with a recognition network for a surrogate potential for p(o|x) (Eq. 10), but point estimates global parameters.  The new 3rd version should enjoy the fast properties of BayesSCDC (each objective evaluation doesn't require marginalizing over all z values), but be more similar to SCDC.   Clarity ------- The paper reads reasonably well. The biggest issue in clarity is that some some key hyperparameters required to reproduce experiments are just not provided (e.g. the Dirichlet concentration for prior on \pi, the Wishart hyperparameters, etc.). These absolutely need to be in a final version.   Other reproducibility concerns: * what is the procedure for model initialization? * how many initializations of each method are allowed? how do you choose the best/worst? accuracy on training?


Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds
In recent years, unfolding iterative algorithms as neural networks has become an empirical success in solving sparse recovery problems. However, its theoretical understanding is still immature, which prevents us from fully utilizing the power of neural networks. In this work, we study unfolded ISTA (Iterative Shrinkage Thresholding Algorithm) for sparse signal recovery. We introduce a weight structure that is necessary for asymptotic convergence to the true sparse signal. With this structure, unfolded ISTA can attain a linear convergence, which is better than the sublinear convergence of ISTA/FISTA in general cases. Furthermore, we propose to incorporate thresholding in the network to perform support selection, which is easy to implement and able to boost the convergence rate both theoretically and empirically. Extensive simulations, including sparse vector recovery and a compressive sensing experiment on real image data, corroborate our theoretical results and demonstrate their practical usefulness. We have made our codes publicly available.
2 .

This paper proves that the unfolded LISTA, an empirical deep learning solver for sparse codes, can guarantee faster (linear) asymptotical convergence than standard iterative ISTA algorithm. The authors proposed a partial weight coupling structure and a support detection schemes. Rooted in standard LASSO techniques, they are both shown to speed up LISTA convergence. The theories are endorsed by extensive simulations and a real-data CS experiment.   Strength: I really like this paper. Simple, elegant, easy-to-implement techniques are backed up by solid theory. Experiments follow a step-by-step manner and accompanies theories fairly well.  - ISTA is generally sub-linearly convergent before its iterates settle on a support. Several prior works [8,15,21] show the acceleration effect of LISTA from different views, but this paper for the first time established the linear convergence of LISTA (upper bound). I view that as an important progress in the research direction of NN sparse solvers. - Both weight coupling and support detection are well motivated by theoretical speedup results. They are also very practical and can be “plug-and-play” with standard LISTA, with considerable improvements observed in experiments  Weakness: I have no particular argument for weakness. Two suggestions for authors to consider: - How the authors see whether their theory can be extended to convolutional sparse coding, which might be more suitable choices for image CS?  - Although IHT and LISTA solve different problems (thus not directly "comparable" in simulation), is it possible that the l0-based networks [4,21] can also achieve competitive performance in real data CS? The authors are suggested to compare in their updated version.


Large Scale computation of Means and Clusters for Persistence Diagrams using Optimal Transport
Persistence diagrams (PDs) are now routinely used to summarize the underlying topology of complex data. Despite several appealing properties, incorporating PDs in learning pipelines can be challenging because their natural geometry is not Hilbertian. Indeed, this was recently exemplified in a string of papers which show that the simple task of averaging a few PDs can be computationally prohibitive. We propose in this article a tractable framework to carry out standard tasks on PDs at scale, notably evaluating distances, estimating barycenters and performing clustering. This framework builds upon a reformulation of PD metrics as optimal transport (OT) problems. Doing so, we can exploit recent computational advances: the OT problem on a planar grid, when regularized with entropy, is convex can be solved in linear time using the Sinkhorn algorithm and convolutions. This results in scalable computations that can stream on GPUs. We demonstrate the efficiency of our approach by carrying out clustering with diagrams metrics on several thousands of PDs, a scale never seen before in the literature.

# Update after rebuttal  I thank the authors for their comments in the rebuttal. In conjunction with discussions with other reviewers, they helped me understand the salient points of the paper better. I would still add the following suggestions:  1. It would strengthen the paper if advantages over existing methods that are not Wasserstein-based could be showed (persistence landscapes, persistence images, ...), in particular since those methods have the disadvantage of suffering from descriptors that are not easily interpretable and do not yield 'proper' means (in the sense that the mean landscape, for example, cannot be easily converted back into a persistence diagram.  2. The experiment could be strengthened by discussing a more complex scenario that demonstrates the benefits of TDA _and_ the benefits of the proposed method. This could be a more complex data set, for example, that is not easily amenable to non-topological approaches, such as natural image data. I realize that this a balancing act, however.  Having discussed at length how to weigh the originality of the paper, I would like to see the method published in the hopes that it will further promulgate TDA. I am thus raising my score.  # Summary  This paper presents an algorithm for calculating approximate distances between persistence diagrams, i.e. feature descriptors from topological data analysis. The approximation is based on a reformulation of the distance calculation as an optimal transport problem. The paper describes how to apply previous research, most notably entropic regularization, to make calculations efficient and parallelizable. In addition to showing improved performance for distance calculations, the paper also shows how to calculate barycentres in the space of persistence diagrams. Previously, this involved algorithm that are computationally infeasible at larger scales, but the new formulations permits faster calculations. This is demonstrated by deriving a $k$-means algorithm.  # Review  This is an interesting paper and I enjoyed reading it. I particularly appreciated the step-by-step 'instructions' for how to build the algorithm, which increases accessibility even though the topic is very complex with respect to terminology. However, I lean towards rejecting the paper for now: while the contribution is solid and reproducible, the paper appears to be a (clever!) application of previous works, in particular the ones by Altschuler et al., Benamou et al., Cuturi, as well as Cuturi and Doucet. In contrast to these publications, which have an extremely general scope (viz. the whole domain of optimal transport), this publication is only relevant within the realm of topological data analysis.  Moreover, the experimental section is currently rather weak and does not give credence to the claim in the abstract that 'clustering is now demonstrated at scales never seen before in the literature'. While I agree that this method is capable of handling larger persistence diagrams, I think the comparison with other algorithms needs to have more details: the 'Hera' package, as far as I know, also permits the calculation of approximations to the Wasserstein distance and to the bottleneck distance between persistence diagrams. Were these used in the experiments? And if so, what were the parameters such as the 'relative error'? And for that matter, how many iterations of the new algorithm were required and what are the error bounds?  In addition, it might be fairer to use a single-core implementation for the comparison here---this would also show the benefits of parallelization! Such plots could be shown in the supplementary materials of the paper. At present, the observations with respect to speed are somewhat 'obvious': it seems clear that a multi-core algorithm (on a P100!) can always beat a single-core implementation. The same goes for the barycentre calculations, which are not even written in the same programming language I assume. Instead of showing that the algorithm is faster here (which I will definitely believe) can the authors say more about the qualitative behaviour? Are the barycentres essentially the same? How long does convergence take?  Statements of this sort would also strengthen reproducibility; the authors could show all required parameters in addition to their experiments.  Minor comments:  - L. 32: If different kernels for persistence diagrams are cited, the first one, i.e. the multi-scale kernel by Reininghaus et al. in "A Stable Multi-Scale Kernel for Topological Machine Learning" should also be cited. - In the introduction, works about approximations of persistence diagram distances (such as the one by Kerber et al.) could be briefly discussed as well. Currently, they only appear in the experimental section, which is a little bit late, I think. - L. 57: "verifying" --> "satisfying" (this happens at other places as well) - L. 114: $\mathcal{M}_{+}$ should be briefly defined/introduced before usage - How are parameters tuned? While I understand how the 'on-the-fly' error control is supposed to work, I am not sure how reasonable defaults can be obtained here. Or is the algorithm always iterated until the approximation error is sufficiently small? - L. 141: Please add a citation for the C-transform, such as Santambrogio (who is already cited in the bibliography) - L. 155: "criterion" --> "threshold" - L. 168: What does $|D_1|$ refer to? Some norm of the diagram points? This is important in order to understand the discretization error. - L. 178: From what I can tell, the algorithm presented in the paper generalizes to arbitrary powers; does the statement about the convolutions only hold for the squared Euclidean distance? The authors should clarify this. - L. 231: "Turner et al." is missing a citation - Timing information for the first experiment in Section 5 should be included; it is important to know the dependencies between the 'fidelity' of the approximation and the computational speed; in particular since the paper speaks about 'smearing' in L. 251, which prohibits the recovery of 'spiked' diagrams


A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent
We study the generalization error of randomized learning algorithms-focusing on stochastic gradient descent (SGD)-using a novel combination of PAC-Bayes and algorithmic stability. Importantly, our generalization bounds hold for all posterior distributions on an algorithm's random hyperparameters, including distributions that depend on the training data. This inspires an adaptive sampling algorithm for SGD that optimizes the posterior at runtime. We analyze this algorithm in the context of our generalization bounds and evaluate it on a benchmark dataset. Our experiments demonstrate that adaptive sampling can reduce empirical risk faster than uniform sampling while also improving out-of-sample accuracy.

The paper opens the way to a new use of PAC-Bayesian theory, by combining PAC-Bayes with algorithmic stability to study stochastic optimization algorithms. The obtained probabilistic bounds are then used to inspire adaptive sampling strategies, studied empirically in a deep learning scenario.
The  paper is well written, and the proofs are non-trivial. It contains several clever ideas, namely the use of algorithmic stability to bound the complexity term inside PAC-Bayesian bounds. It's also fruitful to express the prior and posterior distributions over the sequences of indexes used by a stochastic gradient descent algorithm. Up to my knowledge, this is very different than any other previous PAC-Bayes theorems, and I think it can inspire others in the future.
The experiments using a (moderately) deep network, trained by both SGD and AdaGrad, are convincing enough; they show that the proposed adaptive sampling strategy can benefit to existing optimization methods simply by selecting the samples during training time.
Small comments:
- Line 178: In order to sell their result, the authors claim that "high-probability bounds are usually favored" over bounds in expectation. I don't think the community is unanimous about this, and I would like the authors to convince me that I should prefer high-probability bounds.
- Section 5 and Algorithm 1: I suggest to explicitly express the utility function f as a function of a model h.
Typos:
- Line 110: van -> can
- Supplemental, Line 183: he -> the
** UPDATE **
I encourage the authors to carefully consider the reviewer's comments while preparing the final version of their paper. Concerning the claim that high-probability bound imply expectation bound, I think it's right, but it deserves to be explained properly to convince the readers.


The Marginal Value of Adaptive Gradient Methods in Machine Learning
Adaptive optimization methods, which perform local optimization with a metric constructed from the history of iterates, are becoming increasingly popular for training deep neural networks. Examples include AdaGrad, RMSProp, and Adam. We show that for simple overparameterized problems, adaptive methods often find drastically different solutions than gradient descent (GD) or stochastic gradient descent (SGD). We construct an illustrative binary classification problem where the data is linearly separable, GD and SGD achieve zero test error, and AdaGrad, Adam, and RMSProp attain test errors arbitrarily close to half. We additionally study the empirical generalization capability of adaptive methods on several stateof-the-art deep learning models. We observe that the solutions found by adaptive methods generalize worse (often significantly worse) than SGD, even when these solutions have better training performance. These results suggest that practitioners should reconsider the use of adaptive methods to train neural networks.

// Summary:
In an optimization and a learning context, the authors compare recently introduced adaptive gradient methods and more traditional gradient descent methods (with potential momentum). Adaptive methods are based on metrics which evolve along the optimization process. Contrary to what happens for gradient descent, Nesterov's method or the heavy ball method, this may result in estimates which are outside of the linear span of past visited points and estimated gradients. These methods became very popular recently in a deep learning context. The main question adressed by the authors is to compare both categories of method. First the authors construct an easy classification example for which they prove that adaptive methods behave very badly while non adaptive methods achieve perfect accuracy. Second the authors report extensive numerical comparisons of the different classes of algorithms showing consistent superiority of non adaptive methods.
// Main comments:
I do not consider myself knowledgable enough to question the relevance and correctness of the numerical section.
This paper asks a very relevant question about adaptive methods. Giving a definitive answer is a complicated task and the authors provide elements illustrating that adaptive methods do not necessarily have a systematic advantage over non adaptive ones and that non adaptive optimization methods could be considered as serious challengers. The content of the paper is simple and really acessible, hilighting the simplicity of the question. Although simple, this remains extremely relevant and goes beyond the optimization litterature, questioning the reasons why certain methods reach very high popularity in certain communities. The elements given illustrate the message of the authors very clearly.
The main points I would like to raise:
- The artificial example given by the authors is only relevant for illustrative purposes. Comparison of algorithmic performances should be based on the ability to solve classes of problems rather than individual instances. Therefore, it is not really possible to draw any deep conclusion from this. For example, a bad choice of the starting point for non adaptive methods would make them equally bad as adaptive methods on this specific problem. I am not sure that the authors would consider this being a good argument against methods such as gradient descent.
- Popularity of adaptive methods should be related to some practical advantage from a numerical point of view. I guess that positive results and comparison with non adaptive methods have already been reported in the litterature. Do the authors have a comment on this?


Training recurrent networks to generate hypotheses about how the brain solves hard navigation problems
Self-localization during navigation with noisy sensors in an ambiguous world is computationally challenging, yet animals and humans excel at it. In robotics, Simultaneous Location and Mapping (SLAM) algorithms solve this problem through joint sequential probabilistic inference of their own coordinates and those of external spatial landmarks. We generate the first neural solution to the SLAM problem by training recurrent LSTM networks to perform a set of hard 2D navigation tasks that require generalization to completely novel trajectories and environments. Our goal is to make sense of how the diverse phenomenology in the brain's spatial navigation circuits is related to their function. We show that the hidden unit representations exhibit several key properties of hippocampal place cells, including stable tuning curves that remap between environments. Our result is also a proof of concept for end-to-end-learning of a SLAM algorithm using recurrent networks, and a demonstration of why this approach may have some advantages for robotic SLAM.

In this study, the authors trained recurrent neural network models with LSTM units to solve a well studied problem, namely Simultaneous Location and Mapping (SLAM). The authors trained the network to solve several different version of the problems, and made some qualitative comparison to rodent physiology regard spatial navigation behavior.
In general, I think the authors' approach follows an interesting idea, namely using recurrent network to study the cognitive functions of the brain. This general idea has recently started to attract substantial attentions in computational neuroscience.
The authors tested several version of the localization tasks, which is extensive. And the attempt to compare the model performance to the neurophysiology should be appreciated. They also attempted to study the inner working of the training network by looking into the dimensionality of the network, which reveals a relatively low dimensional representation, but higher than the dimension of the physical space.
Concerns-
The algorithms used is standard in machine learning, thus my understanding is that the main contribution of the paper presumably should come from either solving the SLAM problem better or shed some light from the neuroscience perspective. However, I am not sure about either of them.
1) For solving the SLAM problem,  I don't see a comparison to the state-of-the-art algorithms in SLAM, so it is unclear where the performance of this algorithm stands.
After reading through other reviewers' comments and the rebuttal, I share similar concerns with Reviewer 1- it is unclear whether the networks learn to solve the SLAM generically, and whether the network can perform well in completely different environments. The set-up the authors assumed seem to be restrictive, and it is unclear whether it can apply to realist SLAM problem. I agree with Reviewer 1 that this represents one important limitation of this study I initially overlooked, and it was not satisfyingly addressed in the rebuttal.
2) From the neuroscience perspective, some of the comparison the authors made in Section 3.2 are potentially interesting. However, my main concern is that the "neuron-like representations" shown in Fig.4A,B are not really neuron-like according to the rodent physiology literature. In particular, the place fields shown in Fig4A and 4B do not resemble the place fields reported in rodent physiology, which typically have roughly circular symmetric shape. I'd certainly appreciate these results better if these spatial tuning resembles more to the known rodent physiology.  Can the authors show the place fields of all the neurons in the trained network, something like Fig 1 in Wilson & McNaughton (1993, Science)? That'd be helpful in judging the similarity of the representation in the trained network and the neural representation of location in rodents.
Also, in the rodent literature, the global remapping and rate remapping are often distinguished and the cause for these two types of remapping are not well-understood at this point. Here the authors only focus on global remapping. One potentially interesting question is whether the rate remapping also exist in the trained network. The paper would be stronger if the authors could shed some light on the mechanisms for these two types of remapping from their model. Just finding global remapping across different environment isn't that surprising in my view.
Related to Fig4E, it is known that neurons in hippocampus CA1 exhibit a heavy tail in their firing rate (e.g, discussed in ref[20]). Do the authors see similar distributions of the activity levels in the trained network?
3) The manuscript is likely to be stronger if the authors could emphasize either the performance of the network in performing SLAM or  explaining the neurophysiology. Right now, it feels that the paper puts about equal weights on both, but neither part is strong enough.
According to the introduction and abstract, it seems that the authors want to emphasize the similarity to the rodent physiology. In that regard, Fig. 2 and Fig. 3 are thus not particularly informative, unless the authors show the performance from the recurrent network is similar to the rodent (or maybe even humans, if possible) behavior in some interesting ways. To emphasize the relevance to neuroscience, it is useful to have a more extended and more thorough comparison to the hippocampus neurophysiology, though I am not sure if that's possible given the place fields of the trained network units do not resemble the neurophysiology that well, as I discussed earlier.
Alternatively, to emphasize the computational power of the RNN model in solving SLAM, it would be desirable to compare the RNN to some state-of-the-art algorithms. I don't think the particle filtering approach the authors implemented represents the state-of-the-art in solving SLAM. But I could be wrong on this, as I am not very familiar with that literature.
Finally, a somewhat minor point-
the authors make the comparison on the capacity of the network to the multi-chart attractor model in the Discussion, but I am not sure how that represents a fair comparison. I'd think one LSTM unit is computational more powerful than a model neuron in a multi-chart attractor model. I'd like to understand why the authors think a comparison can be drawn here.


MaskRNN: Instance Level Video Object Segmentation
Instance level video object segmentation is an important technique for video editing and compression. To capture the temporal coherence, in this paper, we develop MaskRNN, a recurrent neural net approach which fuses in each frame the output of two deep nets for each object instance -a binary segmentation net providing a mask and a localization net providing a bounding box. Due to the recurrent component and the localization component, our method is able to take advantage of long-term temporal structures of the video data as well as rejecting outliers. We validate the proposed algorithm on three challenging benchmark datasets, the DAVIS-2016 dataset, the DAVIS-2017 dataset, and the Segtrack v2 dataset, achieving state-of-the-art performance on all of them.

Last year has seen a large progress in video object segmentation, triggered by the release of the DAVIS dataset. In particular, [Khoreva et al., CVPR'17] and [Caelles et al., CVPR'17] have shown that training a fully-convolutional network on the first frame of the video can greatly improve performance in the semi-supervised scenario, where the goal is to, given ground truth annotation in the first frame of a video, segment the annotated object in the remaining frames. In addition, Khoreva et al., demonstrated that feeding the prediction for the previous frame as an additional input to the network and utilising a parallel branch that operates on the optical flow magnitude further improves the performance. These approaches have indeed set the new state-of-the-art on DAVIS, but remained constrained to operating on a single frame and a single object at a time.
In this work the authors propose to address these limitations as well as integrate some promising techniques from the recent object detection methods into a video object segmentation framework. Starting from the approach of Khoreva et al., which consists of two fully-convolutional networks pertained for mask refinement, taking rgb and flow magnitude as input respectively, as well as an object mask predicted in the previous frame, and fine-tuned on the first frame of the test video, they propose the following extensions:
1. The object mask from the previous frame is warped with optical flow to simplify mask refinement.
2. The model's prediction is "cleaned up" by taking a box proposal, refining it with a specialised branch of the network and suppressing all the segmentation predictions outside of the box.
3. A (presumably convolutional) RNN is attached in the end of the pipeline, which enforces temporal consistency on the predictions.
4. The model is extended to the instance-level scenarios, that is, it can segment multiple objects at once, assigning a distinct label to each of them.
A complex system like this is, of course, challenging to present in an 8 page paper. The authors address this issue by simply skipping the description of some of the components. In particular, the RNN is not described at all and the details of the bounding box generation are omitted. Moreover, the "instance-level" results are obtained by simply running the model separately for each instance and merging the predictions on the frame level, which can just as well be done for any other method.
In general, the paper is well written and the evaluation is complete, studying the effect of each extension in separation. The study shows, that in the scenario when the networks are fine-tuned on the first frame of the video, the effect of all the proposed extensions is negligible. This is in part to be expected, since, as was shown in the recent work, training a network on the first frame of a video serves as a very strong baseline in itself. When no fine-tuning on the test sequence is done though, the proposed extensions do improve the method's performance, but the final result in this setting on DAVIS'16 remains more than 13% bellow the rgb-only, no-test-finetuning model of Khoreva et al. Though their model is trained on a larger COCO dataset, this is hardly enough to explain such a large performance gap with a much simpler model.
In the state-of-the-art comparison the full method outperforms the competitors on most of the measures on 3 datasets (DAVIS'16, DAVIS'17 and Seg-Track-v2). That said, the comparison with [Khoreva et al., CVPR'17] on DAVIS'17 is missing and the performance gap is not always significant relatively to the complexity of the proposed solution (0.9% on DAVIS'16). Especially, given the recent extension of Khoreva et al., (see reference [25] in the paper), which shows significantly better results with much simpler tools.
Overall, the authors have done a good job by combining insights from several recent publications into a single framework but failed to achieve convincing results.


Conservative Contextual Linear Bandits
Safety is a desirable property that can immensely increase the applicability of learning algorithms in real-world decision-making problems. It is much easier for a company to deploy an algorithm that is safe, i.e., guaranteed to perform at least as well as a baseline. In this paper, we study the issue of safety in contextual linear bandits that have application in many different fields including personalized recommendation. We formulate a notion of safety for this class of algorithms. We develop a safe contextual linear bandit algorithm, called conservative linear UCB (CLUCB), that simultaneously minimizes its regret and satisfies the safety constraint, i.e., maintains its performance above a fixed percentage of the performance of a baseline strategy, uniformly over time. We prove an upper-bound on the regret of CLUCB and show that it can be decomposed into two terms: 1) an upper-bound for the regret of the standard linear UCB algorithm that grows with the time horizon and 2) a constant term that accounts for the loss of being conservative in order to satisfy the safety constraint. We empirically show that our algorithm is safe and validate our theoretical analysis.

POST-REBUTTAL:
The authors have answered my concerns and will clarify the point of confusion. I'm changing from a marginal accept to an accept.
OLD REVIEW:
Summary of the paper
This paper proposes a "safe" algorithm for contextual linear bandits. This definition of safety assumes the existence of a current "baseline policy" for selecting actions. The algorithm is "safe" in that it guarantees that it will only select an action that differs from the action proposed by the baseline policy if the new action produces larger expected reward than the action proposed by the baseline policy. Due to the random nature of rewards, this guarantee is with high probability (probability at least 1-delta).

Summary of review
The paper is well written. It is an extremely easy read - I thank the authors for submitting a polished paper. The proposed problem setting and approach are novel to the best of my knowledge. The problem is well motivated and interesting. Sufficient theoretical and empirical justifications are provided to convince the reader of the viability of the proposed approach.
However, I have some questions. I recommend at least weak acceptance, but would consider a stronger acceptance if I am misunderstanding some of these points.

Questions
1. Definition 1 defines a desirable performance constraint. The high probability nature of this constraint should be clarified. Notice that line 135 doesn't mention that (3) must hold with high probability. This should be stated.
2. More importantly, the statement of *how* (3) must hold is critical, since right now it is ambiguous. During my first read it sounds like (3) must hold with probability 1-\delta. However this is *not* achieved by the algorithm. If I am understanding correctly (please correct me if I am wrong), the algorithm ensures that *at each time step* (3) holds with high probability. That is:
\forall t \in \{1,\dotsc,T\}, \Pr \left ( \sum_{i=1}^t r_{b_i}^i - \sum_{i=1}^t r_{a_i}^t \leq \alpha \sum_{i=1}^t r_{b_i}^i \right )
NOT
\Pr \left ( \forall t \in \{1,\dotsc,T\},  \sum_{i=1}^t r_{b_i}^i - \sum_{i=1}^t r_{a_i}^t \leq \alpha \sum_{i=1}^t r_{b_i}^i \right )
The current writing suggests the latter, which (I think) is not satisfied by the algorithm. In your response could you please clarify which of these you are claiming that your algorithm satisfies?
3. If your algorithm does in fact satisfy the former (the per-time step guarantee), then the motivation for the paper is undermined (this could be addressed by being upfront about the limitations of this approach in the introduction, without changing any content).
Consider the actual guarantee that you provide in the domain used in the empirical study. You run the algorithm for 40,000 time steps with delta = 0.001. Your algorithm is meant to guarantee that "with high probability" it performs at least as well as the baseline. However, you only guarantee that the probability of playing a worse action will be at most 0.001 *at each time step*. So, you are guaranteeing that the probability of playing a worse action at some point will be at most 1-.999^40000 = (approximately) 1. That is, you are bounding the probability of an undesirable event to be at most 1, which is not very meaningful. This should be discussed.
For now, I would appreciate your thoughts on why having a per-step high probability guarantee is important for systems where there are large numbers of time steps. If a single failure is damning then we should require a high probability guarantee that holds simultaneously for all time steps. If a single failure is not damning, but rather amortized cost over over thousands of time steps is what matters, then why are we trying to get per-time step high probability guarantees?


Learning Disentangled Representations with Semi-Supervised Deep Generative Models
Variational autoencoders (VAEs) learn representations of data by jointly training a probabilistic encoder and decoder network. Typically these models encode all features of the data into a single variable. Here we are interested in learning disentangled representations that encode distinct aspects of the data into separate variables. We propose to learn such representations using model architectures that generalise from standard VAEs, employing a general graphical model structure in the encoder and decoder. This allows us to train partially-specified models that make relatively strong assumptions about a subset of interpretable variables and rely on the flexibility of neural networks to learn representations for the remaining variables. We further define a general objective for semi-supervised learning in this model class, which can be approximated using an importance sampling procedure. We evaluate our framework's ability to learn disentangled representations, both by qualitative exploration of its generative capacity, and quantitative evaluation of its discriminative ability on a variety of models and datasets.

This paper investigates the use of models mixing ideas from 'classical' graphical models (directed graphical models with nodes with known meaning, and with known dependency structures) and deep generative models (learned inference network, learned factors implemented as neural networks). A particular focus is the use of semi-supervised data and structure to induce 'disentangling' of features.  They test on a variety of datasets and show that disentangled features are interpretable and can be used for downstream tasks.
In contrast with recent work on disentangling ("On the Emergence of Invariance and Disentangling in Deep Representations", "Early visual concept learning with unsupervised deep learning", and others), here the disentangling is not really an emergent property of the model's inductive bias, and more related to the (semi)-supervision provided on semantic variables of the model.
My opinion on the paper is overall positive - the derivations are correct, the model and ideas contribute to the intersection of graphical models / probabilistic modeling and deep learning, and well-made/detailed experiments support the authors' claims.
Negative:
- I don't think the intersection of ideas in structured graphical models and deep generative model is particularly novel at this point (see many of the paper's own references). In particular I am not convinced by the claims of increased generality.
Clearly there is a variety of deep generative models that fall in the PGM-DL spectrum described by the authors and performing inference with flexible set of observed variables mostly involves using different inference networks (with potential parameter reuse for efficiency). Regarding continuous domain latents: I don't see what would prevent the semi-supervised model from Kingma et al from handling continous latent variable: wouldn't simply involve changing q(y|x)  to a continuous distribution and reparametrize (just as they do for the never-supervised z)? Generally, methodologically the papers appears somewhat incremental (including the automated construction of the stochastic computation graph, which is similar to what something like Edward (Tran et al.) does).
- Some numerical experiments results are slightly disappointing (the semi-supervised MNIST numbers seem worse than Kingma et al.; the multi-MNIST reconstructions are somewhat blurry; what do samples look like?)
Positive:
- The paper is well written and tells a compelling story.
- The auxiliary bound introduced in (4)-(5) is novel and and allows the posterior in the semi-supervised case to force learning over the classification network (in corresponding equation (6) of Kingma et al. the classification network does not appear). Do the authors know how the variance between (4) and the analogous expression from Kingma et al compare?
- The rest of the  experiments (figure 2, 5) paint a clear picture of why having semantics inform the structure of the generative model can lead to interesting way to probe the behavior of our generative models.


Fast-Slow Recurrent Neural Networks
Processing sequential data of variable length is a major challenge in a wide range of applications, such as speech recognition, language modeling, generative image modeling and machine translation. Here, we address this challenge by proposing a novel recurrent neural network (RNN) architecture, the Fast-Slow RNN (FS-RNN). The FS-RNN incorporates the strengths of both multiscale RNNs and deep transition RNNs as it processes sequential data on different timescales and learns complex transition functions from one time step to the next. We evaluate the FS-RNN on two character level language modeling data sets, Penn Treebank and Hutter Prize Wikipedia, where we improve state of the art results to 1.19 and 1.25 bits-per-character (BPC), respectively. In addition, an ensemble of two FS-RNNs achieves 1.20 BPC on Hutter Prize Wikipedia outperforming the best known compression algorithm with respect to the BPC measure. We also present an empirical investigation of the learning and network dynamics of the FS-RNN, which explains the improved performance compared to other RNN architectures. Our approach is general as any kind of RNN cell is a possible building block for the FS-RNN architecture, and thus can be flexibly applied to different tasks.

The paper proposed a new RNN structure called Fast-slow RNN and showed improved performance on a few language modeling data set.
Strength:
1. The algorithm combines the advantages of a deeper transition matrix (fast RNN) and a shorter gradient path (slow RNN).
2. The algorithm is straightforward and can be applied to any RNN cells.
Weakness:
1. I find the first two sections of the paper hard to read. The author stacked a number of previous approaches but failed to explain each method clearly.
Here are some examples:
(1) In line 43, I do not understand why the stacked LSTM in Fig 2(a) is "trivial" to convert to the sequential LSTM Fig2(b). Where are the h_{t-1}^{1..5} in Fig2(b)? What is h_{t-1} in Figure2(b)?
(2) In line 96, I do not understand the sentence "our lower hierarchical layers zoom in time" and the sentence following that.
2. It seems to me that the multi-scale statement is a bit misleading, because the slow and fast RNN do not operate on different physical time scale, but rather on the logical time scale when the stacks are sequentialized in the graph. Therefore, the only benefit here seems to be the reduce of gradient path by the slow RNN.
3. To reduce the gradient path on stacked RNN, a simpler approach is to use the Residual Units or simply fully connect the stacked cells. However, there is no comparison or mention in the paper.
4. The experimental results do not contain standard deviations and therefore it is hard to judge the significance of the results.


Generating steganographic images via adversarial training
Adversarial training has proved to be competitive against supervised learning methods on computer vision tasks. However, studies have mainly been confined to generative tasks such as image synthesis. In this paper, we apply adversarial training techniques to the discriminative task of learning a steganographic algorithm. Steganography is a collection of techniques for concealing the existence of information by embedding it within a non-secret medium, such as cover texts or images. We show that adversarial training can produce robust steganographic techniques: our unsupervised training scheme produces a steganographic algorithm that competes with state-of-the-art steganographic techniques. We also show that supervised training of our adversarial model produces a robust steganalyzer, which performs the discriminative task of deciding if an image contains secret information. We define a game between three parties, Alice, Bob and Eve, in order to simultaneously train both a steganographic algorithm and a steganalyzer. Alice and Bob attempt to communicate a secret message contained within an image, while Eve eavesdrops on their conversation and attempts to determine if secret information is embedded within the image. We represent Alice, Bob and Eve by neural networks, and validate our scheme on two independent image datasets, showing our novel method of studying steganographic problems is surprisingly competitive against established steganographic techniques.

The authors studied how to use adversarial training to learn the encoder, the steganalyzer and the decoder at the same time using unsupervised learning. Specifically, the authors designed an adversarial game of three parties. The encoder generates images based on the cover and the message. Then the generated image can be received to both decoder and steganalyzer. The goal of the encoder and decoder is to correctly encode and decode the message, and the goal of the steganalyzer is to determine whether the image is encrypted. Thus it is a minimax game.

Despite the algorithmic part of the paper first, my major concern is the correctness of the methodology. As we all know, the neural networks have great ability to fit or classify the training data. You use two almost same neural network architectures as the decoder and steganalyzer. Then the question raises, as the decoder and steganalyzer use the same information in the training process. If the steganalyzer can correctly predict the label of the image, then the task fails. But if the steganalyzer cannot correctly predict the label, I don't think the decoder can recover the message well. I am confused about this and hope the author can answer the question in the rebuttal phase.

From the experiment we can see the loss of steganalyzer is super large according to figure 3 and 4, which means that the steganalyzer fails to classify almost all examples. However, the authors also shown that steganalyzer is "pretty well" if the steganographic algorithm is "fixed". I am also confused about such self-contradictory explanations. Do you mean that during training, your steganalyzer is not well learnt? And your encoder and decoder is learnt using a bad steganalyzer?

Here are also some minor comments:
1.	Line 151: the loss function L_E has no dependency on \theta_A given C'
2.	Line 152: parameter C should be included into loss function L_E
3.       A large number of details are missed in the experiments. What are the values of hyperparameters? In the paper, the author claimed it is converged after 150 training steps. As the batch size is 32, you cannot even make a full pass to the training examples during training. How can you explain this?


Sample and Computationally Efficient Learning Algorithms under S-Concave Distributions
We provide new results for noise-tolerant and sample-efficient learning algorithms under s-concave distributions. The new class of s-concave distributions is a broad and natural generalization of log-concavity, and includes many important additional distributions, e.g., the Pareto distribution and t-distribution. This class has been studied in the context of efficient sampling, integration, and optimization, but much remains unknown about the geometry of this class of distributions and their applications in the context of learning. The challenge is that unlike the commonly used distributions in learning (uniform or more generally log-concave distributions), this broader class is not closed under the marginalization operator and many such distributions are fat-tailed. In this work, we introduce new convex geometry tools to study the properties of s-concave distributions and use these properties to provide bounds on quantities of interest to learning including the probability of disagreement between two halfspaces, disagreement outside a band, and the disagreement coefficient. We use these results to significantly generalize prior results for margin-based active learning, disagreement-based active learning, and passive learning of intersections of halfspaces. Our analysis of geometric properties of s-concave distributions might be of independent interest to optimization more broadly.

This review is adapted from my review from COLT 2017 - My feedback to this paper has not changed much since then.
This paper studies a new family of distributions, s-concave distributions, which appears in works of (Brascamp and Lieb, 1976; Bobkov, 2007, Chandrasekaran, Deshpande and Vempala, 2009). The main result is a series of upper and lower bounds regarding its probability distribution function and its measure over certain regions. These inequalities can be readily applied to (active) learning linear separators and learning the intersection of two halfspaces.
Overall this is an interesting paper, extending the family of distributions in which the problem of learning linear separators can be efficiently solved. This may spur future research on establishing new distribution-dependent conditions for (efficient) learnability.
Technical quality: on one hand, the results are impressive, since the family of admissible distributions (in the sense of Awasthi, Balcan and Long, 2014 and Klivans, Long and Tang, 2009) is broadened; on the other hand, the analysis only works when -1/(2n+3)<=s<=0. In high dimensional settings, the range of s is fairly small.
Novelty: the result of this paper follows the reasoning in (Lovasz and Vempala, 2007); but this is definitely a non-trivial extension.
Potential Impact: this work may spur future research on distributional conditions for efficient learnability.


Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
Most distributed machine learning systems nowadays, including TensorFlow and CNTK, are built in a centralized fashion. One bottleneck of centralized algorithms lies on high communication cost on the central node. Motivated by this, we ask, can decentralized algorithms be faster than its centralized counterpart? Although decentralized PSGD (D-PSGD) algorithms have been studied by the control community, existing analysis and theory do not show any advantage over centralized PSGD (C-PSGD) algorithms, simply assuming the application scenario where only the decentralized network is available. In this paper, we study a D-PSGD algorithm and provide the first theoretical analysis that indicates a regime in which decentralized algorithms might outperform centralized algorithms for distributed stochastic gradient descent. This is because D-PSGD has comparable total computational complexities to C-PSGD but requires much less communication cost on the busiest node. We further conduct an empirical study to validate our theoretical analysis across multiple frameworks (CNTK and Torch), different network configurations, and computation platforms up to 112 GPUs. On network configurations with low bandwidth or high latency, D-PSGD can be up to one order of magnitude faster than its well-optimized centralized counterparts.

In this paper, the authors present an algorithm for decentralized parallel stochastic gradient descent (PSGD). In contrast to centralized PSGD where worker nodes compute local gradients and the weights of a model are updated on a central node, decentralized PSGD seeks to perform training without a central node, in regimes where each node in a network can communicate with only a handful of adjacent nodes. While this network architecture has typically been viewed as a limitation, the authors present a theoretical analysis of their algorithm that suggests D-PSGD can achieve a linear speedup comparable to C-PSGD, but with significantly lower communication overhead. As a result, in certain low bandwidth or high latency network scenarios, D-PSGD can outperform C-PSGD. The authors validate this claim empirically.
Overall, I believe the technical contributions of this paper could be very valuable. The authors claim to be the first paper providing a theoretical analysis demonstrating that D-PSGD can perform competitively or even outperform C-PSGD. I am not sufficiently familiar with the relevant literature to affirm this claim, but if true then I believe that the analysis provided by the authors is both novel and intriguing, and could have nontrivial practical impact on those training neural networks in a distributed fashion, particularly over high latency networks.
The authors additionally provide a convincing experimental evaluation of D-PSGD, demonstrating the competitiveness of D-PSGD using modern CNN architectures across several network architectures that vary in scale, bandwidth and latency.
I found the paper fairly easy to read and digest, even as someone not intimately familiar with the parallel SGD literature and theory. The paper does contain a number of small typographical errors that should be corrected with another editing pass; a small list of examples that caught my eye is compiled below.
-- Minor comments --
Line 20: "pay" -> "paying"
Line 23: "its" -> "their", "counterpart" -> "counterparts"
Line 33: "parameter server" -> (e.g) "the parameter server topology"
Algorithm 1: the for loop runs from 0 to K-1, but the output averages over x_{K}.
Line 191: "on" -> "in"
Line 220: "of NLP" -> "of the NLP experiment"


Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems
We propose a generic algorithmic building block to accelerate training of machine learning models on heterogeneous compute systems. Our scheme allows to efficiently employ compute accelerators such as GPUs and FPGAs for the training of large-scale machine learning models, when the training data exceeds their memory capacity. Also, it provides adaptivity to any system's memory hierarchy in terms of size and processing speed. Our technique is built upon novel theoretical insights regarding primal-dual coordinate methods, and uses duality gap information to dynamically decide which part of the data should be made available for fast processing. To illustrate the power of our approach we demonstrate its performance for training of generalized linear models on a large-scale dataset exceeding the memory size of a modern GPU, showing an order-of-magnitude speedup over existing approaches.

The paper presents a specific way of exploiting a system with two heterogeneous compute nodes with complementary capabilities. The idea is to perform block co-ordinate algorithm with the subproblem of optimizing with the block variables solved at the node with high-compute-low-memory and the subproblem of choosing the block at the node with low-compute-high-memory.
Here are some comments:
1. Overall, it is not very clear why the above is the only worthwhile strategy to try. Atleast empirical comparison with popular distributed algorithms (that ignore the resource capabilities) is necessary in order to access the merit of this proposal.
2. More importantly, the proposed split-up of workload in inherently not parallelizable. Hence a time-delay strategy is employed. Though empirically it is shown that it works, it not clear why this is the way to go?
3. In literature of block co-ordinate descent scoring functions (similar to (6) ) are popular. An empirical comparison with strategies like those used in SMO (libsvm etc.; first order and second order criteria for sub-optimality) seems necessary.
4. The results of Theorem1 and Theorem2 are interesting (I have not been able to fully check the correctness of the proofs).
5. Refer eqn (2). The duality is lower-bounded by the expression on LHS in (2). It is need not be equal to it.
I have read the rebuttal and would like to stay with the score.


Approximation Bounds for Hierarchical Clustering: Average Linkage, Bisecting K-means, and Local Search
Hierarchical clustering is a data analysis method that has been used for decades. Despite its widespread use, the method has an underdeveloped analytical foundation. Having a well understood foundation would both support the currently used methods and help guide future improvements. The goal of this paper is to give an analytic framework to better understand observations seen in practice. This paper considers the dual of a problem framework for hierarchical clustering introduced by Dasgupta [Das16]. The main result is that one of the most popular algorithms used in practice, average linkage agglomerative clustering, has a small constant approximation ratio for this objective. Furthermore, this paper establishes that using bisecting k-means divisive clustering has a very poor lower bound on its approximation ratio for the same objective. However, we show that there are divisive algorithms that perform well with respect to this objective by giving two constant approximation algorithms. This paper is some of the first work to establish guarantees on widely used hierarchical algorithms for a natural objective function. This objective and analysis give insight into what these popular algorithms are optimizing and when they will perform well.

The paper extends the work of Dasgupta towards defining a theoretical framework for evaluating hierarchical clustering. The paper defines an objective function that is equivalent to that of Dasgupta in the sense that an optimal solution for the cost function defined by Dasgupta will also be optimal for the one defined in this paper and vice versa. The behaviour of approximate solution will differ though because the cost function is of the form D(G) - C(T), where D is a constant (depending on the input G) and C(T) is the cost (defined by Dasgupta) and T is the hierarchical solution. The authors show a constant approximation guarantee w.r.t. their objective function for the popular average linkage algorithm for hierarchical clustering. They complement this approximation guarantee by showing that this approximation guarantee is almost tight. They also show that the decisive algorithm for hierarchical clustering gives poor results w.r.t. their objective function. They also claim to show approximation guarantee for a local search based algorithm.
Significance:
The results discussed in the paper are non-trivial extensions to the work of Dasgupta. Even though the model is similar the the previous work, it is interesting in the sense that the paper is able to give theoretical guarantees for the average linkage algorithm that is a popular technique for hierarchical clustering.
Originality:
The paper uses basic set of techniques for analysis and is simple to read. The analysis is original as per my knowledge of the related literature.
Clarity:
The paper uses simple set of techniques and is simple to read. The authors do a fair job in explaining the results and putting things in context with respect to previous work. The explanations are to the point and simple. There are a few places where more clarity will be appreciated. These are included in the specific comments below.
Quality:
There is a technical error in line 261. It is not clear how the very first inequality is obtained. Furthermore, the last inequality is not justified. The authors should note that as per the analysis, in case (iii) you assume that |B|<=|A| and not the other way around.
Given this, I do not know how to fix these errors in a simple way.
Specific comments:
1. Please use proper references [C] can mean [CC17] or [CKMM17]
2. Line 69: “Let leaves …”. I think you mean leaves denote the set of leaves and not number.
3.Line 91: Please correct the equation. The factor of n is missing from the right hand side.
4. Line 219: Figure 1 is missing from the main submission. Please fix this, as the proof is incomplete without the figure which is given in the appendix.


Cross-Spectral Factor Analysis
In neuropsychiatric disorders such as schizophrenia or depression, there is often a disruption in the way that regions of the brain synchronize with one another. To facilitate understanding of network-level synchronization between brain regions, we introduce a novel model of multisite low-frequency neural recordings, such as local field potentials (LFPs) and electroencephalograms (EEGs). The proposed model, named Cross-Spectral Factor Analysis (CSFA), breaks the observed signal into factors defined by unique spatio-spectral properties. These properties are granted to the factors via a Gaussian process formulation in a multiple kernel learning framework. In this way, the LFP signals can be mapped to a lower dimensional space in a way that retains information of relevance to neuroscientists. Critically, the factors are interpretable. The proposed approach empirically allows similar performance in classifying mouse genotype and behavioral context when compared to commonly used approaches that lack the interpretability of CSFA. We also introduce a semi-supervised approach, termed discriminative CSFA (dCSFA). CSFA and dCSFA provide useful tools for understanding neural dynamics, particularly by aiding in the design of causal follow-up experiments.

The authors propose a factor analysis method called CSFA for modelling LFP data, which is a generative model with the specific assumption that factors, called Elecotomes, are sampled from Gaussian processes with cross-spectral mixture kernel. The generative model is straightforward use of CSM, and the estimation is apparently also a known form (resilient back prop; I never heard of it before). I do like the dCSFA formulation. The proposed method focuses on spectral power and phase relationship across regions, and is claimed to bring both better interpretability and higher predictive power. They also extend CSFA to be discriminative to side information such as genetic and behavioral data by incorporating logistic loss (or the likes of it). The authors applied the proposed method to synthetic and real mouse data, and compared it with PCA.
== issues ==
=== Page limit ===
This paper is 10 pages. (references start at page 8 and goes to page 10)
=== Section 2.1 ===
The authors claim that reduced dimensionality increases the power of hypothesis testing. In general, this is not true. Any dimensionality-reduction decision is made implicitly upon a hypothesis of dimensionality. The hypothesis testing performed after reduction is conditional on it, making the problem more subtle than presented.
=== Section 2.5 ===
lambda in Eq(7) is chosen based on cross-validation of the predictive accuracy. Why was it not chosen by cross-validation of the objective function itself? The authors should report its value along the accuracy in the result section. Would it be extremely large to make the first likelihood term ineffective?
=== FA or PCA ===
Eq 3 shows that the noise is constant across the diagonal. The main difference between FA and PCA is allowing each dimension of signal to have different amount of noise. This noise formulation is closer to PCA, isn't it? But then I was confused reading sec 3.2. What's the covariance matrix of interest in sec 3.2? Over time? space? factors? frequencies?
=== "Causal" ===
The word 'causal' frequently appears in the first 2 pages of the text, setting high expectations. However, the method itself does not seem to have any explanatory power to discover causal structure or inherent causal hypothesis. Or is the reader supposed to get it from the phase plots?
=== comparison with baseline PCA in frequency domain ===
How was the window of data converted to the "Fourier domain"? Was it simply a DFT or was a more sophisticated spectrum estimator used, e.g., multi-taper methods. I ask this because raw FFT/DFT is a bad estimator for the spectrum (very high variance regardless of the length of time window). Neuroscientists often use packages that support such estimation (e.g., MATLAB package chronux). It would not be fair if those are not taken into account.
=== averaging over 6 mice ===
Was the CSFA/dCSFA for Fig 3 computed to produce common factor across 6 mice? What justifies such approach?
== minor ==
 - L 230, 231: absolute KL div numbers are not useful. At least provide standard error of the estimator. (KL div is often very hard to estimate!!)
 - Subsection 2.1 & 2.3.1 reads like filler texts. (not much content)
 - L 124: $s\tilde B_q$? what's $s$ here?
 - L 228: to to
 - overall writing can be improved


Training Deep Networks without Learning Rates Through Coin Betting
Deep learning methods achieve state-of-the-art performance in many application scenarios. Yet, these methods require a significant amount of hyperparameters tuning in order to achieve the best results. In particular, tuning the learning rates in the stochastic optimization process is still one of the main bottlenecks. In this paper, we propose a new stochastic gradient descent procedure for deep networks that does not require any learning rate setting. Contrary to previous methods, we do not adapt the learning rates nor we make use of the assumed curvature of the objective function. Instead, we reduce the optimization process to a game of betting on a coin and propose a learning-rate-free optimal algorithm for this scenario. Theoretical convergence is proven for convex and quasi-convex functions and empirical evidence shows the advantage of our algorithm over popular stochastic gradient algorithms.

This paper presents an optimization strategy using coin betting, and a variant which works well for training neural networks.  The optimizer is tested and compared to various stochastic optimization routines on some simple problems.
I like this paper because it's an unusual optimization method which surprisingly seems to work reasonably well.  It also has fewer tunable parameters than existing stochastic optimizers, which is nice.  However, I'm giving it a marginal accept for the following reasons:
- The experiments aren't very convincing.  They're training some very simple models on some very small datasets, and results in this regime do not necessarily translate over to "real" problems/models.  Further, success on these problems mostly revolves around preventing overfitting.  I'd actually suggest additional experiments in _both_ directions - additional simple unit test-like experiments (like the |x - 10| experiment, but more involved - see e.g. the "unit tests for stochastic optimization" paper), and experiments on realistically large models on realistically large datasets.
- I don't 100% buy the "without learning rates" premise, for the following reason:  If you divide the right-most term of line 10 of Algorithm 2 by L_{t, i}, then the denominator simplifies to max(G_{t, i}/L_{t, i} + 1, \alpha).  So, provided that \alpha is bigger than G_{t, i}/L_{t, i}, the updates effectively are getting scaled by \alpha.  In general I would expect G_{t, i}/L_{t, i} to be smaller than \alpha = 100 particularly at the beginning of training, and as a result I'd expect the setting of \alpha to have a considerable effect on training.  Of course, if \alpha = 100 well in any experiment of interest, we can ignore it, but arguably the default learning rate setting of Adam works reasonably well in most problems of interest too -but of course, we wouldn't call it  "without learning rates" method.
- Overall, this is an unusual and unconventional idea (which is great!), but it is frankly not presented in a clear way.  I do not get a lot of intuition from the paper about _why_ this works, how it is similar/different from SGD, how the different quantities being updated (G, L, reward, \theta, w, etc) evolve over the course of a typical training run, etc. despite spending considerable time with the paper.  I would suggest adding an additional section to the appendix, or something, which gives a lot more intuition about how and why this actually works.
More specific comments:
- Very high level: "backprop without learning rates" is a strange phrase.  Backprop has no learning rate.  It's an efficient algorithm for finding the gradients with respect to all parameters in your model with respect to the output.  SGD has a learning rate.  SGD is often used for training neural networks.  In neural networks backprop is often used for finding the gradients necessary for SGD.  But you don't need a learning rate to backprop; they are disjoint concepts.
- In your introduction, it might be worth mentioning results that show that the learning rate is one of the most "important" hyperparameters, in the sense that if it's set wrong the model may not work at all, so its correct setting can have a strong effect on the outcome.
- The relation of the second and third inequality in the proof after line 112 took me about 15 minutes to figure out/verify.  It would be helpful if you broke this into a few steps.
- Algorithm 1 is missing some input; e.g. it does not take "T" or "F" as input.
- Calling \beta_{t, i} the "fraction to bet" is odd because it can be negative, e.g. if the gradients are consistently negative then \theta_{t, i} will be negative and 2\sigma(...) - 1 will be close to -1.  So you are allowed to bet a negative amount?  Further, it seems that in general 2\theta_{t, i} can be substantially smaller than G_{t, i} + L_i
- I think you have redefined w_t.  When defining coin betting you use w_t to refer to the bet at round t.  In COCOB w_t are the model parameters, and the bet at round t is (I think) \beta_t, i (L_i + Reward_t, i).
- I think most readers will be most familiar with (S)GD as an optimization workhorse.  Further presentation of COCOB vs. variants of SGD would be helpful, e.g. drawing specific analogies between each step of COCOB and each step of some SGD-based optimizer.  Or, perhaps showing the behavior/updates of COCOB vs. SGD on additional simple examples.
- "Rather, it is big when we are far from the optimum and small when close to it."  Not necessarily - for example, in your |x - 10| example, at the beginning you are quite far from the optimum but \sum_i g_i^2 is quite small.  Really the only thing you can say about this term is that it grows with iteration t, I think.
- In Algorithm 2, the loop is just "repeat", so there is no variable T defined anywhere.  So you can't "Return w_T".  I think you mean "return w_t".  You also never increment, or initialize, t.
- Setting L_{t, i} to the running max of the gradients seems like a bad idea in practice with neural networks (particularly for RNNs) because if at some iteration gradients explode in a transient manner (a common occurrence), then for all subsequent iterations the update will be very small due to the quadratic L_{t, i} term in the denominator of line 10 of algorithm 2.  It seems like you would want to set an explicit limit as to the values of |g_{t, i}| you consider, e.g. setting L_{t, i} <- max(L_{t - 1, i}, min(|g_{t, i}|, 10)) or something.
- What version of MNIST are you using that has 55k training samples?  It technically has 60k training images, typically split into 50k for training and 10k for validation.
- I would suggest using an off-the-shelf classifier for your MNIST experiments too, since you are missing some experiment details (how were the weights initialized, etc).
- You say your convolutional kernels are of shape 5 x 5 x 3 and your pooling is 3 x 3 x 3, I think you mean 5 x 5 and 3 x 3 respectively.
- What learning rates did you use for the different optimizers and did you do any kind of search over learning rate values?  This is absolutely critical!


Polynomial time algorithms for dual volume sampling
We study dual volume sampling, a method for selecting k columns from an n ⇥ m short and wide matrix (n  k  m) such that the probability of selection is proportional to the volume spanned by the rows of the induced submatrix. This method was proposed by Avron and Boutsidis (2013), who showed it to be a promising method for column subset selection and its multiple applications. However, its wider adoption has been hampered by the lack of polynomial time sampling algorithms. We remove this hindrance by developing an exact (randomized) polynomial time sampling algorithm as well as its derandomization. Thereafter, we study dual volume sampling via the theory of real stable polynomials and prove that its distribution satisfies the "Strong Rayleigh" property. This result has numerous consequences, including a provably fast-mixing Markov chain sampler that makes dual volume sampling much more attractive to practitioners. This sampler is closely related to classical algorithms for popular experimental design methods that are to date lacking theoretical analysis but are known to empirically work well.

The paper studies efficient algorithms for sampling from a
determinantal distribution that the authors call "dual volume
sampling", for selecting a subset of columns from a matrix.
The main results in the paper are two sampling algorithms for
selecting k columns from an n x m matrix (n<=k<=m):
- an exact sampling procedure with time complexity O(k m^4)
- an approximate sampling, with time complexity O(k^3 n^2 m) (ignoring
log terms).
The approximate sampling algorithm makes use of Strongly Rayleigh
measures, a technique previously used for approximately sampling from
closely related determinantal distributions, like Determinantal Point
Processes. Compared to the exact sampling, it offers a better
dependence on m, at the cost of the accuracy of sampling,
and a worse dependence on k and n. The authors perform some
preliminary experiments comparing their method to other
subset selection techniques for linear regression. It is worth noting
that the plots are not very informative in comparing dual volume
sampling to leverage score sampling which is to my mind the most
relevant baseline.
This determinantal distribution was previously discussed and motivated
in [1], who referred to it just as "volume sampling", and suggested
potential applications in linear regression, experimental design,
clustering, etc., however without providing a polynomial time
algorithm, except for the case of k=n.
The paper addresses an important topic that is relevant to
the NIPS community. The results are interesting, however the
contribution is somewhat limited. To my knowledge, the algorithms (in
particular, the exact sampling algorithm) are much slower compared
to Leverage Score sampling, and the authors did not make a compelling
argument for why DVS is better suited for the task of experimental
design.
[1] Avron and Boutsidis. Faster Subset Selection For Matrices and
Applications, 2013. https://arxiv.org/abs/1201.0127