Deep neural networks trained on a wide range of datasets demonstrate impressive transferability. Deep features appear general in that they are applicable to many datasets and tasks. Such property is in prevalent use in real-world applications. A neural network pretrained on large datasets, such as ImageNet, can significantly boost generalization and accelerate training if fine-tuned to a smaller target dataset. Despite its pervasiveness, few effort has been devoted to uncovering the reason of transferability in deep feature representations. This paper tries to understand transferability from the perspectives of improved generalization, optimization and the feasibility of transferability. We demonstrate that 1) Transferred models tend to find flatter minima, since their weight matrices stay close to the original flat region of pretrained parameters when transferred to a similar target dataset; 2) Transferred representations make the loss landscape more favorable with improved Lipschitzness, which accelerates and stabilizes training substantially. The improvement largely attributes to the fact that the principal component of gradient is suppressed in the pretrained parameters, thus stabilizing the magnitude of gradient in backpropagation.
3) The feasibility of transferability is related to the similarity of both input and label. And a surprising discovery is that the feasibility is also impacted by the training stages in that the transferability first increases during training, and then declines. We further provide a theoretical analysis to verify our observations.

I thank the authors for the clarifications and the modifications to the paper. However, I still lean towards rejection. While the authors provided detailed explanations on some of my points here, most of these are still not in the paper, so the reader would still probably be confused. There are new figures in the appendix, but they are not referred to from the text. The paper feels like a hastily compiled collection of weakly related, somewhat anecdotal and not necessarily clearly explained and motivated experiments. While I believe there is interesting content, with the current presentation style it is really difficult to digest. As an advice, I think it may be better to not squeeze in all the results into one paper, but rather focus on some aspects and analyze them really well, with clear explanation, motivation, and preferably with demonstrating practical gains that result from the analysis - not just hypothetical, but supported by experiments.
---
The paper studies transfer of representations learned by deep networks across datasets and tasks. Namely, the paper analzes the standard setup with pre-training on one dataset and further fine-tuning on a different  dataset. Results include both experiments and theory. On the experimental side, the focus is mainly on TODO. On the theory side, an analysis of transfer in two-layer fully connected networks, based on the ICML-2019 work of Arora et al., is proposed.
I lean towards rejecting the paper, since the presentation and the technical quality are somewhat substandard. This is mainly based on evaluation of the experimental results, since I am not an expert in this subfield of theory and was therefore not able to check the statements and the proofs thoroughly. On the presentation side, many details are missing, which often makes understanding difficult and will likely lead to the results not being reproducible. On the experiments side, the issue is that they are quite anecdotal and have multiple confusing or questionable details, as specified below.
Pros:
1) An interesting topic of trying to understand generalization and transfer in deep learning
2) Multiple types of experiments, including visualizations of loss landscapes at convregence and at initialization, plots of Hessian eigenvalues, measuring the deviation of the weights from their initial values, measuring the variance of the gradients of the weights, measuring the transfer between different datasets, measuring the transfer performance depending on the durarion of pre-training.
3) Theoretical analysis. As mentioned, I cannot really judge about the quality of the results.
Cons:
1) The presentation is quite confusing.
1a) The paper includes many different experiments as well as theory, and it is not very clear how these all come together and what message do they give to the reader. The paper states at one point that it "may stimulate further algorithmic advances", and it would be great if there was a bit more elaboration on this.
1b) Experimental methodology is not presented in the main paper and not referred to. Some of it is described in the appendix, but also not too detailed, for instance the duration of ResNet training is not clear, the details of loss landscape visualization are confusing (for instance, the phrase "... i.e. 0.1× gradient and a total of 200 × 200 grids"),
1c) The paper is over 9 pages, which is more than the recommended length of 8 pages.
1d) Scaling in Figure 8(b) is quite suboptimal, it is impossible to read the test accuracy results.
1e) Minor issues:
 - No color-code in Figure 3 (unclear what values do the colors correspond to) and it does not seem to be referred to in the text.
 - Page 5: "predictive" -> "predictable"?
 - Page 6 "While pretraining on sufficiently large datasets..." - I do not think the experiments tell anything about the dependence of the effect on the size of the dataset, so this phrasing is not justified
2) I found many of the experiments confusing or unconvincing. This is partially affected by the aforementioned issues with presentation.
2a) In Table 1 and Figure 2, it is unclear if difference in generalization between the dataesets is due to similarity to ImageNet (by the way, ImageNet is only mentioned in the caption of Table 1, but not in the text) or due to the inherent properties of the datasets (perhaps some are naturally more difficult or prone to overfitting). I think numbers for training from scratch would be helpful for disambiguating the two cases.
2b) It is unclear why is the norm difference normalized by the square root of the number of target examples. This choice is not justified and it seems it can very noticeably affect the results in counter-intuitive way. For instance, if I understand right, if one duplicates each example in the training set, these values will change, which seems somewhat counter-intuitive. Would it make more sense to normalize by the initial norm of the weights?
2c) In Figure 4, it is unclear if the permutations of the weights are accounted for. The output of a deep network is invariant under a wide variety of weight permutations, and it is natural that networks trained from different random initializations may converge to weights permuted in different ways. In order to meaningfully compare the weight vectors of these networks, one would have to first "align" them, which is a non-trivial task. Another issue I have with the results on Figure 4 is that I don't find them all that counter-intuitive: it seems natural that the weights stay relatively close to the pre-trained network when fine-tuned for that (paritally because the aforementioned permutation symmetry is already largely broken during pre-trianing).
2d) It is unclear which downstream task is analyzed in Figure 6. Moreover, the plot seems to mix two factors: magnitue of the gradient and the smoothness of the loss landscape. Would it not make more sense to make fixed-size steps in the direction of the gradient? Moreover, I am wondering if the difference between the loss values is simply because the overall loss magnitude is larger for the randomly initialized network?
2e) A few parts of subsection 5.1 are confusing. It is not clear where does the interpretation of Figure 8 follow from (perhaps partially because figure 8(b) is partially difficult to read). \psi and W are not defined. What does it move that the weight matrices "show no improvement"? Where are the results on Caltech-101 and Webcam? Why is Food-101 mentioned in the text, but CUB-200 shown in the plots?


The ability to forecast a set of likely yet diverse possible future behaviors of an agent (e.g., future trajectories of a pedestrian) is essential for safety-critical perception systems (e.g., autonomous vehicles). In particular, a set of possible future behaviors generated by the system must be diverse to account for all possible outcomes in order to take necessary safety precautions. It is not sufficient to maintain a set of the most likely future outcomes because the set may only contain perturbations of a dominating single outcome (major mode). While generative models such as variational autoencoders (VAEs) have been shown to be a powerful tool for learning a distribution over future trajectories, randomly drawn samples from the learned implicit likelihood model may not be diverse -the likelihood model is derived from the training data distribution and the samples will concentrate around the major mode of the data. In this work, we propose to learn a diversity sampling function (DSF) that generates a diverse yet likely set of future trajectories. The DSF maps forecasting context features to a set of latent codes which can be decoded by a generative model (e.g., VAE) into a set of diverse trajectory samples. Concretely, the process of identifying the diverse set of samples is posed as DSF parameter estimation. To learn the parameters of the DSF, the diversity of the trajectory samples is evaluated by a diversity loss based on a determinantal point process (DPP). Gradient descent is performed over the DSF parameters, which in turn moves the latent codes of the sample set to find an optimal set of diverse yet likely trajectories. Our method is a novel application of DPPs to optimize a set of items (forecasted trajectories) in continuous space. We demonstrate the diversity of the trajectories produced by our approach on both low-dimensional 2D trajectory data and high-dimensional human motion data. (Video)

This paper presents a novel approach for forecasting object trajectories (e.g., predicted paths of vehicles) forcing diversity of outputs. The authors adopt determinantal point processes (DPPs) to capture the diversity and propose a diversity sampling function (DSF) which consists of a neural network. Its trainable parameter (i.e., \gamma) maps the past trajectory into a set of latent codes and they are decoded to feasible trajectories by a pre-trained conditional variational autoencoder (cVAE). The DSF is trained by maximizing the diversity of the output trajectory. But, since the standard log-likelihood function can be singular, they additionally present an objective function for diversity by maximizing the expected cardinality, which admits replicated outputs. In experiments, the proposed method finds more diverse paths than other competitors under both for synthetic 2D objects and human motions.
This paper is well-written and easy to understand. The contribution can be important as it performs better than other generative networks that are not forcing diversity. Unlike them, the proposed method can capture the diversity trajectories, which requires for safety-critical applications.
Main concerns:
1. The DPP kernel consists of a similarity (equation (8)) and a quality score (equation (9)). However, it is unnatural that the similarity is defined in the data space (x) and the quality is defined in latent space (z). A more naive approach is to define the DPP kernel as a function of x or z. Is there any specific reason to define those scores are defined in different space?
2. To maximize the expected cardinality, it is enough that the eigenvalues of L(\gamma) become large. Does the network find the trivial solution? E.g., all eigenvalues are the same as a very large value. In addition, the proposed diverse loss is not a new approach. The maximum induced cardinality of a DPP was proposed by [1] and its various properties were also studied therein.
3. In a work of [1], a method to optimize the induced cardinality (a similar to diverse loss in this paper) is proposed. And it would be great to compare the maximum induced cardinality (also can be approximated by the greedy algorithm) to the MAP for inference of diverse trajectory.
4. It is also possible to apply DPP MAP inference to a set of latent codes generated from the encoder of cVAE. Then, the decoder can map the diverse latent variables to feasible data trajectories. Are these outputs also diverse? or does the diversity of latent space reflect the diversity in the data trajectory space?
5. Computing the gradient of the proposed diverse loss is expensive since it is the trace of an inverse of a parameterized matrix. How long does it take to learn the proposed DSF compared to other methods?
6. To avoid that the determinants become zero, a popular choice is to shift all eigenvalues with a small amount (this can be done by adding eps * identity matrix to the kernel matrix). Did the authors investigate other practical diverse losses?
7. It seems to be possible to train the parameters of the kernel matrix, i.e., k in equation (8) and \omega in equation (9). Did the author try to learn those parameters?
Overall, this work proposes an approach combined cVAE with a DPP for forecasting diverse trajectories and the empirical results are promising as it outperforms other methods. But, its novelty is incremental and competitors are not actually the models capturing diversity. I vote for a weak acceptance but depending on clarifications on the above concerns in an author response, I would be willing to increase the score.
Minor comments:
1 . Please specify the network architecture of DSF and details on the parameter \gamma.
[1] Gillenwater, Jennifer A., et al. "Maximizing induced cardinality under a determinantal point process." Advances in Neural Information Processing Systems. 2018.


In the multi-objective reinforcement learning (MORL) paradigm, the relative importance of each environment objective is often unknown prior to training, so agents must learn to specialize their behavior to optimize different combinations of environment objectives that are specified post-training. These are typically linear combinations, so the agent is effectively parameterized by a weight vector that describes how to balance competing environment objectives. However, many real world behaviors require non-linear combinations of objectives. Additionally, the conversion between desired behavior and weightings is often unclear. In this work, we explore the use of a language based on propositional logic with quantitative semantics-in place of weight vectors-for specifying non-linear behaviors in an interpretable way. We use a recurrent encoder to encode logical combinations of objectives, and train a MORL agent to generalize over these encodings. We test our agent in several grid worlds with various objectives and show that our agent can generalize to many never-before-seen specifications with performance comparable to single policy baseline agents. We also demonstrate our agent's ability to generate meaningful policies when presented with novel specifications and quickly specialize to novel specifications.

This paper proposes using logical specifications to facilitate Q-learning in multi-objective reinforcement learning (MORL). Empirically the proposed method can generalize to unseen reward specifications with performance competitive to agents being fully trained in the new environment. The proposed setting employs a more expressive objectives space induced by propositional logic. The proposed method uses a recurrent encoder to embed specifications into vectors and uses them to parametrize the Q-function.
Overall, I weakly recommend accepting the submission for the following reasons: (+) it proposes using propositional logic to specify reward functions, which broadens the objective space in an interpretable way, (+) the learned objective embedding demonstrates the ability to generalize to unseen environments(rewards). However, there is still room for improvement. I will raise my score if the following problems are addressed: (-) Needs more diverse experiments (instead of grid worlds) to support the paper (-) It might be hard to express objectives using logic formulas in real-world applications.
More specifically, the problems of the paper are,
(-) The scalability issue. The most complicated problem in the experiments has a 20x20 state space, which is pretty small for a typical RL problem. I wonder whether the model is still generalizable for larger problems. Even in the case of 20x20 grids, we can notice the gap between the baseline and the proposed method.
(-) The assumption. This paper assumes the logical specifications are given by human. However, we usually don't know how to describe the true objective with logic formulas. Sometimes, finding the specification(reward) itself is as difficult as finding a good policy. Is it possible that we can relax this assumption?
Minor comments:
Figure 6: The word "baseline" here is misleading. A better choice would be "upper-bound"?


Deep unfolding methods design deep neural networks as learned variations of optimization methods. These networks have been shown to achieve faster convergence and higher accuracy than the original optimization methods. In this line of research, this paper develops a novel deep recurrent neural network (coined reweighted-RNN) by unfolding a reweighted 1 -1 minimization algorithm and applies it to the task of sequential signal reconstruction. To the best of our knowledge, this is the first deep unfolding method that explores reweighted minimization. Due to the underlying reweighted minimization model, our RNN has a different soft-thresholding function (alias, different activation function) for each hidden unit in each layer. Furthermore, it has higher network expressivity than existing deep unfolding RNN models due to the over-parameterizing weights. Moreover, we establish theoretical generalization error bounds for the proposed reweighted-RNN model by means of Rademacher complexity. The bounds reveal that the parameterization of the proposed reweighted-RNN ensures good generalization. We apply the proposed reweighted-RNN to the problem of video-frame reconstruction from low-dimensional measurements, that is, sequential frame reconstruction. The experimental results on the moving MNIST dataset demonstrate that the proposed deep reweighted-RNN significantly outperforms existing RNN models.

Authors proposed a deep RNN via unfolding reweighted l1-l1 minimization, where reweighted l1-l1 minimization algorithms are applied to a video task.
Overall, the paper is well explained in a theoretical part and exhibits a good result compared with other conventional RNN methods in the experiment. In Section 3, authors formulate Rademacher complexities for both conventional and proposed method, which shows the generalization performance of the proposed method when d increases. And this is empirically highlighted in Table 3 in Section 4.
Major points:
Section 1:
-	First part of the introduction can be confusing because Eq. (1) sounds like representing dictionary learning framework (plus DNN is immediately described after Eq. (1) instead of RNN) and RNN is not explicitly written. It should be clearly written and flow should be considered.
Section 2:
-	It is hard to get how parameter g in Eq. (3) derives.
Section 3:
-	How to build network depth d for the network? A figure should be required.
Section 4:
-	Even though previous papers (e.g., Wisdom et al. and Le et al.) just focus on single dataset like moving MNIST, I believe testing on language data is also quite important (this is a full paper and exhaustive experiments should be mandatory). For example, it may be good to use Penn TreeBank dataset to make a comparison.
-	In Table 3, how did you set LSTM deeper? Is it a stacked LSTM?
-	Existing RNN methods should include other variations of LSTM (in particular, SOTA methods are welcomed) such as bidirectional LSTM and LSTM with attention mechanism. It should be better to compare with these methods.
Appendix:
-	It would be helpful for readers to show interpretabilities of the model additionally. For example, visualizing features from each RNN model would be beneficial.
Minor points:
-	After introduction of unfolding reweighted l1-l1 minimization, how did the computational cost increase compared to previous l1-l1 minimization?
-	In Section3, for easiness to readers, it may be good to briefly summarize how does the predictor’s generalizability and Rademacher complexities relate.


Many times, working with data starts with writing code to build a model from the features x to the response y. Answering questions about the data can require understanding what parts of the features x influence the response y. We show that using the KL-divergence within a randomization test discovers important features, while allowing for the reuse of model code. We call this method AMI-CRT. AMI-CRT requires running models for each randomization. We develop a faster variant called FAST-AMI-CRT, and show it is robust to errors in the randomization. Questions of feature importance can also be asked at the level of an individual sample. We provide an example to show that optimal predictive models are insufficient for instance-wise feature selection. We show that the estimators from FAST-AMI-CRT can also be reused to find important features in a particular instance. We evaluate our method on several simulation experiments, on a genomic dataset, a clinical dataset for hospital readmission, and on a subset of classes in ImageNet. Our methods outperform several baselines in various simulated datasets, identifies biologically significant genes, selects the most important predictors of a hospital readmission event, and identifies distinguishing image regions in an image-classification task.

# Paper summary
This paper addresses supervised feature selection: given a D-dimensional input variable x = (x_1, ..., x_D), and a response variable y, the goal is to find a subset of "useful" features in x. Here, a feature x_j is useful if it is dependent on y even when conditioning on all other input variables (denoted by x_{-j}, which is a set). A generic procedure that can produce a p-value for each feature (allowing on to test each feature whether it is useful) is the conditional randomization test (CRT) proposed in Candes et al., 2018.  For the CRT to produce a valid p-value for each feature (input dimension) x_j, one needs to specify a test statistic that measures conditional dependence between x_j and y given the rest of the features.
This paper contributes the following results:
1. Propose using an estimate of the f-divergence for the conditional dependence measure and use it with the CRT (section 2.2).
2. Measuring the conditional dependence with an f-divergence requires estimating a few conditional density functions. The paper considers the KL divergence as a special of f-divergence. This particular choice turns out the reduce the number of conditional density functions that have to be estimated (section 2.3). The paper also shows that the resulting conditional measure coincides with what is known as the Additional Mutual Information (AMI) studied in Ranganath & Perotte, 2018.
3. The paper also studies instance-wise feature selection i.e., selecting a subset of input features which can explain the response specifically for one instance (example) x.  Yoon et al., 2019 proposed a criterion to decide the importance of a feature for instance-wise feature selection (definition 2). Briefly, a feature x_j is deemed important if q(y | full x) > q(y | x without the jth feature), where q is the conditional density function of y given x. _Contribution_: The paper notes that this criterion may fail and derive sufficient conditions (Definition 3) under which this approach will always work.
In simulation on toy problems, the paper shows that the proposed method (KL divergence + CRT) has the highest mean under the ROC curve (Table 2), compared to competing methods. In real problems on images, the paper shows that the proposed instance-wise feature selection can be used to select relevant image patches (features) that explain the class of the input images. The paper also conducts experiments on hospital readmission data (Section 4.3), and genomics data (Section 4.2).
# Review
The paper is overall well written with some parts that can be improved (details below). Introduction and related work in section 1 are easy to follow. The paper is also mostly self-contained and friendly to non-specialists who may not work on feature selection primarily. My concerns are
1. I find that the amount of contribution is not sufficient. CRT is known from Candes et al., 2018. The present paper proposes using KL-divergence with it. This can be interesting if the combination gives some clear advantages.  Unfortunately I do not find that this is the case. It turns out that one still needs to learn two conditional density functions (see lines 3-4 in Algorithm 1). Further and even more concerning, one has to refit another conditional density function *for each draw from the null distribution* (see "Fit regression" in the loop in Algorithm 1). As an intermediate step for solving the original feature selection problem, I find that learning conditional density functions is a much more difficult problem. All these limit the novelty of the idea. While the title of the paper contains "model-agnostic", the idea of fitting conditional density functions seems to contradict it. The paper could have considered some nonparametric conditional dependence measures but did not. For instance, see
Kernel-based Conditional Independence Test and Application in Causal Discovery
Kun Zhang, Jonas Peters, Dominik Janzing, Bernhard Schoelkopf
2012
and other papers that extend this paper.
Why was the approach of fitting conditional density functions chosen?
2. Related to the previous point, refitting a conditional density model for each draw from the null distribution must be very costly computationally. This point is never addressed in the paper.
3. Lemma 1 states that the expected f-divergence is a "proper statistic" (in the sense of Definition 1) i.e., p-value is uniformly distributed if the feature is not useful, and vanishes (asymptotically) if the feature is useful. This result unfortunately relies on a strong assumption that there is a consistent estimator for the f-divergence. In fact, the proof does not even rely on the fact that it is an f-divergence. It can be any divergence D(p,q) such that D(p,q) > 0 if  p!=q and D(p,p) = 0. In the proof in section C.2 in the appendix, existence of the quantile function $(F^{-1}_N)$ is never discussed. I can see the first part of the proof (under the alternative H1). But I do not see the second part (under H0). Since $\hat{f}$ is a consistent estimator by assumption, as N goes to infinity, the two quantities in the indicator function (in expectation) should both go to the same constant. Isn't this the case?
4. As a contribution, the paper states a sufficient condition in Definition 3 under which instance-wise feature selection with the approach in Definition 2 is *always* possible. When does the condition hold in practice? How do we know if it holds or not? If it does not, what can go wrong?
5. Toy experiments: What is D in Xor and Orange? Where is the "selector" problem in Table 2? In table 2, "lime" and "shap" also seem to perform well. The paper never explains why the proposed approach is better than other methods (only reporting higher mean are under the ROC curve). This should be possible for toy problems.
# Minor but does affect the evaluation
* Paragraph after Lemma 1: it is unclear why those conditional distributions are required instead of conditional distributions in Eq. 3.
* Section E.1 (appendix), page 16: I think you should have $N( 0.5x_1 + 0.5x_2, \sigma^2_\epsilon)$ instead of
$0.5N(x_1, \sigma^2_\epsilon) + 0.5N(x_2, \sigma^2_\epsilon)$.
# Things that can be improved. Did not affect the score.
* Section 1.1: the sentence about permutation tests is vague.
* Page 2, our contributions: "necessary" should be "sufficient"?
* Section 2, conditional randomization tests: This paragraph is unfortunately not well written even though it is a very important prerequisite of this work. For instance, at " ... replaced by samples of $\tilde{x}_j^{(i)}$ that is conditionally independent of the outcome...", at that point, it is unclear "conditioning on what". Following this sentence, one approach might be to replace $\tilde{x}_j^{(i)}$ with a constant (which is independent of everything else). It is not until definition 1 that this becomes clearer. Also, the "null hypothesis" (which is in the first line of equation 2) is never stated throughout the paper.
* Eq 1: that (i) is unclear. Should state that for i=1,...,N.
* Eq 2: rewrite the second line. The left hand side states that the p-value "converges in distribution to". The second line should be just 0.
* After eq.6, how to choose T (the number of bins) in practice?
* Definition 3 is actually a proposition? It is unclear what is being defined there.
* The word "complete conditional knockoffs (CCKs)" appears for the first time in Section 3.2 without any explanation.
* Orange skin on page 8: what is "~ exp(...)"? An exponential distribution, or just exponential function?


Variational Bayesian Inference is a popular methodology for approximating posterior distributions in Bayesian neural networks. Recent work developing this class of methods has explored ever richer parameterizations of the approximate posterior in the hope of improving performance. In contrast, here we share a curious experimental finding that suggests instead restricting the variational distribution to a more compact parameterization. For a variety of deep Bayesian neural networks trained using Gaussian mean-field variational inference, we find that the posterior standard deviations consistently exhibit strong low-rank structure after convergence. This means that by decomposing these variational parameters into a low-rank factorization, we can make our variational approximation more compact without decreasing the models' performance. Furthermore, we find that such factorized parameterizations improve the signal-to-noise ratio of stochastic gradient estimates of the variational lower bound, resulting in faster convergence.

The paper considers variational Bayesian inference (learning) for neural networks assuming that both the prior distributions and the posterior distributions of the network weights are factorising Gaussians. It is well known that the respective optimisation task for the parameters of the posterior weight distributions (ELBO) is tractable by stochastic gradient ascent.  The authors propose to simplify (restrict) the model even further, by assuming that the posterior variances for fully connected layers, (when seen as a matrix) have low rank. It is pretty obvious that the ELBO optimisation task remains tractable in this case.
The authors perform two types of experiments to show that the proposed simpler model does not decrease the performance of the network, when compared to the full rank factorising model. They first show that the learned variances of dense layers indeed exhibit a low rank structure for three models learned on the corresponding data (MLP on MNIST, LeNet on CIFAR and LSTM on IMDB). Then, in a second step, they consider the proposed rank constraint during learning and show that with rank k >= 2 the models are able to achieve performance competitive with the full rank model and, moreover, exhibit better signal to noise ratio for the gradients during learning.
The paper is well written, clearly structured and technically correct. However, in my opinion, its novelty and new insights are restricted, which is why I suggest to reject the paper. The reasons for this are the following.
- The authors restrict their analysis to dense layers only. Moreover, it remains conceptually unclear, why and when the proposed model should be useful and represent a good approximation of the full rank model.
- The experimental study is restricted to small models, e.g. in the case of CIFAR a quite shallow version of LeNet which achieves only ~45% validation accuracy. It remains unclear, whether the observed low rank structure of the variance matrices (of the dense layers) will scale to deeper models that could achieve competitive validation accuracies.
- When measuring the impact of the low rank tying of the posterior variances, the authors compare to the full rank model only. I am missing the "point estimate" baseline for these models. For if the the positive impact of the Bayesian inference approach with the full rank model is small when compared to the point estimate, then, as a consequence, the impact of the low rank tying must be small when compared to the full rank model.
- The numbers reported in Table 3 (second group of experiments) raise some questions not addressed by the authors. It remains unclear, why the low rank model with ranks 1,2,3 gives better accuracies on CIFAR than the full rank model. The same holds for the LSTM experiment. This may indicate some kind of "overfitting" and should have been analysed by the authors in order to "disentangle" possible overfitting issues from the question of validity of the proposed low rank model.


The hippocampus has long been associated with spatial memory and goal-directed spatial navigation. However, the region's independent role in continual learning of navigational strategies has seldom been investigated. Here we analyse populationlevel activity of hippocampal CA1 neurons in the context of continual learning of two different spatial navigation strategies. Demixed Principal Component Analysis (dPCA) is applied on neuronal recordings from 612 hippocampal CA1 neurons of rodents learning to perform allocentric and egocentric spatial tasks. The components uncovered using dPCA from the firing activity reveal that hippocampal neurons encode relevant task variables such decisions, navigational strategies and reward location. We compare this hippocampal features with standard reinforcement learning algorithms, highlighting similarities and differences. Finally, we demonstrate that a standard deep reinforcement learning model achieves similar average performance when compared to animal learning, but fails to mimic animals during task switching. Overall, our results gives insights into how the hippocampus solves reinforced spatial continual learning, and puts forward a framework to explicitly compare biological and machine learning during spatial continual learning.

This work focuses on the problem of continual learning. It first proposes an analysis of neural recordings from rodents hippocampus that perform a task related to continual learning. The authors then claim to identify similarities in representations of behaviorally relevant variables between biological networks and standard artificial RL systems. Finally the authors propose a DQN implementation of the task reaching performance similar to rodents, although the implementation fail to perform continual learning.
I have found the results of the analysis of neural data interesting and well explained. However I haven’t been able to judge positively the main part of the manuscript pertaining to the comparison with artificial agents, maybe because of the lack of clarity in the exposition of the results (see detailed comments below). As such I think this manuscript is not ready yet for publication in ICLR.
Detailed comments:
Introduction:
The introduction reads well, it would be useful to have a short paragraph explaining the vast problem of continual learning, and situating the approach of the authors in this vast field. In particular it would be useful to explain in what sense the task considered is related to continual learning (switches between allocentric and egocentric tasks that are not informed by experimenter, but that the rodent has to figure out), because for now it is only briefly stated in the legend of figure 1.
Analysis of neural data:
Analysis of neural data using dPCA reveals interesting results regarding the representations of correct/incorrect trials, allocentric/egocentric tasks, temporal structure. Globally these results are well presented. Regarding figure 3, it would be nice to define what are early and late trials, is it a distinction inside a block where late trials are just before a switch. Or are late trials after weeks of training, in which case I do not really see the link with continual learning ?
Comparison between standard RL and hippocampal representations:
The idea to compare hippocampal representations and standard RL implementations is original and interesting. The exposition of the results, however, lacks important information for me to assess the relevance of these comparisons:
Fig4A(i): could the author explain what is on the x-axis ? What is the 1-D environment mentioned in the caption (in SM a 5*5 grid world is mentioned) ? Why not run TD on the same setting as Q-learning ? What is the black curve « value after learning » ? Why value depends on the x-axis ?
Fig4C: How are the two curves obtained ? As both tasks share the same Q function, it might be worth explaining how the two curves are obtained.
The authors mention that « policy information is override as starting location changes over » ? Could this fact be explained ? It would seem to me that starting points north s_north and south s_south would have distinct Q(s_north,.) and Q(s_south,.) avoiding overriding.
Also I am quite surprised that Q-learning fails on such a simple task (cf Fig5B), could the authors explain how many trials would be required to reach perfect performance by focusing on either of the two tasks ?
In this section a natural extension could be to use hierarchical RL, or options to model behavioral strategies such as allocentric and egocentric and compare with the hippocampal recordings.
Comparing DQN and animal performance:
A first crucial point to clarify is whether the DQN receives information about what task to perform (it is mentioned: « task-specific information provided to the neural network as input«, «this network with task-specific memory«, then a DQN « without task-specific information » is mentioned). This seems rather crucial, because if information about allocentric VS egocentric task is provided, contrary to the behavioral task for rats, then the network is asked to do strategy switches based on some cues, which is a different problem than continual learning.
If the DQN receives information about whether it is in a allocentric or egocentric trial, I am very surprised it is not able to perform this simple task.

Could the authors explain why the DQN has similar performance as the animal while it is shown to relearn slower than animals (is it because it learns better before task switch).
In figure 7, the curves for the model are very noisy and hence difficult to interpret, it would be nice to show averages over many same task switches, to get a clearer picture.


Deep learning models are known to be vulnerable to adversarial examples. A practical adversarial attack should require as little as possible knowledge of attacked models. Current substitute attacks need pre-trained models to generate adversarial examples and their attack success rates heavily rely on the transferability of adversarial examples. Current score-based and decision-based attacks require lots of queries for the attacked models. In this study, we propose a novel adversarial imitation attack. First, it produces a replica of the attacked model by a two-player game like the generative adversarial networks (GANs). The objective of the generative model is to generate examples which lead the imitation model returning different outputs with the attacked model. The objective of the imitation model is to output the same labels with the attacked model under the same inputs. Then, the adversarial examples generated by the imitation model are utilized to fool the attacked model. Compared with the current substitute attacks, imitation attack can use less training data to produce a replica of the attacked model and improve the transferability of adversarial examples. Experiments demonstrate that our imitation attack requires less training data than the black-box substitute attacks, but achieves an attack success rate close to the white-box attack on unseen data with no query.

The authors propose to use a generative adversarial network to train a substitute that replicates (imitates) a learned model under attack. They then show that the adversarial examples for the substitute can be effectively used to attack the learned model. The idea is straightforward. The proposed approach leads to better success rates of attacking than other substitute-training approaches that require more training examples. Promising experimental results against decision and score-based attach schemes also demonstrate the effectiveness of the proposed approach.
My main concern is that the comparison to other approaches seem unfair, because
(1) Substitute models typically use all training data and query the learned model once per training example; the proposed approach uses fewer training data but needs 1800 queries per example. So it is not surprising that the proposed approach can be better than substitute models. As the authors focus on "practical" scenarios, the number of queries should be fixed as a constraint for all approaches to be fair.
(2) Having said (1), there is not enough detail in this paper to understand the similarity and difference between the proposed approach and decision-based ones (which is claimed to have similar query complexity during training). The authors mix the results with score-based (requiring more information), making the results more confusing to the readers. The authors are encouraged to present more detailed discussions on the comparison with decision-based competitors.
Also,
(3) One thing that is missing from the experiments is how well the replica clones the original model. All the information in the experiments are somewhat "indirect" (success rate, test accuracy, etc.) to answer this question, but there is no direct evidence. Is a well-cloned replica really a better substitute to construct adversarial examples? For instance, for replica with different "cloning accuracy" (rather than test accuracy), is a better replica also a better substitute? The paper fails to answer this question that best matches its design motivation.
The above are my main concerns. A minor one is
(4) The abstract that directly uses notations like T, G and D is horribly hard to read.


Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes. Unfortunately, using large batches is slow and expensive on conventional hardware. Thus, it would be nice if we could generate batches that were effectively large though actually small. In this work, we propose a method to do this, inspired by the use of Coreset-selection in active learning. When training a GAN, we draw a large batch of samples from the prior and then compress that batch using Coreset-selection. To create effectively large batches of 'real' images, we create a cached dataset of Inception activations of each training image, randomly project them down to a smaller dimension, and then use Coreset-selection on those projected activations at training time. We conduct experiments showing that this technique substantially reduces training time and memory usage for modern GAN variants, that it reduces the fraction of dropped modes in a synthetic dataset, and that it allows GANs to reach a new state of the art in anomaly detection.

Training with large batches provides a disproportionate improvement for GANs (e.g. FID drops from 18.65 to 12.39 by simply increasing the batch size by a factor of 8 for BigGAN). The authors point out that not everybody has access to the computing power which is required to run large batches. Therefore, this paper proposes a method to select the image of the batch and thereby obtaining the benefits of large batches while running only small batches.
The authors perform coreset selection of a large set of images (actually a greedy variant from Sener & Savarese). The selected images are then used for training with small batch size. The coreset selection is applied to the inception activations of the images (or a randomly downsampled version of them). Experiments show that the trained GANs obtain better mode coverage, improve anomaly detection,  and obtain GANs with higher FID scores on image synthesis.
The paper is well motivated and solutions that prevent having to use large batches will have a significant impact on the field.
Conclusion: At the moment I recommend a weak reject. The technical contribution of the paper is rather small, and also the depth of analysis could be improved. Moreover, I think experimental results could have been more complete. However, given the importance of the problem addressed in the paper (and the little existing work) I could be open to increasing my score if my concerns are addressed.
Main points:
1. GANs aim to generate high-quality images. In addition, they aim to generate these high-quality images according to the distribution of the train dataset. The main danger of not randomly sampling from the train distribution is that the trained GAN does no longer generate images according to the train distribution. I think this issue should be more clearly discussed and evaluated in the paper (for some applications this might not be a big problem for others yes). For example, if you would introduce mixing coefficients for the Gaussian mixture and Gaussians with different variances (section 4.2), it could be possible that the GAN would generate according to these mixing coefficients with more accuracy than small-GAN (because the coreset selection would introduce a bias). The proposed metrics do not measure this.
Maybe this could simply be solved by adding some final epochs which just sample randomly again?
2. A naive approach to creating large batches is by only updating the network every N batches, and summing the gradients of the N batches. I would like to see this option discussed and compared to. I can see how the BN is different from a large batch but it would still be interesting to see.
3. I think the examination of sampling factors should be explained in more detail. The importance of the sampling factor of the prior is harder to understand (and might be necessary to counter the effect I discuss in 1?) Is it not expected that both these sampling factors should be equal?
Minor points (do not need to be addressed in rebuttal):
-Since FID several other GAN evaluation metrics have been proposed. I think the authors could also consider ‘Assessing Generative Models via Precision and Recall’ or ‘Improved precision and recall metric for assessing generative models’ for a more complete insight insight
-I am not convinced this GAN needs a name (small-GAN), especially since it can be applied to other GAN architectures.
-The claim in the abstract for state of the art in anomaly detection should be removed (no extensive study nor comparison is performed)
-I prefer to see the venue of publication when possible (for example Sener & Savarese is ICLR 2018)


In-silico protein-ligand binding prediction is an ongoing area of research in computational chemistry and machine learning based drug discovery, as an accurate predictive model could greatly reduce the time and resources necessary for the detection and prioritization of possible drug candidates. Proteochemometric modeling (PCM) attempts to make an accurate model of the protein-ligand interaction space by combining explicit protein and ligand descriptors. This requires the creation of information-rich, uniform and computer interpretable representations of proteins and ligands. Previous work in PCM modeling relies on pre-defined, handcrafted feature extraction methods, and many methods use protein descriptors that require alignment or are otherwise specific to a particular group of related proteins. However, recent advances in representation learning have shown that unsupervised machine learning can be used to generate embeddings which outperform complex, human-engineered representations. We apply this reasoning to propose a novel proteochemometric modeling methodology which, for the first time, uses embeddings generated via unsupervised representation learning for both the protein and ligand descriptors. We evaluate performance on various splits of a benchmark dataset, including a challenging split that tests the model's ability to generalize to proteins for which bioactivity data is greatly limited, and we find that our method consistently outperforms state-of-the-art methods.

The authors present a model with state-of-the-art performance for predicting protein-ligand affinity and provide a thorough set of benchmarks to illustrate the superiority of combining learned low-dimensional embedding representations  of both ligands and proteins. The authors then show that these learned representations are more powerful than handcrafted features such as circular fingerprints, etc. when combined into a model that jointly takes as input both the ligand and protein.
I originally suggested to accept the paper, but agree with the other reviewers that the novelty of the work in this paper likely doesn't meet the bar for acceptance given that the most significant contributions of this paper are around combining good ideas from other papers without much additional novelty.
Major comments
Unless I'm misunderstanding the naming conventions used in Table 1, it seems like an omission not to include the performance of DeepPCM+HP+HC in Table 1 to more fully decouple the performance due to the DeepPCM architecture versus the contribution due to using both unsupervised descriptors in combination.
I might expect the performance of the DeepPCM + HP + HC model (not currently shown unless I'm missing it) to exceed the performance of the NIB + HP + HC model based upon the reasoning given in the discussion (i.e., usefulness of joint input training), even if the individual representations were inferior to the unsupervised counterparts.
Since the primary point of the paper is to illustrate the power of combining the unsupervised representations, I'm surprised the aforementioned performance comparison is not prominent. That is, the performance comparison of DeepPCM+HP+HC versus DeepPCM+UP+UC seems like a very central quantity to present and discuss, but appears to be missing currently.
Minor comments
Table 1 was a bit confusing to me at first, because it appears that UD = UP + UC, and HD = HP + HC, but this wasn't obvious to me initially; I'd change the NIB rows to be NIB + HP + HC and NIB + UP + UC, and then you don't even need to define the UD and HD acronyms, which simplifies the table's cognitive load for me and also makes the relationship between the DeepPCM variants more obvious.
> "On the low-coverage-split, we also find that our method significantly outperforms the benchmark."
It would be good to add the %improvement inline here to parallel the other dataset splits listed just before so that you can directly compare the magnitudes.
> "using unsupervised-learned descriptors than when using handcrafted descriptors"
Would be more compelling if you added the DeepPCM + HC + HP performance to Table 1.
> "All hyperparameter optimization of our model was performed on the temporal split"
I think it would improve the paper if the extent to which various architecture choices were optimized over could be included as well; for example, which types, layer sizes, and depths of network architectures were considered in the hyperparam tuning, and any accompanying justification for these design choices.


Optimal Linear Estimation under Unknown Nonlinear Transform
Linear regression studies the problem of estimating a model parameter β
We consider a significant generalization in which the relationship between x i , β * and y i is noisy, quantized to a single bit, potentially nonlinear, noninvertible, as well as unknown. This model is known as the single-index model in statistics, and, among other things, it represents a significant generalization of one-bit compressed sensing. We propose a novel spectral-based estimation procedure and show that we can recover β * in settings (i.e., classes of link function f ) where previous algorithms fail. In general, our algorithm requires only very mild restrictions on the (unknown) functional relationship between y i and x i , β * . We also consider the high dimensional setting where β * is sparse, and introduce a two-stage nonconvex framework that addresses estimation challenges in high dimensional regimes where p n. For a broad class of link functions between x i , β * and y i , we establish minimax lower bounds that demonstrate the optimality of our estimators in both the classical and high dimensional regimes.

This paper concerns the problem of estimating a vector \beta, from +/-1 measurements y_i which depend statistically on linear functions < x_i, \beta >, where the x_i are Gaussian random vectors. The statistical relationship between y_i and < x_i, \beta > is captured through a link function P[ y_i = 1 | X_i = x_i ] = 1/2 f( < x_i, \beta > ) + 1/2. This model is general enough to capture compressed sensing and phase retrieval problems with binary measurements. The paper assumes that f is completely unknown ahead of time, but that it satisfies certain moment conditions. The paper shows how, under these moment conditions, to reduce the problem of estimating \beta to a sparse PCA problem, with covariance matrix generated by pairs of observations. The idea is to look at differences of x_i - x_{i'} and y_i - y_{i'}; the paper proves that the population covariance matrix of \delta_y \delta_x is a spiked identity matrix, where the spike is of the form \beta* \beta*^T. This clever reduction appears to be the main contribution of the work. The paper then proposes and analyzes algorithms for solving the resulting sparse PCA problem. In the ``low-dimensional'' setting in which n > p, it analyzes the power method, while in the ``high-dimensional'' setting with \beta sparse and n < < p, it analyzes a combination of SDP relaxation and truncated power method. The paper presents bounds on estimation error and a minimax lower bounds. The rates of convergence are optimal, once the number of samples is large (n > s^2 log p).  In the high-dimensional regime, the proposed approach requires s^2 log p samples to recover an s-sparse vector. This matches the best known sample complexity for sparse phase retrieval (which may be optimal for efficient algorithms - nobody currently knows). However, it is suboptimal for one-bit compressed sensing. This seems to be the main disadvantage of the proposed approach: for the sparse setting, there seems to be no way to use this reduction with less than s^2 samples; even though the problem might be solvable with about s samples by some other efficient algorithm. Because an improvement to the second stage of the algorithm would consist of a better algorithm for sparse PCA, this bottleneck seems insurmountable. Because of the requirement that n > s^2 log p, I feel the title overclaims a bit. Of course, if there was some result indicating that about s^2 samples are necessary to solve problems in this class (maybe by reduction *from* sparse PCA?), this would go a long way to addressing this issue.   The paper obtains a minimax lower bound of the correct order \sqrt{s log p / n}. This lower bound requires a gap condition on the link function f, such that f is bounded away from 1. This restriction may only be technical in nature, but it seems to rule out most of the interesting examples. For example, in noisy one-bit compressed sensing, where y_i = sign( < x_i, w > + z_i ), with z_i independent gaussian noise, the link function f(t) approaches one as t -> \infty. [As the paper comments, the minimax lower bound also does not apply to the noiseless version of one bit compressed sensing]. It would be helpful if the authors' response can specify some interesting settings which do satisfy the gap condition. There seems to be a minor notational issue in the theorems, as the algorithms can produce either an accurate estimate of \beta* or an accurate estimate of - \beta*. This ambiguity seems inevitable in this setting; please comment. The two stage algorithm for the sparse regime involves SDP and then a nonconvex refinement. The SDP initialization limits its applicability to problems in several thousands of dimensions. This may be find for machine learning problems, but for phase retrieval (where the dimensionality is the number of pixels in the image to be recovered), it is limiting. Is it possible to avoid the SDP initialization, e.g., by simply solving one nonconvex problem?


On the Global Linear Convergence of Frank-Wolfe Optimization Variants
The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity thanks in particular to its ability to nicely handle the structured constraints appearing in machine learning applications. However, its convergence rate is known to be slow (sublinear) when the solution lies at the boundary. A simple lessknown fix is to add the possibility to take 'away steps' during optimization, an operation that importantly does not require a feasibility oracle. In this paper, we highlight and clarify several variants of the Frank-Wolfe optimization algorithm that have been successfully applied in practice: away-steps FW, pairwise FW, fully-corrective FW and Wolfe's minimum norm point algorithm, and prove for the first time that they all enjoy global linear convergence, under a weaker condition than strong convexity of the objective. The constant in the convergence rate has an elegant interpretation as the product of the (classical) condition number of the function with a novel geometric quantity that plays the role of a 'condition number' of the constraint set. We provide pointers to where these algorithms have made a difference in practice, in particular with the flow polytope, the marginal polytope and the base polytope for submodular optimization.
The Frank-Wolfe algorithm [9] (also known as conditional gradient) is one of the earliest existing methods for constrained convex optimization, and has seen an impressive revival recently due to its nice properties compared to projected or proximal gradient methods, in particular for sparse optimization and machine learning applications.
On the other hand, the classical projected gradient and proximal methods have been known to exhibit a very nice adaptive acceleration property, namely that the the convergence rate becomes linear for strongly convex objective, i.e. that the optimization error of the same algorithm after t iterations will decrease geometrically with O((1 − ρ) t ) instead of the usual O(1/t) for general convex objective functions. It has become an active research topic recently whether such an acceleration is also possible for Frank-Wolfe type methods.
Contributions. We clarify several variants of the Frank-Wolfe algorithm and show that they all converge linearly for any strongly convex function optimized over a polytope domain, with a constant bounded away from zero that only depends on the geometry of the polytope. Our analysis does not depend on the location of the true optimum with respect to the domain, which was a disadvantage of earlier existing results such as [34,12,5], and the newer work of [28], as well as the line of work of [1,19,26] which rely on Robinson's condition [30]. Our analysis yields a weaker sufficient condition than Robinson's condition; in particular we can have linear convergence even in some cases when the function has more than one global minima, and is not globally strongly convex. The constant also naturally separates as the product of the condition number of the function with a novel notion of condition number of a polytope, which might have applications in complexity theory.
Related Work. For the classical Frank-Wolfe algorithm, [5] showed a linear rate for the special case of quadratic objectives when the optimum is in the strict interior of the domain, a result already subsumed by the more general [12]. The early work of [23] showed linear convergence for strongly 1
The Frank-Wolfe algorithm [9] (also known as conditional gradient) is one of the earliest existing methods for constrained convex optimization, and has seen an impressive revival recently due to its nice properties compared to projected or proximal gradient methods, in particular for sparse optimization and machine learning applications.
On the other hand, the classical projected gradient and proximal methods have been known to exhibit a very nice adaptive acceleration property, namely that the the convergence rate becomes linear for strongly convex objective, i.e. that the optimization error of the same algorithm after t iterations will decrease geometrically with O((1 − ρ) t ) instead of the usual O(1/t) for general convex objective functions. It has become an active research topic recently whether such an acceleration is also possible for Frank-Wolfe type methods.
Contributions. We clarify several variants of the Frank-Wolfe algorithm and show that they all converge linearly for any strongly convex function optimized over a polytope domain, with a constant bounded away from zero that only depends on the geometry of the polytope. Our analysis does not depend on the location of the true optimum with respect to the domain, which was a disadvantage of earlier existing results such as [34,12,5], and the newer work of [28], as well as the line of work of [1,19,26] which rely on Robinson's condition [30]. Our analysis yields a weaker sufficient condition than Robinson's condition; in particular we can have linear convergence even in some cases when the function has more than one global minima, and is not globally strongly convex. The constant also naturally separates as the product of the condition number of the function with a novel notion of condition number of a polytope, which might have applications in complexity theory.
Related Work. For the classical Frank-Wolfe algorithm, [5] showed a linear rate for the special case of quadratic objectives when the optimum is in the strict interior of the domain, a result already subsumed by the more general [12]. The early work of [23] showed linear convergence for strongly convex constraint sets, under the strong requirement that the gradient norm is not too small (see [11] for a discussion). The away-steps variant of the Frank-Wolfe algorithm, that can also remove weight from 'bad' atoms in the current active set, was proposed in [34], and later also analyzed in [12]. The precise method is stated below in Algorithm 1. [12] showed a (local) linear convergence rate on polytopes, but the constant unfortunately depends on the distance between the solution and its relative boundary, a quantity that can be arbitrarily small. More recently, [1,19,26] have obtained linear convergence results in the case that the optimum solution satisfies Robinson's condition [30]. In a different recent line of work, [10,22] have studied a variation of FW that repeatedly moves mass from the worst vertices to the standard FW vertex until a specific condition is satisfied, yielding a linear rate on strongly convex functions. Their algorithm requires the knowledge of several constants though, and moreover is not adaptive to the best-case scenario, unlike the Frank-Wolfe algorithm with away steps and line-search. None of these previous works was shown to be affine invariant, and most require additional knowledge about problem specific parameters.
Setup. We consider general constrained convex optimization problems of the form:
with only access to:
where A ⊆ R d is a finite set of vectors that we call atoms. 1 We assume that the function f is µ-strongly convex with L-Lipschitz continuous gradient over M. We also consider weaker conditions than strong convexity for f in Section 4. As A is finite, M is a (convex and bounded) polytope. The methods that we consider in this paper only require access to a linear minimization oracle LMO A (.) associated with the domain M through a generating set of atoms A. This oracle is defined as to return a minimizer of a linear subproblem over M = conv(A), for any given direction r ∈ R d .

Report for "On the Global Linear Convergence of Frank-Wolfe Optimization Variants".  ## SummaryThe authors consider the problem of minimizing a strongly convex or partially strongly convex function over a polyhedron. They describe and analyse different variants of the Frank-Wolfe algorithm. The authors show that the different variants of the algorithm allow to bypass the known fact that the vanilla Frank-Wolfe algorithm is not able to take advantage of strong convexity properties of the objective function. This leads to proof of linear convergence for all the variants presented. The analysis is based on the introduction of a combinatorial positive constant that relates to a notion of condition number for polyhedra. Numerical results on both synthetic and real examples are presented.## Main commentsThe paper present various versions of the original algorithm which were introduced previously in the literature. Linear convergence is a new finding for some of the variants and constitutes an interesting contribution. Furthermore, the pyramidal width introduced in the paper may have interesting applications beyond linear oracle based method. The proof arguments are reasonable up to my understanding and I do not see any major issue related to the content of this work. I detail a few comments that should be taken into account in the next paragraph, some of them may need to be reflected in the text. Additional minor comments are given in the following section. ## Detailed comments- It may be worth mentioning what happens for these algorithms when the strong convexity assumption is not verified.- There is a lot of emphasis on the notion of affine invariance. I understand the idea and it is important. But I think it could be clarified. For example, I think that the fact that the linear map is surjective is important. Also I do not think that the statement that A is arbitrary in [17] (line 710) is true as the paragraph dedicated to affine invariance in [17] mentions explicitly the surjectivity. - In the proof of Theorem 3 in the appendix, I think that the conclusion of line 640 is correct but the arguments are not. First, it could be explicitly mentioned what is meant by KKT. Second KKT only expresses stationarity at the first order and I am not sure that this is enough to conclude that the solution of minimizing a linear function on the intersection of a cone and the unit (euclidean) sphere is on the relative boundary of the cone. I do not know what the authors expicitly meant by "KKT", but for example for the cone $\RR_+ \times \RR$, $d= (-1,0)$ does not belong to the cone and $y = (1,0)$ is in the interior of the cone and satisfies some stationarity conditionwhich I guess correspond more or less to the KKT condition the authors have in mind.- The MNP variant actually requires to have a subroutine that minimizes the objective over an affine hull. Since in the model, $f$ is only defined on script M, the setting in which this make sense is slightly different. Furthermore, this has very important practical implications in terms of implementation. This could be emphasized. I guess the line search in step 7 involves the constraints. In this case, this is another significant requirement compared to other variants for which this is explicitly avoided (see line 163).- The authors claim that everything works the same for the partially strongly convex model just by replacing a constant by another. It is a bit fast since the constants in (21) and (34) look quite different, in particular, there are multiplicative 2 factor that do not occur at the same place. Some hints are welcome.- The away step variant under the partial strongly convex model was treated in [4]. Is there a difference in the estimated rates? It would be nice to have a discussion, for example based on the cases for which the pyramidal width is known.- For the simplex, the fact that the pyramidal width is the same as the width deserves more justification. Also the presentation would benefit from a more precise citation of [2] (which result the authors refere to in this paper).- Up to my understanding, the convergence analysis of [12] does not treat the case "strongly convex objective + strongly convex set". This paper actually proposes the linear convergence as an open problem contrary to what is stated in the introduction.  ## Minor comments- The abstract mentions "weaker condition than strong convexity". I would add "of the objective".- May be add some reference about the convergence rates of projected gradient method- Line 99: typo "While the tracking the active set"- Regarding footnote 3, the lower bound actually holds for specific sequences that never reach the face of interest. It does not hold for every sequence, it depends on the initialization and the problem at hand.- Line (151) mentions that the active set is small, while after a large number of iterations it is potentially large, unless there is some specific step performed to optimize the vertex representation.- Line 178: "is to in each step only move", I would add commas- Line 188: typo "due our"- Line 210: typo "more computed more efficiently"- Line 212: "overarching"?- (7) could actually be summarized as $h_t \geq \min{g_t/2, g_t^2/(LM^2)}- The authors conjecture a nonincreasingness property for the constant they introduced. Do they have an intuition why this should be the case beyond the fact that this constant is greater for smaller for the cube than the simplex?- Figure 2 is hardly readable- On page 12 (appendix), the notation $r$ is used in the main text and in footnote 7 to denote two different thing. Similar comment holds for y.- The argument following (14) go really fast and non trivial details are missing. It should be written explicitly that (14) is equivalent to the problem mentioned on line 640. Here also, $y$ and $y^*$ are used to denote the decision variable and solution of two different problems.- On line 689, (14) should be (15).- Line 700 makes a self reference to "the appendix": it is itself part of the appendix.- Line 950, I guess "drop step" should be "swap step".- In (33), $B$ should be $B_1$.


A Recurrent Latent Variable Model for Sequential Data
In this paper, we explore the inclusion of latent random variables into the hidden state of a recurrent neural network (RNN) by combining the elements of the variational autoencoder. We argue that through the use of high-level latent random variables, the variational RNN (VRNN) 1 can model the kind of variability observed in highly structured sequential data such as natural speech. We empirically evaluate the proposed model against other related sequential models on four speech datasets and one handwriting dataset. Our results show the important roles that latent random variables can play in the RNN dynamics.

This paper presents a variational inference approach to learn Recurrent Neural Networks (RNNs) that have stochastic transitions between hidden states. The model is a stochastic nonlinear dynamical system similar to a nonlinear state-space model where the noise between hidden states (i.e. the process noise) is not identically distributed but depends on the hidden state. The parameters of the generative model are learnt simultaneously with a recognition model via maximization of the evidence lower bound.This is a solid paper which bridges the gap between RNNs and nonlinear state-space models. The presentation is clear with an emphasis on presenting the general ideas rather than the technical details. The model itself is simple to describe and the variational training procedure is now standard. There is probably an important engineering effort to achieve the reported results which is not obvious when reading the paper. Also there is no mention of computing machinery or training times.The notation with subindices "< t" sometimes hides the Markovian structure of the model which can be useful when interpreting the factorizations of joint distributions.The literature review does not seem to mention nonlinear state-space models (i.e. nonlinear versions of linear state-space models aka "Kalman filters"). Those models are popular in engineering, robotics, neuroscience and many other fields. The model presented in this paper could have an impact on fields beyond machine learning. It could be useful to be more explicit in the connection with nonlinear state-space models. A few questions:If VRNN-Gauss was not able to generate well-formed handwritting, are all the plots labelled VRNN actually VRNN-GMM?Is there any regularization applied to the model? If not, how was overfitting prevented?Equation (1) gives h_t as a function of x_t while Equation (2) gives a distribution over x_t as a function of h_t. Is this intended?


Fast Convergence of Regularized Learning in Games
We show that natural classes of regularized learning algorithms with a form of recency bias achieve faster convergence rates to approximate efficiency and to coarse correlated equilibria in multiplayer normal form games. When each player in a game uses an algorithm from our class, their individual regret decays at O(T 3/4 ), while the sum of utilities converges to an approximate optimum at O(T 1 )-an improvement upon the worst case O(T 1/2 ) rates. We show a blackbox reduction for any algorithm in the class to achieveÕ(T 1/2 ) rates against an adversary, while maintaining the faster rates against algorithms in the class. Our results extend those of Rakhlin and Shridharan [17] and Daskalakis et al. [4], who only analyzed two-player zero-sum games for specific algorithms.

The paper introduces a new method for regret minimization in multiplayer normal-form games. Specifically, the paper generalizes previous algorithms (Optimistic Mirror Descent and Optimistic Follow the Regularized Leader) for two-player zero-sum games to multiplayer general-sum games.The experimental results compared to Hedge is impressive. The content of the paper would appeal to the NIPS community.I have some specific questions and comments:- pg. 3: why is the root removed from the log when claiming "converges to O(n log(d) sqrt(T)", where in r_i(T) the root contains the log? Is this a typo?- pg. 6: "This is the first fast convergence result to CCE using natural, decoupled no-regret dynamics." I have two problems with this statement: 1. "natural" and "decoupled" are not clearly defined; and how is it "decoupled" anyway-- Corollary 12 upon which this claim is base requires all players to use OFTRL with specific choices of M_i^t and \eta. 2. It is a bit strong: what about Hart & Mas-Colell's regret-matching (i.e. [13])? Either clarify, rephrase, or remove this claim.Some things to think about:- Any ideas or comments on whether these results can be extended in a straight-forward way to the partial information setting using sampling (e.g. Optimistic Exp3)?Some minor points for camera-ready if accepted: - pg. 1: ".. a chink that hints", what is "chink"? - pg. 3: Remove the space after PoA in "(PoA )" - pg. 4: end of Thm4 proof, is there a max_{j \in N} missing in the right-most inequality? Maybe just remove the j subscript after removing the \sum_{j \neq i}? - Fix argmax so that the set being maxed over is entirely under the argmax (i.e. arg is not separated from max) - A number of items in the bibliography need to be fixed. Volume, number, pages are missing for [4], write out the full conference names for [19] and [20].


Regressive Virtual Metric Learning
We are interested in supervised metric learning of Mahalanobis like distances. Existing approaches mainly focus on learning a new distance using similarity and dissimilarity constraints between examples. In this paper, instead of bringing closer examples of the same class and pushing far away examples of different classes we propose to move the examples with respect to virtual points. Hence, each example is brought closer to a a priori defined virtual point reducing the number of constraints to satisfy. We show that our approach admits a closed form solution which can be kernelized. We provide a theoretical analysis showing the consistency of the approach and establishing some links with other classical metric learning methods. Furthermore we propose an efficient solution to the difficult problem of selecting virtual points based in part on recent works in optimal transport. Lastly, we evaluate our approach on several state of the art datasets.

Summary: This paper proposed a novel approach for efficient metric learning. The objective of classic metric learning is to solve the positive semi-definite matrix $M$ such that the distance between examples from the same class get closer while the distance between examples from different classes get further in the distance define by M. Unlike these methods, this paper proposed a idea of virtual point-based regression formulation. The points from the same class are pulled toward the virtual points, thus their distance to each other become closer and their distance to points in other classes become further. The paper describes a closed form solution to compute the regression matrix and two different ways to discover the virtual points. The method is evaluated in 13 different metric learning datasets and compared with several standard baselines.  Pros: - The experiment result is complete and supports the proposed method. It is evaluated in 13 different datasets, linear and RBF kernel, and compared with several baselines. - When learning the metric, the number of constraints only grows with number of virtual points, which is linear to the training size. For existing metric learning, the number of constraints is quadratic to the training size. - The paper is well written and easy to follow. The introduction and related work clearly explains the novelty and key contribution of the proposed method clearly. Supplementary material provides detail explanation of theorem.  Cons: - If I understand it correctly, the selection process described in in Algorithm 1 is quadratic to the size of S because of two argmax over x? - The performance of the proposed method doesn't work well with linear kernel. Suggestion: - Since the proposed method requires less constraints, the learning process should be more efficient than other methods. It would be helpful if we could also see some results for the training time comparison against other methods.- Compare to LMNN, could the proposed method provide additional advantage in adaptive(online) metric learning? - To further demonstrate the benefit of OT on learning set, the author could consider adding the baseline by randomly selecting an example from each class as virtual point.


Streaming, Distributed Variational Inference for Bayesian Nonparametrics
This paper presents a methodology for creating streaming, distributed inference algorithms for Bayesian nonparametric (BNP) models. In the proposed framework, processing nodes receive a sequence of data minibatches, compute a variational posterior for each, and make asynchronous streaming updates to a central model. In contrast to previous algorithms, the proposed framework is truly streaming, distributed, asynchronous, learning-rate-free, and truncation-free. The key challenge in developing the framework, arising from the fact that BNP models do not impose an inherent ordering on their components, is finding the correspondence between minibatch and central BNP posterior components before performing each update. To address this, the paper develops a combinatorial optimization problem over component correspondences, and provides an efficient solution technique. The paper concludes with an application of the methodology to the DP mixture model, with experimental results demonstrating its practical scalability and performance.

Review Summary ============== Score: 6, Marginally above the acceptance thresholdThe proposed method for streaming, distributed inference of DP mixture models presents a nice solution to the cluster identification problem, backed by experiments that are convincing though not rock solid. I'm hesitant to recommend unconditional acceptance, because basic information about how new clusters are created at each minibatch are totally absent, hurting reproducibility.Summary of Paper ================This paper develops a new algorithm for streaming, distributed variational inference for the DP mixture model, with some supplementary material suggesting how to use these insights for many other BNP models. Using a mean-field approximation, the authors consider how to allow multiple worker nodes to process data batches in parallel and then aggregate these results asynchronously. In particular, the authors offer a new solution to the "component identification" problem: how to find correspondence between new clusters created independently by two separate worker nodes. This solution finds an optimal assignment for each new cluster from the current minibatch, merging it with another new cluster if advantageous but keeping it separate otherwise. The optimization objective for this assignment is derived via applying Jensen's inequality to the exchangeable partition probability function (EPPF), which works out nicely for the DP case and applies to some other BNP models.Experiments compare the proposed algorithm in detail to several other methods for DP mixture models.Toy data experiments show that with the proposed component identification scheme, they see big speedup using many workers (up to 48) without loss in predictive performance.Furthermore, the authors report computation time and test log likelihood on 3 "real-world" datasets, showing their method achieves among the best testLL scores in much less time, often with 10x-100x speedup over other methods.Technical Comments on Method ============================My biggest concern here is that the paper lacks details about what exactly happens at each minibatch, and especially how exactly new clusters are created for that minibatch.Line 356 says the proposed approach is coupled with "standard variational inference" (Blei & Jordan 2006), but this is a well-known fixed truncation method and does not create new clusters at all.It seems the proposed component identification method applies regardless of the specific creation procedure used, but not specifying any details about creation is serious cause for concern. In my experience, the single most important feature of a BNP inference method is how successfully it creates high-quality new clusters that are missing from the current model.A related concern is that the presented merge scheme in Sec. 3.2 only applies to merging *new* clusters with other new clusters (created by different nodes). Many existing papers highlight ways to merge or remove existing clusters in approximate inference [eg Lin's ref 7, Bryant & Sudderth ref 11]. Without these moves, in my experience junky new clusters can overwhelm good ones. The proposed method does not handle this at all, and so likely underperforms.A final concern is that the current exposition does not compare and contrast the proposed DP mixture regularization bound (Eq 9) with existing mean-field variational objectives (e.g. from Blei and Jordan 2006 or Hughes & Sudderth 2013). Adding some of these bounds to Fig 2 would be interesting. It's unclear whether the proposed bound leads to a coherent overall bound on the marginal likelihood... I think so but it should be stated more obviously. It's also unclear even what objective function is used within a minibatch for SDA-DP: the proposed one? or the one from Blei and Jordan 2006?.Technical Comments on Experiments =================================The toy data results (Fig 3) are pretty convincing, though if you have 2D data showing plots of learned cluster parameters overlaid with data is nice. My big question here is that from Fig 3e it seems most runs do not recover all K=100 true clusters, instead usually about 90 or 95. This screams local optima to me, and I'm curious whether this might go away with a longer run (if so, how long would you need to run?), or whether the algorithm does really gets stuck if even run for a while and fails to create these missing clusters. Finally, figure 3b shows no visible speedup from 24 to 48 workers... is this because the problem is too small for effective parallelization? Or due to overhead from locking/unlocking?I'm less convinced by the experiments in Fig. 4. First, please plot traces of algorithm performance, rather than using a table, to communicate more information in the same amount of space. Second, lots of experimental details are missing: how was prior set? how were competitor methods initialized? what final number of clusters is reached for each dataset? was just one run used, or the best of multiple runs of each method? did you do many passes through the dataset, or just one pass? Third, some of the Airplane testLL numbers are so low (esp. for SVA and SC) that it's hard to know whether to blame local optima, buggy code, or what, but I'd expect both of these to perform better.Overall, the lack of detail makes these experiments hard to evaluate and hard to reproduce. It's also unclear how performance is sensitive to batch size, and whether competitor algorithms were allowed to vary the truncation... did the SVA runs use that paper's merge/prune moves? did the moVB runs include that paper's birth and merge moves? did SVI and Batch just use fixed truncation?Novelty ======= Other work provides the basic framework for mean-field inference [ref 24, etc.] and streaming/asynchronous Bayesian computation [ref 5, etc] used here. The real contribution is the new framing of the "component identification" problem, which I have not seen adequately addressed in prior work, and especially the concrete bound for the DP mixture case.Significance ============ The problem of general-purpose scalable inference for BNP models is definitely worth studying. The proposed approach could likely be adopted by others considering that it is based on the EPPF.Clarity =======I could follow the main arguments without too much trouble, but in many places (like around Eq 10) the authors relied too heavily on math notation and lacked motivating narrative. Around Eq. 10, I'd recommend adding text offering interpretations of the statistics s and t used for computing the regularization bound.Another possible improvement: In figure 1, remove the nodes for y_t illustrating data batches (which dont add much) and instead make the focus on the cluster parameters. Perhaps show two different cycles of a,b,c,d, so the reader better understands how the number of cluster parameters grows over time and what the "component identification" problem is ... maybe highlight the edges that need to be optimized in c from those that are known in advance.Line-by-line Comments ===================== (not necessary to respond to these in rebuttal)Lines 51-52: ref 13 does not seem to require a learning rateFig 2a: The "U"-shaped bend with large alpha is real but not very intuitive. Could use some commentary. Also, labeling some specific alpha values would help interpret the plot.Line 289: Saying "the bound increases as K -> 0" might suggest to non-experts that for any N, the DP's preferred number of clusters is very small, which is definitely not true.


Compressive spectral embedding: sidestepping the SVD
Spectral embedding based on the Singular Value Decomposition (SVD) is a widely used "preprocessing" step in many learning tasks, typically leading to dimensionality reduction by projecting onto a number of dominant singular vectors and rescaling the coordinate axes (by a predefined function of the singular value). However, the number of such vectors required to capture problem structure grows with problem size, and even partial SVD computation becomes a bottleneck. In this paper, we propose a low-complexity compressive spectral embedding algorithm, which employs random projections and finite order polynomial expansions to compute approximations to SVD-based embedding. For an m×n matrix with T non-zeros, its time complexity is O ((T + m + n) log(m + n)), and the embedding dimension is O(log(m + n)), both of which are independent of the number of singular vectors whose effect we wish to capture. To the best of our knowledge, this is the first work to circumvent this dependence on the number of singular vectors for general SVD-based embeddings. The key to sidestepping the SVD is the observation that, for downstream inference tasks such as clustering and classification, we are only interested in using the resulting embedding to evaluate pairwise similarity metrics derived from the ℓ 2 -norm, rather than capturing the effect of the underlying matrix on arbitrary vectors as a partial SVD tries to do. Our numerical results on network datasets demonstrate the efficacy of the proposed method, and motivate further exploration of its application to large-scale inference tasks.

The authors propose a novel algorithm for computing "compressive embeddings" of a matrix, which is most often done via approximate partial SVD. They provide detailed motivation and background for the problem, reviewing applications in graph processing, NLP, etc. The proposed algorithm "side-steps" the SVD by leveraging the JL lemma to compute approximate pair-wise distances and polynomial approximations to incorporate approximate functions of the singular values (e.g., thresholds). Some implementation details are highlighted.Overall, I think this a very good paper that presents simple, fast, effective, and (to my knowledge) novel algorithm for computing compressive embeddings, which is likely to be superior to Randomized SVD and pure random projections for many applications. The presentation is also relatively clear, if dense, despite a few minor errors and formatting issues (outlined below). One issue is that the authors do not flesh out a detailed story of how the performance changes relative to the embedding dimension and types of polynomials that might be approximated. This is difficult due to space-constraints, but especially for the former it would have been helpful to have a plot relating accuracy to dimension (not just a comparison btw two dimension sizes) that essentially provided an empirical view of theorem 1. Moreover, since the top 500 eigenvalues were computed the \delta could be reasonably approximated and so an empirical vs. theoretical expectation plot is possible.  Related to the above, more concrete recommendations from the authors w.r.t. how to choose embedding dimensions and eigenvalue thresholds would be helpful. And it is somewhat misleading that the authors set the polynomial approximations given knowledge of the first 500 eigenvalues. Discussion of what to do when such prior computations are infeasible would be helpful. Minor issues:-\cite{foo}\cite{bar} -> \cite{foo,bar} everywhere - line 73: point (a) is unclear - line 113: iteration\cite{foo} -> iteration \cite{foo} - line 237: "of good" -> "of a good" - line 340: "(=" -> "=" - capitalize Euclidean, Hermitian etc.


A hybrid sampler for Poisson-Kingman mixture models
This paper concerns the introduction of a new Markov Chain Monte Carlo scheme for posterior sampling in Bayesian nonparametric mixture models with priors that belong to the general Poisson-Kingman class. We present a novel compact way of representing the infinite dimensional component of the model such that while explicitly representing this infinite component it has less memory and storage requirements than previous MCMC schemes. We describe comparative simulation results demonstrating the efficacy of the proposed MCMC algorithm against existing marginal and conditional MCMC samplers.

Summary: The authors present a new MCMC method for a general class of bayesian nonparametric models, namely Poisson-Kingman random probability measures.They describe the class of models, the generative procedure and the insight that yields an mcmc routine for inference, as well as existing methods for the same/similar class of models, namely marginal and conditional samplers. They compare their sampler to standard marginal and conditional samplers, comparing effective sample size and running time.  - QualityThis paper develops an inference method for a very general class of models.It seems technically sound and provides an adequate summary of existing methods and how they relate to the method they present.  - ClarityThis paper isn't that clearly written, which may be due to the page limit.For instance, it is very difficult to decipher the notation in the generative procedure of section 2.1, which is arguably the most important section for appreciating the authors' contribution.  - Originality It is my understanding that a sampler for this general class of models is novel. - Significance If the idea were presented more fluidly, this paper could serve as a nice review and reference for this type of sampler.I think the idea has the potential to be impactful, but may fall short due to presentation.  Comments:- A figure or diagram of your generative process (the stick breaking construction) and how it is distinct from the standard DP stick breaking construction of Ishwaran and James would make section 2 (as well as your methodological contribution in general) easier to understand.


Evaluating the statistical significance of biclusters
Biclustering (also known as submatrix localization) is a problem of high practical relevance in exploratory analysis of high-dimensional data. We develop a framework for performing statistical inference on biclusters found by score-based algorithms. Since the bicluster was selected in a data dependent manner by a biclustering or localization algorithm, this is a form of selective inference. Our framework gives exact (non-asymptotic) confidence intervals and p-values for the significance of the selected biclusters.

The authors provide a framework for constructing p-values for the output of a variety of "selection" tasks, focusing on biclusters in particular. Their approach is to characterize the distribution of test statistics as multivariate normal random variables constrained to linear polytopes. This framework allows them to construct exact tests (assuming normality) for subsets of variables that have been selected in different ways.Quality: This is a quality paper. The contribution is non-trivial, and the material is presented in a way that goes beyond solving "just the bicluster significance problem." (Which is important in and of itself.) Clarity: Good but I have suggestions for improvement; see below. Originality: I believe that the specialization of theorem from reference [10] to biclustering and scan statistics and the rates that go with are novel. Significance: Addressing statistical significance issues in this domain is very important and widely applicable.-Specific comments:028: '"significant" compared to the rest of X. 063: "...statistically significant bicluster." 088: "We ask whether a submatrix selected by a biclustering algorithm captures the hidden association(s)." I think captures "some" of the hidden association(s) would be more accurate.The inference is that there is *something* in the computed bicluster that has some signal. In particular, supposing that rows correspond to genes, the specified alpha level does not control false discovery of genes within the computed bicluster; it controls false discovery of an "empty" bicluster. (Similarly, "power" does not indicate the probability of having found the entire bicluster, but only a part of it.) I think it is extremely important to make this clear. If on the other hand I am mistaken about this, I think other readers would be as well.039: Z ~ N(...) I have to ask the cliched question, what about non-normal noise models? How sensitive is the procedure to different kinds of non-normality? Depends on k maybe?071: "...for the amount of the signal in..." I suggest explicitly defining the selected parameter, maybe before (1.2), so that you can state this formally. I also think it would be worth re-iterating that the procedure will trap the *selected* parameter with probability (1-\alpha), where the procedure is to run the bi-clustering algorithm, select the parameter of interest, and then compute/invert the p-value.077: "In the supplementary materials, we discuss the problem in the more general setting where there are multiple emnbedded [SIC -- fix this] submatrices..." I recommend finding a way to lift this discussion into the main paper, even if you can't find room to give the full details. Not saying that needs to go here exactly, but somewhere in the main paper.129: Use either the e^TXe form or the trace form, but choose one and use it consistently.150: "The amount of signal in..." See comment above.162: "Let eta ... a linear function of y" The "linear function" comment confused me; just let it be a vector.168: I assume V0 is relevant in other contexts; if it's not relevant for this work perhaps omit?162--174: There is a lot of notation overloading here that is made worse by the fact that vectors are typeset indistinguishably from scalars. In particular, see if there is a way to remove overloading in arguments of F (2.7).378--397: These plots are difficult to distinguish, and I'm not sure what the intended message is. If it is to compare power of random initialization against intelligent initialization, then they should appear on the same plot. Then have different plots for (e.g.) 3 different n.