The success of deep learning research has catapulted deep models into production systems that our society is becoming increasingly dependent on, especially in the image and video domains. However, recent work has shown that these largely uninterpretable models exhibit glaring security vulnerabilities in the presence of an adversary. In this work, we develop a powerful untargeted adversarial attack for action recognition systems in both white-box and black-box settings. Action recognition models differ from image-classification models in that their inputs contain a temporal dimension, which we explicitly target in the attack. Drawing inspiration from image classifier attacks, we create new attacks which achieve state-of-the-art success rates on a two-stream classifier trained on the UCF-101 dataset. We find that our attacks can significantly degrade a model's performance with sparsely and imperceptibly perturbed examples. We also demonstrate the transferability of our attacks to black-box action recognition systems.

Summary:
This paper presents a framework for generating adversarial perturbations for videos. Specifically, the paper proceeds by using a standard image-based adversarial noise generation setup (such as the FGSM scheme), and applies it to the motion stream of a two-stream action recognition pipeline; this motion stream typically using optical flow images. As such flow generation is usually done offline and thus is not differentiable, the paper resorts to the recent FlowNet 2.0 scheme that uses an end-to-end learnable deep flow generation model. Three variants of the scheme are provided, (i) that perturbs all frames in a flow stack, (ii) that perturbs only a sparse set of frames as decided by the importance of a frame to action classification, and (iii) a variant of (ii) that recalculates the gradients for all frames if the ones selected in (ii) were not adversarial. Experiments are provided on UCF101 dataset and show promise. Analysis is presented on the transferability of  the learned noise to flow images generated via external means.
Strengths:
1) The different variants of the scheme and sparse selection of the frames to be perturbed are interesting.
2) The paper makes some interesting observations, namely that (i) only a single frame perturbation might be sufficient to make the video adversarial, and (ii) perturbations computed via FlowNet models are not transferable to those with flow computed via external software -- which is often the practice.
Weaknesses:
1) I think the main weakness of this paper is the lack of any surprise/significant novelty in the presented approach. The main idea follows the common trend in adversarial noise generation for image classification problems, except that the inputs are a stack of frames instead of  a single one; however, such a setting do not seem to bring along any non-trivial challenges. In the second contribution of this paper -- on the sparse selection of frames to attack, there is a lack of clarity in how one would do the iterative attacks at test time, given such sparse frame selection is done via computing the frame level saliency values via the classification loss, which depends on ground truth class labels, which are unavailable at test time.
2) There are previous works that have attempted video level adversarial perturbation generation, which the paper do not cite or contrast to; such as a few below. Further, the literature survey fails to provide any compelling motivation as to why video perturbation generation is any difficult than image based noise generation -- it does not appear so from the subsequent text that this problem deserves any special treatment in the considered context.
[a] Learning Discriminative Video Representations Using Adversarial Perturbations, Wang and Cherian, ECCV 2018
[b] Sparse Adversarial Perturbations for Videos, Wei et al., arxiv, 2018
3) It is unclear why the paper chose to consider flow produced by a FlowNet model as their inputs for the attack? Why not consider the flow images directly? Of course, the optical flow algorithm may not be differentiable, but that is perhaps besides the point; the focus should be in perturbing flow, in whatever way it is generated. To that end, given that flow (on static camera images) can be sparse, it would be interesting to see how would a perturbation be generated that needs to operate on local regions (where motion happens). In my opinion, using a FlowNet model for flow generation trivializes the proposed algorithm.
4) It could have been interesting if the paper also provided some qualitative results of the optical flow images generated by FlowNet after adding perturbations to the input frames. Are these flow images also quasi-impercitable?
Overall, I think the paper has some observations that may be slightly interesting; however, it lacks novelty and the analysis or presentation are unconvincing.


DIMENSIONALITY REDUCTION FOR REPRESENTING THE KNOWLEDGE OF PROBABILISTIC MODELS
Most deep learning models rely on expressive high-dimensional representations to achieve good performance on tasks such as classification. However, the high dimensionality of these representations makes them difficult to interpret and prone to over-fitting. We propose a simple, intuitive and scalable dimension reduction framework that takes into account the soft probabilistic interpretation of standard deep models for classification. When applying our framework to visualization, our representations more accurately reflect inter-class distances than standard visualization techniques such as t-SNE. We experimentally show that our framework improves generalization performance to unseen categories in zero-shot learning. We also provide a finite sample error upper bound guarantee for the method.

Authors propose a method of embedding training data examples into low-dimensional spaces such that mixture probabilities from a mixture model on these points are close to probability predictions from the original model in terms of KL divergence. Authors suggest two use-cases of such an approach: 1) data visualization, and 2) zero-shot learning. For the visualization use-case, authors compare against other dimensionality reduction methods with qualitative analysis on a synthetic problem, as well as evaluation metrics such as Neighborhood-Preservation Ratio and Clustering Distance Preservation Ratio. For zero-shot use-case, they take pre-trained models on two zero-shot tasks, and improve the accuracy by using probability outputs from pre-trained models as target.
Regarding the benefit of using the proposed method for visualization, the DRPR is making a strong assumption that representations of data points that belong to the same class form a uni-modal, Gaussian distribution (since authors don't experiment with distance functions other than L2). This inductive bias comes with a strong benefit when the assumption is true - as demonstrated in the toy dataset experiment - but when it is not true, the visualization would strongly distort the underlying structure of the model. And I don't believe this is a realistic assumption, because there has to be a reason that most deep-learning based classification models in the literature don't always use a model like (3) or Prototypical Networks instead of typical fully-connected + softmax layer, unless the data size is small and we need stronger inductive bias to improve the performance of the model.  That is, we usually don't think unimodality is the right assumption, even with learned representations. I suspect that the while DRPR might be good at visualizing relationships between class labels - especially which class can be easily confused with another - but would be worse at faithfully representing each data point, especially the ambiguity of class labels on individual ones. I would argue, however, that faithful representation of each data point is more important for scatter plots than relationship between classes, because the latter can be more effectively analyzed with other methods such as confusion matrices. As it is typical in most dimensionality reduction papers, I would encourage authors to consider more types of synthetic datasets which nonlinearity and multimodality are critical to be learned. I don't believe quantitative evaluation in Table 1 and 2 are very meaningful, because DRPR's objective function is much better aligned with these metrics than others.
Zero-shot experiments show a promising lift over the baseline pre-trained models. The kind of bias we should be careful about, however, is that when we distillate one model into another, the performance generally improves even when the same exact model is both the teacher and the student: (Furlanello et al, ICML 2018 https://arxiv.org/abs/1805.04770 ). Therefore, it would be interesting to compare against distillation with baseline models themselves.
Pros:
* Extensive theoretical and empirical analysis
* Simple idea that generalizes to multiple use-cases, which implies robustness of the approach as a methodology
Cons:
* Unimodal assumption is likely not realistic, which would result in misleading visualization of data
* Visualization analysis focuses on how class-relationships are preserved rather than faithful representation of each data point, which is a wrong target
* Synthetic experiment is conducted on a single, too simplistic one; more examples are needed to understand the capabilities of the model in more detail
* The bias of knowledge distillation is not controlled


Generative Adversarial Networks (GAN) can achieve promising performance on learning complex data distributions on different types of data. In this paper, we first show that a straightforward extension of an existing GAN algorithm is not applicable to point clouds, because the constraint required for discriminators is undefined for set data. We propose a two fold modification to a GAN algorithm to be able to generate point clouds (PC-GAN). First, we combine ideas from hierarchical Bayesian modeling and implicit generative models by learning a hierarchical and interpretable sampling process. A key component of our method is that we train a posterior inference network for the hidden variables. Second, PC-GAN defines a generic framework that can incorporate many existing GAN algorithms. We further propose a sandwiching objective, which results in a tighter Wasserstein distance estimate than the commonly used dual form in WGAN. We validate our claims on the ModelNet40 benchmark dataset and observe that PC-GAN trained by the sandwiching objective achieves better results on test data than existing methods. We also conduct studies on several tasks, including generalization on unseen point clouds, latent space interpolation, classification, and image to point clouds transformation, to demonstrate the versatility of the proposed PC-GAN algorithm.

Summary:
This paper proposes a generative point cloud model based on adversarial learning and definitti’s representation theorem of exchangeable variables.
The main focus in experiments and the exposition is on 3D point clouds representing object shapes (seems the surface, but could also be the interior of objects, please clarify).
The main idea is to represent a point cloud using a global latent variable that captures the overall shape, and a collection of local latent variables that code for the position of a point on the shape.
The model consists of thee components:
(i) an “encoder” that takes a point cloud as input and maps it to a (point estimate of) the global latent variable of the shape represented by the input cloud, a point-net architecture is used here
(ii) a “decoder” that takes the estimated global latent variable, and a local latent variable, and maps it to an “output” point in the cloud to be produced by the model.
(iii) a “discriminator” network that aims to distinguish points from a *given* shape, and the points produced by pipe-lining the encoder and decoder. Critically different from conventional GANs, the discriminator is optimized *per shape*, ie each point cloud is considered as a *distribution* over R^3 specific to that shape.
(iv) a “shape prior” that, once the encoder-decoder model from above is trained, is used to model the distribution over the global latent variables. This model is trained, presumably, in a conventional GAN style using the global latent variable representations inferred across the different training point clouds.
As compared to prior work by Achiloptas et al (2017), the proposed approach has the advantage to allow for sampling an arbitrary number of points from the target shape, rather than a fixed pre-defined number.
In addition, the authors propose to minimize a weighted average of a lower bound and upper bound on the Wasserstein distance between the distributions of points corresponding to given shapes.
This approach translates to improved quantitative evaluation measures,
Experiments are conducted on a simple toy data set, as  a proof of concept, and on data from ModelNet10 and ModelNet40.
Two performance metrics are introduced to assess the auto-encoding ability of the model: to what extent does the encoder-decoder pipeline result in point clouds similar to the shape from which the input point-cloud is generated.
Overall I find the idea of the paper interesting and worth publishing, but the exposition of the paper is less than ideal and needs further work.
The experimental validation of the proposed approach can also be further improved, see more specific comments below.
Specific comments:
- The counter example at the bottom of page 2 is limited, in the sense that the oracle assumption seems highly non-realistic, casting doubt on the relevance of the argument.
- The notation in section 3 (before 3.1) is rather sloppy.
For example,
- please define P and G, the elements of the divergence D(P||G) that appears in the first paragraph of section 3.
- it is not defined in which space theta lives, it is not clear what the authors intend with the notation G_theta(u) \sim p(theta).
- what prior distributions p(z) and p(u) are used? What is the choice based on?
- abbreviation IPM is referred several times in the paper, but remains undefined in the paper until end of page 4, please define earlier.
- The model G_theta does not appear in the training objective function (4), how is this module trained precisely?
- Lack of clarity in the following passage: “In our setting, each point xi in the point cloud can be considered to correspond to single images when we train GANs over images”
- The notion of divergence D(P|G) is not made concrete in section 3 and 3.1, which makes the notation of rather little use.
- The following paper merits a discussion in the related work section:
“TOWARDS A NEURAL STATISTICIAN”, ICLR’17, https://openreview.net/pdf?id=HJDBUF5le
- The manuscript contains many typos. For example
 “vedio” op page 4, “circile” on page 5, “condct” on page 8, etc.
Please proof read your paper and fix these.
The refenence to  Bengio 2018 is incomplete: what do you refer to precisely?
- There seems to be no mention of the dimension of the “local” latent variables z_i.
Please comment on the choice, and its impact on the behavior of the model.
- The quantitative evaluation in table 1 is interesting and useful.
It is limited, however, in the sense that it (only) measures auto-encoding capabilities: to what extent can the shape be reproduced given a sample point cloud from the given shape.
Quantitative evaluation of generative modeling performance is unfortunately missing from this paper, as it is in much of the GAN literature.
Could you please comment on how this can/will be fixed?
- The toy data set experiments could be dropped  to make room for experiments suggested below.
- An experimental study of the effect of the mixing parameter “s” would be useful to include.
For example, by taking s on a grid from 0 to 1, one could plot the coverage and distance-to-face measures.
- Experimental evaluation of auto-encoding using a variable number of input points is interesting to add: ie how do the two evaluation measures evolve as a function of the number of points in the input point cloud?
- Similar, it is interesting to evaluate how auto encoding performs when non-uniform decimation of the input cloud is performed, eg what happens if we “chop off” part of the input point cloud (eg the legs of the chair), does the model recover and add the removed parts? This is potentially useful to practitioners which have to deal with incomplete point clouds acquired by range scanners.
- Analysis of shapes with different genus and dimensions would be interesting.
Does the model manage to capture that some shapes have holes, or consists of a closed 2D surface (ball) vs an open surface (disk),  despite a simple prior on the local latent variables z?


We propose a novel framework for combining datasets via alignment of their associated intrinsic dimensions. Our approach assumes that the two datasets are sampled from a common latent space, i.e., they measure equivalent systems. Thus, we expect there to exist a natural (albeit unknown) alignment of the data manifolds associated with the intrinsic geometry of these datasets, which are perturbed by measurement artifacts in the sampling process. Importantly, we do not assume any individual correspondence (partial or complete) between data points. Instead, we rely on our assumption that a subset of data features have correspondence across datasets. We leverage this assumption to estimate relations between intrinsic manifold dimensions, which are given by diffusion map coordinates over each of the datasets. We compute a correlation matrix between diffusion coordinates of the datasets by considering graph (or manifold) Fourier coefficients of corresponding data features. We then orthogonalize this correlation matrix to form an isometric transformation between the diffusion maps of the datasets. Finally, we apply this transformation to the diffusion coordinates and construct a unified diffusion geometry of the datasets together. We show that this approach successfully corrects misalignment artifacts, and allows for integrated data.

The authors pointed out that the measurements in biology and natural science suffer from batch effects such as the variations between batches of data measured at different times or by different sensors. In order to analyze different batches of data, an alignment or a calibration is frequently needed. The authors propose to use that though there is variation among different batches, these batches all share an underlying intrinsic manifold structure, which may admit a set of alignable coordinates.
Technically, the authors propose to choose the diffusion kernel method, which is one of the spectral methods, to extract the harmonic like eigenfunctions defined on the manifold, for each of the batches of data. Using these harmonic-like coordinates, the authors assume there exists an isometric rotation in the between each pair of batches such that their coordinates can be aligned under this orthogonal rotation.
Comments:
Overall I think the problems pointed out do exist and this is an interesting proposal to use the manifold structure to align the data. But there are some weak points in this proposal:
1. It's well known that spectral methods are frequently sensitive to perturbations of the datasets. At the beginning of section 2.2 the authors propose to use a normalization to construct the kernel, however, I don't quite understand how this would solve the instability to perturbations.
2. In my opinion, the equation (1) is the most interesting construction in this paper. This motivation for this tensors construction is not strong enough and I would suggest put more detail into this construction. My understanding is the window functions g introduced here serve for an invariance purpose such that when the frequency slightly shift (or rotate), the correlation computed should be stable. But the tradeoff of choosing a proper window should be discussed carefully, potentially with different dataset since different dataset may have a different sensitivity to perturbations across different batches.
3. In section 3.1, the first motivating example is quite confusing. The authors demonstrated the alignment of two rotated MNIST digits, 3. For each digit, the underlying manifold is S1. S1 is diffeomorphic to its rotation. So I'm not so sure what's the underlying manifold geometry used to align them.  My understanding is that this alignment doesn't come from the S1 manifold but comes from some additional structure in the image signal. Fig1(b) is also a little confusing that I couldn't figure out what's drawn there.
Overall, to use the underlying manifold structure to align data batches is an interesting and straightforward proposal, but I hope the authors can address these question carefully and make the argument stronger.


GENERATING MULTI-AGENT TRAJECTORIES USING PROGRAMMATIC WEAK SUPERVISION
We study the problem of training sequential generative models for capturing coordinated multi-agent trajectory behavior, such as offensive basketball gameplay. When modeling such settings, it is often beneficial to design hierarchical models that can capture long-term coordination using intermediate variables. Furthermore, these intermediate variables should capture interesting high-level behavioral semantics in an interpretable and manipulatable way. We present a hierarchical framework that can effectively learn such sequential generative models. Our approach is inspired by recent work on leveraging programmatically produced weak labels, which we extend to the spatiotemporal regime. In addition to synthetic settings, we show how to instantiate our framework to effectively model complex interactions between basketball players and generate realistic multi-agent trajectories of basketball gameplay over long time periods. We validate our approach using both quantitative and qualitative evaluations, including a user study comparison conducted with professional sports analysts.

This paper proposes training multiple generative models that share a common latent variable, which is learned in a weakly supervised fashion, to achieve high level coordination between multiple agents. Each agent has a separate VRNN model which is conditioned on the agent’s own trajectory history as well as the shared latent variable. The model is trained to maximize the ELBO objective and log-likelihood over macro-intent labels. Experimental results are conducted over a basketball gameplay dataset (to model the trajectories of the offensive team members) and a synthetic dataset. The results show that the proposed model is on-par with the baseline models in terms of ELBO while showing that it can model multi-modality better and is preferred more by humans.
In general, the paper is well written and the overall framework captures the essence of the problem that the authors are trying to solve.
Furthermore, incorporating an auxiliary latent variable to model the coordination between multiple agents is interesting.
I have several comments related to the strength of the baselines and contribution of individual components in the proposed model.
Major Comments
- It seems that VRNN-single and VRNN-indep are two models on the far two ends of a spectrum. To understand the contribution of the shared macro-intent, how would an intermediate baseline model where a set of parameters are shared between agents and each agent also has an independent set of parameters perform? This could be accomplished by sharing the parameters of the first layer of GRU networks and learning the second layer parameters independently.
- How is the threshold for macro-intent generation selected? How does this parameter affect the overall performance? Since the smoothness of the segments between two macro-intents depend on this parameter, I am wondering its effect on the learned posterior distribution.
- Rather than using the prediction of the macro-intent RNN as a single global vector (\hat{g}_t), could using separate vectors for each agent (corresponding blocks of \hat{g}_t) as inputs to VRNN give the same results? Since the macro-intent RNN is already aware of all the macro-intents, it would be interesting to see if individual macro-intents are sufficient for VRNN to generate corresponding trajectories.
Minor Comments
- Do results in Table (1) come from sampling or using mode of the distributions? How peaked are the learned posterior distributions?
- What is the performance of the macro-intent RNN model?
- In Eq (2), “<=T” should be “<=t” (as in Eq (11) in Chung 2015).
- In Page 6, bullet point 4: it should be “except we maximize the mutual information…”


Deep learning has shown that learned functions can dramatically outperform handdesigned functions on perceptual tasks. Analogously, this suggests that learned optimizers may similarly outperform current hand-designed optimizers, especially for specific problems. However, learned optimizers are notoriously difficult to train and have yet to demonstrate wall-clock speedups over hand-designed optimizers, and thus are rarely used in practice. Typically, learned optimizers are trained by truncated backpropagation through an unrolled optimization process. The resulting gradients are either strongly biased (for short truncations) or have exploding norm (for long truncations). In this work we propose a training scheme which overcomes both of these difficulties, by dynamically weighting two unbiased gradient estimators for a variational loss on optimizer performance. This allows us to train neural networks to perform optimization faster than well tuned first-order methods. Moreover, by training the optimizer against validation loss (as opposed to training loss), we are able to learn optimizers that train networks to better generalization than first order methods. We demonstrate these results on problems where our learned optimizer trains convolutional networks in a fifth of the wall-clock time compared to tuned first-order methods, and with an improvement in validation loss.

Summary:
The paper presents a method for "learning an optimizer"(also in the literature Learning to Learn and a form of Meta-Learning) by using a Variational Optimization for the "outer" optimizer loss. The mean idea of the paper is to combine both the reparametrized gradient and the score-function estimator for the Variational Objective and weight them using a product of Gaussians formula for the mean. The method is simple and clearly presented. The paper also presents issues with the standard "learning to learn" optimizers, one being the short-horizon bias and as credited by the authors has been observed before in the literature, and the second one is what is termed the "exponential explosion of gradients" which I think lacks enough justification as currently presented (see below for details). The ideas are clearly stated, although the work is not groundbreaking, but more on combining several ideas into a single one.
Experiments:
The authors evaluate their method on a single task which consists of optimizing a 3-layer convolutional neural network on downsampled images from ImageNet. A key idea, not new to this work, is to optimize the meta-optimizer with respect to the validation dataset rather than the training, which seems to be crucial for any meaningful training to happen. Although the experiments do show so promising results, they seem to be somewhat limited (see below for details). There is also a small ablation study on how do different features presented to the optimizer affect its performance. Given the still small-scale experiments, I'm not sure this is a significant result for the community.
Conclusion:
As a whole, I think the idea in the paper is a good one and worth investigating further. However, the objections I have on section 2.3 and the experiments seem to indicate that there needs to be more work into this paper to make it ready for publication.
On section 2.3 and the explosion of gradients:
There is a mistake in the equation on page 4 regarding the "gradient with respect to the learning rate". Although the derivation in Appendix A is correct, the inner product in the equation starts wrongly from j=0, where it should in fact start at j = i + 1. To be more clear the actual enrolled equation for dw^T/dt for 3 steps back is:
dw^T/dt = (I - tH^{T-1})(I - tH^{T-2})(I - tH^{T-3}) dw^{T-3} - (I - tH^{T-1})(I - tH^{T-2}) g^{T-3} - (I - tH^{T-1}) g^{T-2} - g^{T-1}
Hence the product must start at j = i + 1.
It is correct that in this setting the equation is a polynomial of degree T of the Hessian, however, there are several important factors that the authors have not discussed. Namely, if the learning rate is chosen accordingly such that the spectral radius of the Hessian is less than 1/t then rather than the gradient exploding the higher order term will vanish. However, even if they do vanish for large T since the Hessian plays with smaller and smaller power to more recent gradients (after correcting the mistake in the equation) than the actual T-step gradient will never vanish (in fact even if tH = I then dw^T/dt = g^{T-1}). Hence the claims of exploding gradients made in this section coupled with the very limited theoretical analysis seem to unconvincing that this is nessacarily an issue and under what circumstances they are.
The toy example with l(w) = (w - 4)(w - 3) w^2 is indeed interesting for visualizing a case where the gradient explosion does happen. However, surprisingly here the authors rather than optimizing the learning rate, which they analyzed in the previous part of the section, they are now optimizing the momentum. The observation that at high momentum the training is unstable are not really surprising as there are fundamental reasons why too high momentum leads to instabilities and these have been analyzed in the literature. Additionally, it is not mentioned what learning rate is used, which can also play a major role in the effects observed.
As a whole, although the example in this section is interesting, the claims made by the authors and some of the conclusions seem to lack any significant justifications, in addition to the fact that usually large over-parameterized models behave differently than small models.
Experiments:
I have a few key issues with the experimental setup, which I think need to be addressed:
1. The CNN being optimized is quite small - only 3 layers. This allows the authors to train everything on a CPU. The key issue here, as well with previous work on Learning to Learn, is that it is not clear how scalable is this method to very Deep Networks.
2. Figure 1 - The setup is to optimize the problem for 10000 iterations, however, I think it is pretty clear even to the naked eye that the standard first-order optimizers (Adam/RMS/Mom) have not fully converged on this problem. Hence I think its slightly unfair to compare their "final performance" after this fixed period. Additionally using the curriculum the "meta"-optimizer is trained explicitly for 10000 iterations. Hence, it is also unclear if it retains its stability after letting it run for longer. From the text it is also unclear whether the authors have optimized the parameters of the first-order methods with respect to their training or validation performance - I hope this is the latter as that is the only way to fairly compare the two approaches.
3. Figure 6 - the results here seem to indicate that the learned optimizer transfers reasonably well, achieving similar performance to first-order methods (slightly faster validation reduction). Given however that these are plots for only 10000 iterations it is still unclear if this is scalable to larger problems.


Deep neural networks provide state-of-the-art performance for image denoising, where the goal is to recover a near noise-free image from a noisy image. The underlying principle is that neural networks trained on large datasets have empirically been shown to be able to generate natural images well from a lowdimensional latent representation of the image. Given such a generator network, or prior, a noisy image can be denoised by finding the closest image in the range of the prior. However, there is little theory to justify this success, let alone to predict the denoising performance as a function of the networks parameters. In this paper we consider the problem of denoising an image from additive Gaussian noise, assuming the image is well described by a deep neural network with ReLu activations functions, mapping a k-dimensional latent space to an n-dimensional image. We state and analyze a simple gradient-descent-like iterative algorithm that minimizes a non-convex loss function, and provably removes a fraction of p1´Opk{nqq of the noise energy. We also demonstrate in numerical experiments that this denoising performance is, indeed, achieved by generative priors learned from data.

The paper analyzes the recovery accuracy of a "tweaked" gradient descent algorithm for imaging denoising and compressive sensing under deep generative priors. In particular, when assuming Gaussian randomness of the network weights and extremely stringent conditions of network sizes, they demonstrate a specific denoising rate of O(k/n), with k and n being the input and output dimension of the generative network. This is seemingly optimal in terms of the dependence on the latent code dimensionality and the signal dimensionality and is the first result of this kind.
Two papers are closely related, but are not sufficiently discussed in the introduction. [Bora et al., 2017] does not require Gaussian randomness of the network weights, but achieves only O(1) error bound assuming the empirical risk minimization problem can be solved to optimality.  [Hand & Voroninski, 2018] showed that under same assumptions as in this paper, the nonconvex empirical risk minimization problem exhibits a nice geometric landscape - no spurious stationary points. This implies that virtually anything reasonable would converge to global optimum. Combing both facts, it is not surprising to arrive at the results in this paper.
While the paper makes some novel theoretical contributions, two concerns stand out. First, there is a lack of intuition or justification of the tweak in gradient descent - flipping the sign of the iterate at times.  The author argued that around approximately -x*, the loss function is larger than around the optimum x* . So simple gradient descent is likely to get stuck in this region, so the negation check is needed. I am not so convinced by the argument. There could be other critical points that are not necessarily in the negative regime of true optimal, right?  So why would this be sufficient or necessary for global convergence? Second, even ignoring the unrealistic Gaussian assumption on the network weights, the theorem requires very narrow regimes for the expansivity condition and the noise variance bound. It's hard to verify whether these conditions can be satisfied at all.
The experiment on denoising with learned prior from MNIST data is interesting, as it suggests that the theoretical assumptions are not necessary in practice to observe the optimal recovery rate. It would be more convincing if more experiments are provided, especially for the compressive sensing application.


CONDITIONAL NETWORK EMBEDDINGS
Network Embeddings (NEs) map the nodes of a given network into d-dimensional Euclidean space R d . Ideally, this mapping is such that 'similar' nodes are mapped onto nearby points, such that the NE can be used for purposes such as link prediction (if 'similar' means being 'more likely to be connected' or 'having similar neighborhoods') or classification (if 'similar' means 'being more likely to have the same label'). In recent years various methods for NE have been introduced, all following a similar strategy: defining a notion of similarity between nodes, a distance measure in the embedding space, and a loss function that penalizes large distances for similar nodes and small distances for dissimilar nodes. A difficulty faced by existing methods is that certain networks are fundamentally hard to embed due to their structural properties: (approximate) multipartiteness, certain degree distributions, assortativity, etc. To overcome this, we introduce a conceptual innovation to the NE literature and propose to create Conditional Network Embeddings (CNEs); embeddings that maximally add information with respect to given structural properties (e.g. node degrees, block densities, etc.). We use a simple Bayesian approach to achieve this, and propose a block stochastic gradient descent algorithm for fitting it efficiently. We demonstrate that CNEs are superior for link prediction and multi-label classification when compared to state-ofthe-art methods, and this without adding significant mathematical or computational complexity. Finally, we illustrate the potential of CNE for network visualization.

The authors propose a generative model of networks by learning embeddings and pairing the embeddings with a prior distribution over networks. The idea is that the prior distribution may explain structure that the embeddings would not have to capture.
The motivation for doing this is that this structure is typically hard to model for network embeddings.
The authors propose a clean -if improper- prior on networks and proceed to perform maximum likelihood inference on it.
The experiments show that the approach works fine for link porediction and can be used for visualization.
Two points:
a) Why not try to do this with Variational inference? It should conceptually still work and be fast and potentially more robust.
b) The prior seems to be picked according to properties of the observed data and expressed in a product of constraints. This seems clunky, I would have been more impressed with a prior structure that ties in closer with the embeddings and requires less hand-engineering.
A key point of interest is the following: very exciting recent work (GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models by You et al ICML2018) has proposed neural generative models of networks with a high degree of fidelity and much less hand-picked features.  The work here tries to not learn a lot of these structures but impose them. Do the authors think that ultimately learning priors with models like GraphRNN might be more promising for certain applications?
The drawback in this model here is that ultimately networks are embedded, but not really generated during test time. A more predictive generative model that makes less hard assumptions on graph data would be interesting.
Update After rebuttal:
Given the authors' rebuttal to all reviews, I am upgrading my score to a 6. I still feel that more learning (as inGraphRNN) to build a fuller generative model of the graph would be interesting, but the authors make a strong case for the usefulness and practicality of their approach.


Making decisions requires information relevant to the task at hand. Many real-life decision making situations allow acquiring further relevant information at a specific cost. For example, in assessing the health status of a patient we may decide to take additional measurements such as diagnostic tests or imaging scans before making a final assessment. More information that is relevant allows for better decisions but it may be costly to acquire all of this information. How can we trade off the desire to make good decisions with the option to acquire further information at a cost? To this end, we propose a principled framework, named EDDI (Efficient Dynamic Discovery of high-value Information), based on the theory of Bayesian experimental design. In EDDI we propose a novel partial variational autoencoder (Partial VAE), to efficiently handle missing data over varying subsets of known information. EDDI combines this Partial VAE with an acquisition function that maximizes expected information gain on a set of target variables. EDDI is efficient and demonstrates that dynamic discovery of high-value information is possible; we show cost reduction at the same decision quality and improved decision quality at the same cost in benchmarks and in two health-care applications. We believe there is great potential for realizing these gains in real-world decision support systems.

The authors present an information discovery approach based on (partial) variational autoencoders and an information theoretic acquisition function that seeks to maximize the expected information gain over a set of unobserved variables. Results are presented on image inpainting, UCI datasets and health data, namely ICU and NHANES.
It is not clear why multiple recurrent steps improve perfromance. This is not conceptually justified and empirically (see Figure 8), it is also unclear whether PNP5 significantly outperforms PNP1. Further, results seem to support that PNP is always better than PN, so why introduce the methodology around PN or even present it at all. Note that the authors do not offer an explanation about the perfromance differences between PN and PNP.
In the inpainting regions section, the authors write about well-calibrated uncertainties without any context. What do they mean by calibration, well-calibrated and how can they support their claim about it?
In Figure 3 it is not clear that PNP+Ours outperforms PNP+SING. For Boston hosing seems to be marginally better but the error bars (which I assume are standard deviations, not stated) make difficult to ascertain whether the differences are significant. Although I understand the value of having "personalized" decisions, one wonders whether this personalization comes with any generalizable measurable gains given the results.
The results in Table 2 need to be clarified and further explained. 1) what are the error bars, considering multiple runs and datasets? 2) How can EDDI be so much better than SING when individual AUICs in Tables 6-11, the only significant difference (accounting for error bars) is on Boston data? 3) according to Tables 6-11, PNP is only the best in 1 of 5 datasets, so how come is the overall beast by a large margin? This being said, the results in Table 2 are at best misleading.
In Table 4, how can PNP-EDDI be so much better than PNP-SING, when in Figure 6 error bars overlap almost everywhere?
I enjoyed reading the paper, the motivation is clear and the problem is important. The approach is modestly novel compared to existing approaches and in general well explained despite the fact that the need for multiple recurrent steps is not well justified and the differences between PN and PNP, advantages/disadvantages and when to use each are not described or explored in the experiments.


Increasing the mini-batch size for stochastic gradient descent offers significant opportunities to reduce wall-clock training time, but there are a variety of theoretical and systems challenges that impede the widespread success of this technique (Das et al., 2016;Keskar et al., 2016). We investigate these issues, with an emphasis on time to convergence and total computational cost, through an extensive empirical analysis of network training across several architectures and problem domains, including image classification, image segmentation, and language modeling. Although it is common practice to increase the batch size in order to fully exploit available computational resources, we find a substantially more nuanced picture. Our main finding is that across a wide range of network architectures and problem domains, increasing the batch size beyond a certain point yields no decrease in wall-clock time to convergence for either train or test loss. This batch size is usually substantially below the capacity of current systems. We show that popular training strategies for large batch size optimization begin to fail before we can populate all available compute resources, and we show that the point at which these methods break down depends more on attributes like model architecture and data complexity than it does directly on the size of the dataset.

The work presented relates to the impact batch-size on the learning performances of common neural network architectures.
Pro: having comprehensive study of the limit of gradient-based methods is very useful in practice. This work can help practitioner to limit the number of machines used for optimization.
Cons: very little can be deduced from these experiments:
- "Increasing the batch size beyond a certain point yields no improvement in wall-clock time to convergence, even for a system with perfect parallelism." was a know fact (they cite Ma et al (2017) who even proved it theoretically.
- "Increasing the batch size leads to a significant increase in generalization error, which cannot be mitigated by existing techniques.". It is not clear that all the regularization techniques have been tried by the authors, the increase of generalization error is very small, and there is no explanation or insight given by the authors to explain this phenomenon, making this finding of limited interest.
- "Dataset size is not the only factor determining the computational efficiency of large batch training." is something obvious to say, as there are plenty of factors that determine the computational efficiency (network connection, map-reduce implementation, etc.)
Even the suggestions for future work of the authors in the conclusion does not help much: they suggest to look at "alternative forms of parallelism", without citing or giving any clue of what could be such alternative forms.
Also, there is no discussion around lock-free
The authors refer to Ma et al. (2017) for a theoretical analysis of the effect of the batch size, but they skip all the past and very relevant literature on the topic of the effect of the batch size on the convergence. For example, it is recommended to increase the size of the batch size as the iterations increase.
Finally, there is no discussion on the lock-free gradient descent, that is often suggested as an alternative to batching.
In conclusion, I'm not convinced there is enough material to accept this paper at the next ICLR conference.


Deep models are state-of-the-art for many computer vision tasks including image classification and object detection. However, it has been shown that deep models are vulnerable to adversarial examples. We highlight how one-hot encoding directly contributes to this vulnerability and propose breaking away from this widely-used, but highly-vulnerable mapping. We demonstrate that by leveraging a different output encoding, multi-way encoding, we can make models more robust. Our approach makes it more difficult for adversaries to find useful gradients for generating adversarial attacks. We present state-of-the-art robustness results for black-box, white-box attacks, and achieve higher clean accuracy on four benchmark datasets: MNIST, CIFAR-10, CIFAR-100, and SVHN when combined with adversarial training. The strength of our approach is also presented in the form of an attack for model watermarking, raising challenges in detecting stolen models.

This paper argues that the vulnerability of classifiers to (black-box) adversarial attacks stems from the use of a final cross-entropy layer trained on one-hot labels. The authors propose replacing this layer by encoding each label as a high-dimensional vector and then training the classifier to minimize the L2 distance of the classifier output from the encoding of the correct class. While the approach is interesting and the paper well-written, both the motivation and the experimental evaluation is insufficient. Hence I consider it below the ICLR bar.
I find the approach weakly motivated. The argument in Figure 1 is very hand-wavy with no clear experimental or theoretical support. The authors argue that cross-entropy with one hot labels causes gradient correlation in the last layer and this propagates all the way through the network (bottom of page 3) but there are no experiments supporting this conjecture.
Moreover, the approach is not fundamentally different from standard networks with cross-entropy training. One can consider adding an extra layer (with number of neurons equal to the encoding vector dimensions) and keeping the weights of these neurons fixed (the output weights are essentially the encoding dictionary). Then training with cross-entropy is increasing the inner product with these vectors. This is qualitatively very similar to the proposed approach of this paper. Is there a benefit from explicitly considering the encoding vectors?
Moreover, why is the length of the encoding vectors important from a conceptual point of view? As far as I can tell, this is simply encouraging the output of the network to be large in norm. This could be leading to gradient masking, similar to the phenomena observed for defensive distillation.
I find the proposed approach to watermark evasion interesting. However I consider it orthogonal to the rest of the results so it is hard to consider it as a contribution to the main point of the paper.
Figure 3 is missing white-box evaluation of RO classifiers. Is this on purpose? It is important to understand if the claimed improvement in robustness actually stems from RO rather than mostly from combining it with adversarial training.
The authors report an increase in white-box adversarial robustness. However, I don't believe that the evaluation of their method is thorough. There are plenty of examples by now where PGD has not been sufficient to evaluate the ground-truth robustness of a model. This can distort the relative robustness of different approaches. Given that the increase from baselines in white-box robustness is relatively small (<10% for most datasets) a much more thorough evaluation is required to conclusively demonstrate the benefit of this method. For instance, applying the SPSA attack from Uesato et al. (2018, https://arxiv.org/abs/1802.05666) or a variant of the CW (https://arxiv.org/abs/1608.04644) attack adapted to the particular method used.
As an additional point of concern emphasizing this issue, the authors present the results of Kannan et al. (2018) as state-of-the-art. So far, there is no conclusive evidence about ALP improving the robustness of neural networks beyond adversarial training. The original paper was found to be not as robust as claimed and retracted from NIPS. A similar paper reporting ALP to improve robustness in smaller datasets (CIFAR10) was submitted to ICLR (https://openreview.net/forum?id=Bylj6oC5K7) but was withdrawn after the authors performed additional experiments. The fact that the authors find the approach of Kannan et al. (2018) to offer an increase over the robustness of Madry et al. (2017) thus raises concerns about the reliability of the evaluation.
Other comments:
-- When the authors perform PGD, what is exactly the loss it is applied on? Is it clear that this is the optimal loss to use when attacking RO classifiers?


During recent years there has been an increased interest in stochastic adaptations of limited memory quasi-Newton methods, which compared to pure gradientbased routines can improve the convergence by incorporating second order information. In this work we propose a direct least-squares approach conceptually similar to the limited memory quasi-Newton methods, but that computes the search direction in a slightly different way. This is achieved in a fast and numerically robust manner by maintaining a Cholesky factor of low dimension. This is combined with a stochastic line search relying upon fulfilment of the Wolfe condition in a backtracking manner, where the step length is adaptively modified with respect to the optimisation progress. We support our new algorithm by providing several theoretical results guaranteeing its performance. The performance is demonstrated on real-world benchmark problems which shows improved results in comparison with already established methods.

This paper presents a new quasi-Newton method for stochastic optimization that solves a regularized least-squares problem to approximate curvature information that relaxes both the symmetry and secant conditions typically ensured in quasi-Newton methods. In addition to this, the authors propose a stochastic Armijo backtracking line search to determine the steplength that utilizes an initial steplength of 1 but switches to a diminishing steplength in later iterations. In order to make this approach computationally tractable, the authors propose updating and maintaining a Cholesky decomposition of a crucial matrix in the Hessian approximation. Although it is a good attempt at developing a new method, the paper ultimately lacks a convincing explanation (both theoretical and empirical) supporting their ideas, as I will critique below.
1. Stochastic Line Search
Determining a steplength in the stochastic setting is a difficult problem, and I appreciate the authors’ attempt to attack this problem by looking at stochastic line searches. However, the paper lacks much detail and rigorous reasoning in the description and proof for the stochastic line search.
First, the theory gives conditions that the Armijo condition holds in expectation. Proving anything about stochastic line searches is particularly difficult, so I’m on board with proving a result in expectation and doing something different in practice. However, much of the detail on how this is implemented in practice is lacking.
How are the samples chosen for the line search? If we go along with the proposed theory, then when the function is reevaluated in the line search, a new sample is used. If this is the case, can one guarantee that the practical Armijo condition will hold? How often does the line search fail? How does the choice of the samples affect the cost of evaluating the line search?
The theory also suggests that the particular choice of c is dependent on each iteration, particularly the inner product between the true search direction and the true gradient at iteration k. Does this allow for a fixed c to be used? How is c chosen? Is it fixed or adaptive? What happens as the true gradient approaches 0?
The algorithm also places a limit on the number of backtracks permitted that decreases as the iteration count increases. What does the algorithm do when the line search fails? Does one simply take the step although the Armijo condition does not hold?
In deterministic optimization, BFGS typically needs a smaller steplength in the beginning as the algorithm learns the scale of the problem, then eventually accepts the unit steplength to obtain fast local convergence. The line search proposed here uses an initial steplength of $\min(1, \xi/k)$ so that in early iterations, a steplength of 1 is used and in later iterations the algorithm uses a $\xi/k$ steplength. When this is combined with the diminishing maximum number of backtracking iterations, this will eventually yield an algorithm with a steplength of $\xi/k$. Why is this preferred? Are the other algorithms in the numerical experiments tuned similarly?
The theory also asks for a descent direction to be ensured in expectation. However, it is not the case that $E[\hat{p}_k^T \hat{g}_k] = E[\hat{p}_k]^T g_k$, so it is not correct to claim that a descent direction is ensured in expectation. Rather, the condition is requiring the angle between the negative stochastic gradient direction and search direction to be acute in expectation.
All the proofs also depend on a linear Taylor approximation that is not well-explained, and I’m wary of proofs that utilize approximations in this way. Indeed, the precise statement is that $\hat{f}_{z’} (x + \alpha \hat{p}_z) = \hat{f}_{z’} + \alpha \hat{p}_z’ \hat{g}_z(x + \bar{\alpha} \hat{p}_z)$, where $\bar{\alpha} \in [0, \alpha]$. How does this affect the proof?
Lastly, I would propose for the authors to change the name of their condition to the “Armijo condition” rather than using the term “1st Wolfe condition” since the Wolfe condition is typically associated with the curvature condition (p_k’ g_new >= c_2 p_k’ g_k), hence referring to a very different line search.
2. Design of the Quasi-Newton Matrix
The authors develop an approach for designing the quasi-Newton matrix that does not strictly impose symmetry or the secant condition. The authors claim that this done because “it is not obvious that enforced symmetry necessarily produces a better search direction” and “treating the [secant] condition less strictly might be helpful when [the Hessian] approximation is poor”. This explanation seems insufficient to me to explain why relaxing these conditions via a regularized least-squares approach would yield a better algorithm, particularly in the noisy or stochastic setting. The lack of symmetry seems particularly strange; one would expect the true Hessian in the stochastic setting to still be symmetric, and one would still expect the secant condition to hold if the “true” gradients were accessible. It is also unclear how this approach takes advantage of the stochastic structure that exists within the problem.
Additionally, the quasi-Newton matrix is defined based on the solution of a regularized least squares problem with a regularization parameter lambda. It seems to me that the key to the approximation is the balance between the two terms in the objective. How is lambda chosen? What is the effect of lambda as a tuned parameter, and how does it affect the quality of the Hessian approximation? It is unclear to me how this could be chosen in a more systematic way.
The matrix also does not ensure positive definiteness, hence requiring a multiple of the gradient direction to be added to the search direction. In this case, the key parameter beta must be chosen carefully. What is a typical value of beta that is used for each of these problems? One would hope that beta is small, but if it is large, it may suggest that the search direction is primarily dominated by the stochastic gradient direction and hence the quasi-Newton matrix is not useful. The interplay of these different parameters needs to be investigated carefully.
Lastly, since (L-)BFGS use a weighted Frobenius norm, I am curious why the authors decided to use a non-weighted Frobenius norm to define the matrix. How does changing the norm affect the Hessian approximation?
All of these questions place the onus on the numerical experiments to see if these relaxations will ultimately yield a better algorithm.
3. Numerical Experiments
As written, although the range of problems is broad and the numerical experiments show much promise, I do not believe that I could replicate the experiments conducted in the paper. In particular, how is SVRG and L-BFGS tuned? How is the steplength chosen? What (initial) batch sizes are used? Is the progressive batching mechanism used? (If the progressive batching mechanism is not used, then the authors should refer to the original multi-batch paper by Berahas, et al. [1] which do not increase the batch size and use a constant steplength.)
In addition, a more fair comparison would include the stochastic quasi-Newton method in [2] that also utilize diminishing steplengths, which use Hessian-vector products in place of gradient differences. Multi-batch L-BFGS will only converge if the batch size is increased or the steplength diminished, and it’s not clear if either of these are done in the paper.
Typos/Grammatical Errors:
- Pg. 1: Commas are needed in some sentences, i.e. “Firstly, for large scale problems, it is…”; “…compute the cost function and its gradients, the result is…”
- Pg. 2: “Interestingly, most SG algorithms…”
- Pg 3: Remove “at least a” in second line
- Pg. 3: suboptimal, not sup-optimal
- Pg. 3: “Such a solution”, not “Such at solution”
- Pg. 3: Capitalize Lemma
- Pg. 4: fulfillment, not fulfilment
- Pg. 7: Capitalize Lemma
- Pg. 11: Before (42), Cov \hat{g} = \sigma_g^2 I
- Pg. 11: Capitalize Lemma
Summary:
In summary, although the ideas appear to provide better numerical performance, it is difficult to evaluate if the ideas proposed in this paper actually yield a better algorithm. Many algorithmic details are left unanswered, and the paper lacks mathematical or empirical evidence to support their claims. More experimental and theoretical work is needed before the manuscript can be considered for publication.
References:
[1] Berahas, Albert S., Jorge Nocedal, and Martin Takác. "A multi-batch l-bfgs method for machine learning." Advances in Neural Information Processing Systems. 2016.
[2] Byrd, Richard H., et al. "A stochastic quasi-Newton method for large-scale optimization." SIAM Journal on Optimization 26.2 (2016): 1008-1031.
[3] Schraudolph, Nicol N., Jin Yu, and Simon Günter. "A stochastic quasi-Newton method for online convex optimization." Artificial Intelligence and Statistics. 2007.


ADVERSARIAL DOMAIN ADAPTATION FOR STABLE BRAIN-MACHINE INTERFACES
Brain-Machine Interfaces (BMIs) have recently emerged as a clinically viable option to restore voluntary movements after paralysis. These devices are based on the ability to extract information about movement intent from neural signals recorded using multi-electrode arrays chronically implanted in the motor cortices of the brain. However, the inherent loss and turnover of recorded neurons requires repeated recalibrations of the interface, which can potentially alter the day-to-day user experience. The resulting need for continued user adaptation interferes with the natural, subconscious use of the BMI. Here, we introduce a new computational approach that decodes movement intent from a low-dimensional latent representation of the neural data. We implement various domain adaptation methods to stabilize the interface over significantly long times. This includes Canonical Correlation Analysis used to align the latent variables across days; this method requires prior point-to-point correspondence of the time series across domains. Alternatively, we match the empirical probability distributions of the latent variables across days through the minimization of their Kullback-Leibler divergence. These two methods provide a significant and comparable improvement in the performance of the interface. However, implementation of an Adversarial Domain Adaptation Network trained to match the empirical probability distribution of the residuals of the reconstructed neural signals outperforms the two methods based on latent variables, while requiring remarkably few data points to solve the domain adaptation problem.

Here the authors define a BMI that uses an autoencoder -> LSTM -> EMG. The authors then address the problem of data drift in BMI and describe a number of domain adaptation algorithms from simple (CCA to more complex ADAN) to help ameliorate it. There are a lot of extremely interesting ideas in this paper, but the paper is not particularly well written, and the overall effect to me was confusion. What problem is being solved here? Are we describing using latent variables (AE approach) for BMI?  Are we discussing domain adaptation, i.e. handling the nonstationarity that so plagues BMI and array data?  Clearly the issue of stability is being addressed but how?  A number of different approaches are described from creating a pre-execution calibration routine whereby trials on the given day are used to calibrate to an already trained BMI (e.g. required for CCA) to putting data into an adversarial network trained on data from earlier days.  Are we instead attempting to show that a single BMI can be used across multiple days?
This paper is extremely interesting but suffers from lack of focus, rigor, and clarity.
Focus :
AE to RNN to EMG is that the idea to compare vs. Domain adaptation via CCA/KLDM/ADAM.
Of course a paper can explore multiple ideas, but in this case the comparisons and controls for both are not adequate.
Rigor:
What are meaningful comparisons for all for the AE and DA portions? The AE part is strongly related to either to Kao 2017 or Pandarinath 2018 but nothing like that is compared.  The domain adaptation part evokes data augmentation strategies of Sussillo 2016 but that is not compared.

If I were reviewing this manuscript for a biological journal a rigorous standard would be online BMI results in two animals.  Is there a reason why this isn’t the standard for ICLR? Is the idea that non-biological journals / conferences are adequate to vet new ideas before really putting them to the test in a biological journal?  The manuscript is concerned with the vexing problem of BMI stability of time, which seems to be a problem where online testing in two animals would be critical. (I appreciate this is a broader topic relevant to the BMI field beyond just this paper, but it would be helpful to get some thinking on this in the rebuttal).
Clarity :
This paper needs to be pretty seriously clarified.  The mathematical notation is not adequate to the job, nor is the motivation for the varied methodology. I cannot tell if the subscript is for time or for day. Also, what is the difference between z_0 vs. Z_0? I do not know what exactly is going into the AE or the ADAN.
The neural networks are not described to a point where one could reproduce this work. The notation for handling time is inadequate.   E.g. despite repeated readings I cannot tell how time is handled in the auto-encoder, e.g. nxt is vectorized vs feeding n-sized vector one time step at a time?
Questions
What is the point of the latent representation in the AE if it is just fed to an LSTM? Is it to compare to not using it?
Page 3, how precisely is time handled in the AE?  If time is just vectorized, how can one get real-time readouts? In general there is not enough detail to understand what is implemented in the AE. If only one time slice is entered into AE, then it seems clear AE won’t be very good because one desires latent representation of the dynamics, not single time slices.
How big is the LSTM used to generate the EMG?
It seems like a the most relevant baseline is to compare to the data perturbation strategies in Sussillo 2016.  If you have an LSTM already up and running to predict EMG, this seems very doable.
Page 4, “We then use an ADAN to align either the distribution of latent variables or the distributions of the residuals of the reconstructed neural data, the latter a proxy for the alignment of the neural latent variables.”  This sentence is not adequate to explain the concepts of the various distributions, the residuals of reconstructed neural data (where do the residuals come from?), and why is one a proxy for the other.  Please expand this sentence into a few sentences, if necessary to define these concepts for the naive reader.
Page 5, What parameters are minimized in equation (2)? Please expand the top sentence of page 5.
Page 6, top - “In contrast, when the EMG predictor is  trained simultaneously with the AE…” Do you mean there is again a loss function defined by both EMG prediction and AE and summed, and then backprop is used to train both in an end-to-end fashion?  Please clarify.
Page 8, How do the AE results and architecture fit into the EMG reconstruction “BMI” results? Is that all decoding results are first put through the AE -> LSTM -> EMG pipeline? I.e. your BMI is neural data -> AE -> LSTM -> EMG?  If so, then how does the ADAN / CCA and KLDM fit in?  You first run those three DA algorithms and then pipe it through the BMI?
Page 8, How can you say that the BMI improvement of 6% is meaningful to the BMI user if you did not test the BMI online?


PREDICTING THE GENERALIZATION GAP IN DEEP NETWORKS WITH MARGIN DISTRIBUTIONS
Recent research has demonstrated that deep neural networks can perfectly fit randomly labeled data, but with very poor accuracy on held out data. This phenomenon indicates that loss functions such as cross-entropy are not a reliable indicator of generalization. This leads to the crucial question of how generalization gap can be predicted from training data and network parameters. In this paper, we propose such a measure, and conduct extensive empirical studies on how well it can predict the generalization gap. Our measure is based on the concept of margin distribution, which are the distances of training points to the decision boundary. We find that it is necessary to use margin distributions at multiple layers of a deep network. On the CIFAR-10 and the CIFAR-100 datasets, our proposed measure correlates very strongly with the generalization gap. In addition, we find the following other factors to be of importance: normalizing margin values for scale independence, using characterizations of margin distribution rather than just the margin (closest distance to decision boundary), and working in log space instead of linear space (effectively using a product of margins rather than a sum). Our measure can be easily applied to feedforward deep networks with any architecture and may point towards new training loss functions that could enable better generalization.

After author response, I have increased my score. I'm still not 100% sure about the interpretation the authors provided for the negative distances.
The paper is well written and is mostly clear. (1st line on page 4 has a typo, \bar{x}_k in eq (4) should be \bar{x}^l?)
Novelty: I am not sure whether the paper adds any significant on top of what we know from Bartlett et al., Elsayed et al. since:
(i). The fact that "normalized" margins are strongly correlated with the test set accuracy was shown in Bartlett et al. (figure 1.). A major part of the definition comes from there or from the reference they cite;
(ii). Taylor approximation to compute the margin distribution is in Elsayed et al.;
(iii). I think the four points listed in page 2 (which make the distinction between related work) is misleading: the way I see it is that the authors use the margin distribution in Elsayed et al which simply overcomes some of the obstacles that norm based margins may face. The only novelty here seems to be that the authors use the margin distribution at each layer.
Technical pitfalls: Computing the d_{f,x,i,j} using Equation (3) is missing an absolute value in the numerator as in equation (7) Elsayed et al.. The authors interpret the negative values as misclassification: why is it true? The margin distribution used in Bartlett et al. (below Figure 4 on page 5 in arxiv:1706.08498) uses labeled data and it is obvious in this case to interpreting negative values as misclassification. I don't see how this is true for eq (3) here in this paper. Secondly, why are negative points ignored?? Misclassified points in my opinion are equally important, ignoring the information that a point is misclassified doesn't sound like a great idea. How do the experiments look if we don't ignore them?
Experiments: Good set of experiments. However I find the results to be mildly taking the claims of the authors made in four points listed in page 2 away: Section 4.1, "Empirically, we found constructing this only on four evenly-spaced layers, input, and 3 hidden layers, leads to good predictors.". How can the authors explain this?
By using linear models, authors implicitly assume that the relationship between generalization gaps and signatures are linear (in Eucledian or log spaces). However, from the experiments (table 1), we see that log models always have better results than linear models. Even assuming linear relationship, I think it is informative to also provide other metrics such as MSE, AIC, BIC etc..


HINDSIGHT POLICY GRADIENTS
A reinforcement learning agent that needs to pursue different goals across episodes requires a goal-conditional policy. In addition to their potential to generalize desirable behavior to unseen goals, such policies may also enable higher-level planning based on subgoals. In sparse-reward environments, the capacity to exploit information about the degree to which an arbitrary goal has been achieved while another goal was intended appears crucial to enable sample efficient learning. However, reinforcement learning agents have only recently been endowed with such capacity for hindsight. In this paper, we demonstrate how hindsight can be introduced to policy gradient methods, generalizing this idea to a broad class of successful algorithms. Our experiments on a diverse selection of sparse-reward environments show that hindsight leads to a remarkable increase in sample efficiency.

This paper extends the work of Hindsight Experience Replay to (goal-conditioned) policy gradient methods. Hindsight, which allows one to learn policies conditioned on some goal g, from off-policy experience generated by following goal g’, is cast in the framework of importance sampling. The authors show how one can simply rewrite the goal-conditioned policy gradient by first sampling a trajectory, conditioned on some goal $g’$ and then computing the closed form gradient in expectation over all goals. This gradient is unbiased if the rewards are off-policy corrected along the generated trajectories. While this naive formulation is found to be unstable , the authors propose a simple normalized importance sampling formulation which appears to work well in practice. To further reduce variance and computational costs, the authors also propose goal subsampling mechanisms, which sample goals which are likely along the generated trajectories. The method is evaluated on the same bit-flipping environment as [1], and a variety of discrete environments (grid worlds, Ms. Pac-Man, simulated robot arm) where the method appears highly effective. Unfortunately for reasons which remain unclear, hindsight policy gradients with value baselines appear unstable.
Quality:
This paper scores high wrt. quality. The theoretical contributions of the method are solid, the experiments are well designed and highlight the efficacy of the method, as well as areas for improvement. In particular, I commend the authors for the rigorous analysis (bootstrapped error estimates, separate seeds for hyper-parameters and reporting test error, etc.), including the additional results found in the appendix (sensitivity and ablative analyses). That being said, the paper could benefit from experiments in the continuous control domain and a direct head-to-head comparison with HER. While I do not anticipate the proposed method to outperform HER in terms of data-efficiency (due to the use of replay) the comparison would still be informative to the reader.
Clarity:
The paper is well written and easy to follow. If anything, the authors could have abridged sections 2 and 3 in favor of other material found in the Appendix, as goal-conditioned policy gradients (and variants) are straightforward generalizations of standard policy gradient methods.
Originality:
Novelty is somewhat low for the paper as Hindsight Experience Replay already presented a very similar off-goal-correction mechanism for actor-critic methods (DDPG). The method is also very similar to [2], the connection to which should also be discussed.
Significance.
Despite the low novelty, I do believe there is value in framing “hindsight” as importance sampling in goal-conditioned policy gradients. This combined with the clear presentation and thorough analysis in my opinion warrants publication and will certainly prove useful to the community. Significance could be improved further should the paper feature a more prominent discussion / comparison to HER, along with a fix for the instabilities which occur when using their method in conjunction with a value baseline.
[1] Hindsight Experience Replay. Marcin Andrychowicz et al.
[2] Data-Efficient Hierarchical Reinforcement Learning. Ofir Nachum, Shixiang Gu, Honglak Lee, Sergey Levine.
Detailed Comments:
* Section 2: “this formulation allows the probability of a state transition given an action to change across time-steps within an episode”. I do not understand this statement, as $p(s_{t+1} \mid s_t, a_t)$ is the same transition distribution found in standard MDPs, and appears stationary wrt. time.
* Theorems 3.1 - 3.1 (and equations). A bit lengthy and superfluous. Consider condensing the material.
* Section 5: I found the change in notation (from lower to upper-case) somewhat jarring. Also, the notation used for empirical samples from the mini-batch is confusing. If $A^{(i)}_t}$ is meant to be the action at time-step $t$ for the $i$-th trajectory in the minibatch, then what does $G^{(i)} = g$ mean ? I realize this means evaluating the probability by setting the goal state to $g$, but this is confusing especially when other probabilities are evaluated conditioned on $G^{(i)}$ directly.
* Section 6. “Which would often require the agent to act after the end of an episode”. Do you mean that most episodes have length T’ < T, and as such we would “waste time” generating longer trajectories ?
* RE: Baseline instabilities. Plotting the loss function for the value function could shed light on the instability.


Online learning has attracted great attention due to the increasing demand for systems that have the ability of learning and evolving. When the data to be processed is also high dimensional and dimension reduction is necessary for visualization or prediction enhancement, online dimension reduction will play an essential role. The purpose of this paper is to propose new online learning approaches for supervised dimension reduction. Our first algorithm is motivated by adapting the sliced inverse regression (SIR), a pioneer and effective algorithm for supervised dimension reduction, and making it implementable in an incremental manner. The new algorithm, called incremental sliced inverse regression (ISIR), is able to update the subspace of significant factors with intrinsic lower dimensionality fast and efficiently when new observations come in. We also refine the algorithm by using an overlapping technique and develop an incremental overlapping sliced inverse regression (IOSIR) algorithm. We verify the effectiveness and efficiency of both algorithms by simulations and real data applications.

This paper studies sufficient dimension reduction problem, and proposes an incremental sliced inverse regression algorithm. Numerical experiments are provided to demonstrate the effectiveness of the proposed algorithms.
The sliced inverse regression here is nothing but generalized eigenvalue decomposition:
Ax=lambda Bx.
Note that Multiclass Fisher Linear Discriminant Analysis, Canonical Correlation Analysis, Nonlinear Manifold Embedding and many subspace learning methods can also be formulated as generalized eigenvalue decomposition. All these methods need to compute covariance-like matrices in the additive form, which makes incremental update very convenient.
The incremental generalized eigenvalue decomposition has been extensively studied for over decades, especially between 1995 and 2005 in the face recognition community. I am just listing a few here:
Ye et al., IDR/QR: An Incremental Dimension Reduction Algorithm via QR Decomposition, 2005
Law and Jain, Incremental Nonlinear Dimensionality Reduction by Manifold Learning, 2006
Yan et al. Towards incremental and large scale face recognition, 2011
Ghassabeh et al. A New Incremental Face Recognition System, 2007
Song et al. A Novel Supervised Dimensionality Reduction Algorithm for Online Image Recognition, 2006.
Wang et al. Incremental two-dimensional linear discriminant analysis with applications to face recognition, 2010.
Salman et al. Efficient update of the covariance matrix inverse in iterated linear discriminant analysis, 2010
Park and Park, A comparison of generalized linear discriminant analysis algorithms, 2008
Wang, INCREMENTAL AND REGULARIZED LINEAR DISCRIMINANT ANALYSIS, 2012
These algorithms become less popular/known now, because (1) they are not scalable and efficient for large p, and (2) these classical dimensionality reduction methods perform poorly in many tasks, compared with the state of art results.
This paper only cites a few papers on incremental LDA,  but does not even mention that both LDA and SIR are essentially solving similar optimization problems. Moreover, it does not compare the results with any of the above references, either.
This paper even claims applying the Sherman–Morrison formula as the contribution. However, such an  update has been used in Salman et al. 2010, Park and Park 2008, Wang 2012.
In summary, this paper is far below the bar of ICLR.
Minor: There are numerous typos in this paper. The authors even misspell "Morrison" in the Sherman–Morrison formula as "Morison".


FUNCTION SPACE PARTICLE OPTIMIZATION FOR BAYESIAN NEURAL NETWORKS
While Bayesian neural networks (BNNs) have drawn increasing attention, their posterior inference remains challenging, due to the high-dimensional and overparameterized nature. Recently, several highly flexible and scalable variational inference procedures based on the idea of particle optimization have been proposed. These methods directly optimize a set of particles to approximate the target posterior. However, their application to BNNs often yields sub-optimal performance, as they have a particular failure mode on over-parameterized models. In this paper, we propose to solve this issue by performing particle optimization directly in the space of regression functions. We demonstrate through extensive experiments that our method successfully overcomes this issue, and outperforms strong baselines in a variety of tasks including prediction, defense against adversarial examples, and reinforcement learning.

Based on the revision, I am willing to raise the score from 5 to 7.
==========================================
The authors address the problems of variational inference in over-parameterized models and the problem of the collapse of particle-optimization-based variational inference methods (POVI). The authors propose to solve these problems by performing POVI in the space of functions instead of the weight space and propose a heuristic approximation to POVI in function spaces.
Pros:
1) I believe that this work is of great importance to the Bayesian deep learning community, and may cause a paradigm shift in this area.
2) The method performs well in practice, and alleviates the over-parameterization problem, as shown in Appendix A.
3) It seems scalable and easy to implement (and is similar to SVGD in this regard), however, some necessary details are omitted.
Cons:
1) The paper is structured nicely, but the central part of the paper, Section 3, is written poorly; many necessary details are omitted.
2) The use of proposed approximations is not justified
In order to be able to perform POVI in function-space, the authors use 4 different approximations in succession. The authors do not check the impact of those approximations empirically, and only assess the performance of the final procedure. I believe it would be beneficial to see the impact of those approximations on simple toy tasks where function-space POVI can be performed directly. Only two approximations are well-motivated (mini-batching and approximation of the prior distribution), whereas the translation of the function-space update and the choice of mu (the distribution, from which we sample mini-batches) are stated without any details.
Major concerns:
1) As far as I understand, one can see the translation of the function-space update to the weight-space update (2) as one step of SGD for the minimization of the MSE \sum_x (f(x; \theta^i) - f^i_l(x) - \eps v(f^i_l)(x))^2, where the sum is taken over the whole space X if it is finite, or over the current mini-batch otherwise. The learning rate of such update is fixed at 1. This should be clearly stated in the paper, as for now the update (2) is given without any explanation.
2) I am concerned with the theoretical justification paragraph for the update rule (3) (mini-batching). It is clear that if each marginal is matched exactly, the full posterior is also exactly matched. However, it would usually not be possible to match all marginals using parametric approximations for f(x). Moreover, it is not clear why would updates (3) even converge at all or converge to the desired point, as it is essentially the update for an optimization problem (minimization of the MSE done by SGD with a fixed learning rate), nested into a simulation problem (function-space POVI). This paragraph provides a nice intuition to why the procedure works, but theoretical justification would require more rigor.
3) Another approximation that is left unnoted is the choice of mu (the distribution over mini-batches). It seems to me from the definition of function-space POVI that we need to use the uniform distribution over the whole object space X (or, if we do not do mini-batching, we need to use the full space X). However, the choice of X seems arbitrary. For example, for MNIST data we may consider all real-values 28x28 matrices, where all elements lie on the segment [0,1]. Or, we could use the full space R^28x28. Or, we could use only the support of the empirical distribution. I have several concerns here:
3.1) If the particles are parametric, the solution may greatly depend on the choice of X. As the empirical distribution has a finite support, it would be dominated by other points unless the data points are reweighted. And as the likelihood does not depend on the out-of-dataset samples, all particles f^i would collapse into prior, completely ignoring the training data.
3.2) If the prior is non-parametric, f(x) for all out-of-dataset objects x would collapse to the prior, whereas the f(x) for all the training objects would perfectly match the training data. Therefore we would not be able to make non-trivial predictions for the objects that are not contained in the training set unless the function-space kernel of the function-space prior somehow prevents it. This poses a question: how can we ensure the ability of our particles to interpolate and extrapolate without making them parametric? Even in the parametric case, if we have no additional regularization and flexible enough models, they could overfit and have a similar problem.
These two concerns may be wrong, as I did not fully understand how the function-space prior distribution works, and how the function-space kernel is defined (see concern 4).
4) Finally, it is not stated how the kernels for function-space POVI are defined. Therefore, it is not clear how to implement the proposed technique, and how to reproduce the results. Also, without the full expression for the weight-space update, it is difficult to relate the proposed procedure to the plain weight-space POVI with the function value kernel, discussed in Appendix B.
Minor comments:
1) It is hard to see the initial accuracy of different models from Figure 3 (accuracy without adversarial examples). Also, what is the test log-likelihood of these models?
2) It seems that sign in line 5 on page 4 should be '-'
I believe that this could be a very strong paper. Unfortunately, the paper lacks a lot of important details, and I do not think that it is ready for publication in its current form.


The ability to detect when an input sample was not drawn from the training distribution is an important desirable property of deep neural networks. In this paper, we show that a simple ensembling of first and second order deep feature statistics can be exploited to effectively differentiate in-distribution and out-of-distribution samples. Specifically, we observe that the mean and standard deviation within feature maps differs greatly between in-distribution and out-of-distribution samples. Based on this observation, we propose a simple and efficient plug-and-play detection procedure that does not require re-training, pre-processing or changes to the model. The proposed method outperforms the state-of-the-art by a large margin in all standard benchmarking tasks, while being much simpler to implement and execute. Notably, our method improves the true negative rate from 39.6% to 95.3% when 95% of in-distribution (CIFAR-100) are correctly detected using a DenseNet and the out-of-distribution dataset is TinyImageNet resize. The source code of our method will be made publicly available.

Summary: A relatively simple approach for detecting out-of-distribution samples by having a parallel logistic regression model using simple statistics (mean and variance) over output of each batch normalisation layer, in order to discriminate between in-distribution and out-of-distribution samples. Results are appealing but presentation is lacking clarity at time and some doubts on the correctness of the experiments remain.
With the goal of detecting out-of-distribution sets, the authors propose to use logistic regression over simple statistics (mean and variance) of each batch normalization layer of CNN in order to discriminate between in-distribution (ID) and out-of-distribution (OOD) samples. They argue that ID and OOD samples can be discriminated with these statistics.
Quality: The motivations of the paper are clear, it aims at having better capacity to detect OOD samples with a method that involves less computations. However, the quality of the experiments is not good enough and I have doubts on their validity.
Originality: Ok. The proposal is relatively simple and is based on the intuition that statistics for the batch normalization is useful to detect OOD samples. The problem is not new, the approach is relatively ad hoc, but it works.
Significance of the work: The results reported are unreasonably good. Although the authors claim the improvement of detection of OOD is significant, the results achieved by detecting **all** the out-distribution samples sounds weird and irrational. How rejecting all Tiny-ImageNet is possible while there are several overlap between the classes presented in TinyImageNet vs. Cifar10/Cifar100 (cf. Table 8)? To me, it looks like the model either overfitted something else than the content of the images, maybe the background noise or similar regarding the nature of the data. More experiments with different in-distribution datasets should be made to be convincing. All experiments reported are using either Cifar10 or Cifar100 as in-distribution datasets.
The author also claim using few samples from a single OOD set is enough for training the regressor that provides OOD-ness score. Is it true for any OOD set or only a carefully chosen OOD set can demonstrate this behavior? What is the criteria for selecting a good OOD set for training the regressor?
Despite of the fact that the proposed method is heavily dependent on the threshold, the authors barely discuss of it. I am assuming that threshold is on OOD-ness score, is that correct? How does look like the OOD-ness score for an ID set over different OOD sets? Providing the OOD-ness score for ID and OOD could reflect how the proposed method is sensitive to a selected threshold. In other words, is selecting a fix threshold will to the TPR / FPR across different OODs.
The overall writing style is perfectible. I did not found the paper super clear in the presentation and it is difficult to really get all useful information for it. However, the authors appear knowledgeable of the literature and the overall structure is clear.
An example of lack of clarity in the explanations: in Table 7, I have difficulty to make sense of the 100% achieved for “Ours (pair)” vs “Ours”. Is the “Ours (pair)” the rate obtained with the exact pair used for adjusting the threshold, while “Ours” is on another dataset? If not, what this mean? Moreover, reporting columns all with 100% is not a good practice, it seems to be a stunt to impress the reader, while not carrying much in term of content and understanding.
In Table 8, I do not understand what the values in parenthesis means.
Another element: why for training the regressor, the IN and OOD samples are not selected from their corresponding training sets instead of splitting their test sets to a validation and test sets?


Assessing distance between the true and the sample distribution is a key component of many state of the art generative models, such as Wasserstein Autoencoder (WAE). Inspired by prior work on Sliced-Wasserstein Autoencoders (SWAE) and WAE-MMD (WAE using maximum mean discrepancy based distance function) we construct a new generative model -Cramer-Wold AutoEncoder (CWAE). CWAE cost function is based upon a characteristic kernel we introduce, called CramerWold kernel, and has a a simple closed-form in the case of normal prior. As a consequence, while simplifying the optimization procedure (no need of sampling necessary to evaluate the distance function in the training loop), CWAE performance matches quantitatively and qualitatively that of WAE-MMD and often improves upon SWAE.

The paper introduces a novel regularized auto-encoder architecture called the Cramer-Wold AutoEncoders (CWAE). It's objective (Eq. 7) consists of two terms: (i) a standard reconstruction term making sure the the encoder-decoder pair aligns nicely to accurately reconstruct all the training images and (ii) the regularizer, which roughly speaking requires the encoded training distribution to look similar to the standard normal (which is a prior used in the generative model being trained). The main novelty of the paper is in the form of this regularizer. The authors introduce what they call "the Cramer-Wold distance" (for definitions see Theorems 3.1 and 3.2) which is defined between two finite sets of D-dimensional points. The authors provide empirical studies showing that the proposed CWAE method achieves the same quality of samples (measured with FID scores) as the WAE-MMD model [1] previously reported in the literature, while running faster (by up to factor of 2 reduction in the training time, as the authors report).
While on the AE model / architecture side I feel the contribution is very marginal, I still think that the improvement in the training speed is something useful. Otherwise it is a nicely written and polished piece of work.
Detailed comments:
(1) My main problem with this paper is that the novel objective proposed by the authors in Eq. 7 is equivalent to the objective of WAEs appearing in Eq. 4 of [1] (up to a heuristic of applying logarithm to the divergence measure, which is not justified but meant to "improve the balance between two terms", see footnote 2), where the authors use the newly introduced Cramer-Wold divergence as a choice of the penalty term in Eq. 4 of [1].
(2) When viewed in this way, CW-distance introduced in Eq (2) closely resembles the unbiased U-statistic estimate of the MMD used in WAE-MMD [1, Algorithm 2]. In other words: it may be the case that there is a choice of a reproducing kernel k such that Eq. 2 of this paper is an estimate of MMD_k between two distributions based on the i.i.d. samples X and Y. Note that if it is indeed the case, this corresponds to the V-statistic and thus biased: in U-statistic the diagonal terms (that is i = i' and j = j' in forst two terms of eq 2) would be omitted. If this all is indeed the case, it is not surprising that the numbers the authors get in the experiments are so similar to WAE-MMD, because CWAE would be exactly WAE-MMD with a specific choice of the kernel.
(3) The authors make a big deal out of their proposed divergence measure not requiring samples from the prior as opposed to WAE-MMD. However, WAE-MMD does not necessarily need to sample from the prior when used with Gaussian prior and Gaussian RBF kernel, because in this case the prior-related parts of the MMD can be computed analytically.  In other words, if the computational advantage of CWAE compared to WAE-MMD comes from CWAE not sampling Pz, the computational overhead of WAE-MMD can be eliminated at least in the above-mentioned setting.
(4) based on the name "CW distance" I would expect the authors to actually prove that it is indeed a distance (i.e. all the main axioms).
(5) The authors override the CW distance: first in Theorem 3.1 they define it as a distance between two finite point clouds, and later in Theorem 3.2 they redefine it as a distance between a point cloud and the Gaussian distribution.
(6) What is image(X) in Remark 4.1?
[1] Tolstikhin et al., Wasserstein Auto-Encoders, 2017.