Training Deep Neural Networks (DNNs) that are robust to norm bounded adversarial attacks remains an elusive problem. While verification based methods are generally too expensive to robustly train large networks, it was demonstrated by Gowal et al. (2019)  that bounded input intervals can be inexpensively propagated from layer to layer through deep networks. This interval bound propagation (IBP) approach led to high robustness and was the first to be employed on large networks. However, due to the very loose nature of the IBP bounds, particularly for large/deep networks, the required training procedure is complex and involved. In this paper, we closely examine the bounds of a block of layers composed of an affine layer, followed by a ReLU, followed by another affine layer. To this end, we propose expected bounds (true bounds in expectation), which are provably tighter than IBP bounds in expectation. We then extend this result to deeper networks through blockwise propagation and show that we can achieve orders of magnitudes tighter bounds compared to IBP. Using these tight bounds, we demonstrate that a simple standard training procedure can achieve impressive robustness-accuracy trade-off across several architectures on both MNIST and CIFAR10.

Disclaimer: I have already reviewed this paper for another conference. I re-read it to assess the modifications made by the authors but I am aware of the comments that they received in the previous round of reviews. I'm also re-raising some concerns that I made in a previous review that the authors didn't address.
Summary
Interval Bound Propagation (IBP) is a fast method to propagate bounds through the activation of a Neural Network. It is however quite loose. The authors of this paper propose a different way of propagating the bounds, which is not a rigorous bound computation method, but for which they show that, with some strong assumptions on the distribution of the weights, the expectation of the generated results are valid bounds that are tighter than the ones generated by IBP.
The paper explains clearly the "how" of how these bounds are achieved. Equation 3 and the paragraph before it are good. (Essentially, as long as a ReLU is not in a "blocking" state where all its input are negative, treat it as an identity, and do a forward pass of the center of the input region).
The logical argument explaining the "why" these are valid is harder to understand and could be clarified. My understanding so far is:
- All the analysis is dependent on all weights being drawn from iid gaussian of zero mean.
- In addition to that, there is an assumption (Assumption 1) between the relation of bounds of network with random weight and some quantity (L_approx, U_approx) with random uniform inputs x.
The proof is done that the expectation of the bound proposed is "correct" with regards to (L_approx, U_approx) but then, the relation to the true bounds is given by the Assumption?
So the relation to the true bounds is only given by the Assumption?
Regarding the experiments:
* The caption of Figure 1 is misleading. It says that it shows that the proposed bounds are a super set of the interval bounds, while they are actually not (ratio is strictly smaller than 1). You can argue that they are close, but now that they are a superset when they aren't!
* The reporting of Figure 5 and 6 is weird because according to the text, each datapoint seems to be the average robustness of networks trained with different hyperparameters, so it's hard to interpret.
* I appreciate the effort of the author to include experiments involving a MIP solver returning the true bounds. This is very helpful in building confidence that the bounds generated are correct. How is the real network trained? Is this based on a robustly trained network or is it just standard training? Is 99% the nominal accuracy?
In general, I think that the paper would be better if there was more discussion of the failure modes of the method. There are easy to identify failure cases where the proposed bound is incorrect or loose. My opinion is that the paper would be stronger if it acknowledged them but then made the points that the bounds proposed are still good most of the time (which their experiment show) and useful (for example for training where the exact correctness of the bounds may not be the most important), rather than ignoring them and pretending that the bounds are perfect.
[From previous review]
Simple example of a network where the results would be quite loose, while IBP is tight:
Two layer NN, x = 0 \in R^n, eps = 1, W1 = 1000 * Identity, b1 = -999 * 1_n, a2 = -10 1_{1,n} , b2 = 0
u1 would be all positive, so the matrix M would simply be the identity.
L_M = a2^T M b1 - eps | a2^T M A1| vec(1) = (-10)*(-999)*n - 10 * (1000) * n = -19990 * n
U_M = a2^T M b1 + eps | a2^T M A1| vec(1) = (-10)*(-999)*n + 10 * (1000) * n = n
The actual bound (which also correspond in that case to what would be computed by IBP) is
L_gt = -10*n
u_gt = 0
Comments:
* "We prove to be true bounds in expectation" -> This is a bad formulation that should be rephrased. The expectation of the proposed bound is correct with regards to the expectation of the true bounds, but "True in expectation" is a bad formulation that doesn't reflect what is happening. Maybe True in expectation would mean that E_{A_1, a_2}(indicator(L_true > L_M)) -> 1 as n grows
* The assumption about gaussian weights in A1 and a2 seems strong but is at least partially motivated at the end of 3.2 (although not all networks are trained with l2 regularizer) What about the iid assumption? In case of a CNN for example, the weights are shared, so definetely won't fit this framework?
* I think that the paper would benefit from having some discussion of other methods that can derive "bounds" which may not actually be bounds. The works by Stefan Webb (A Statistical Approach to Assessing Neural Network Robustness, ICLR 2019) or Tsui Wei Weng (Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach, ICLR 2018) being others of the top of my head . While this paper clearly propose to do things very differently, I think the discussion would have been very valuable.
* While reviewing the paper, I also spotted some strong similarities between the methods proposed (particularly subsection 3.3) and the Fastlin method. The computation mechanism of Fastlin (propagate recursively  from the end through the linear layers and through diagonal matrices that replace the ReLU activation function) is exactly the same as the proposed one (Fastlin goes through the notational burden of handling the bias). The only difference is in the way the coefficients of the diagonal matrix are computed. (here, it's 0 if the Relu is blocking, and 1 otherwise. Fastlin has 0 if blocking, 1 if passing but a coefficient in between (u/u-l) if the reLU is ambiguous).
If that connection is correct, then the benefits of the proposed methods are limited: neither Fastlin nor IBP dominate each other, and the computational costs for Fastlin is higher, due to having to propagate the full linear maps here noted G from the end to the input to obtain the bounds, as the authors seem to have also realised ("it is expensive to compute our bounds using the procedure in Section 3.3, so instead, we obtain matrices Mi using the easy-to-compute IBP upper bounds.")
Opinion:
I am bothered by the framing of the results that the authors employ, as their theoretical results are, to the best of my understanding, dependent on strong assumptions. Some of the experimental results are also over-exaggerated (caption saying that things are superset while the experimental results show something different), and the relation and contextualization with regards to existing literature is lacking ( both against other works with similar aims but different methods, and with works with similar method but a slightly different aim)
Typos:
Page 7, "are are much"
Page 8,"When kappa=0", no need for uppercase
The references should ideally be cleaned and put in a consistent format. In the text of the paper, Some citation have first name + last name of one author, some have two out of all the authors, some have the et al. format.... It's all over the place.
In the References section, some authors list are shuffled (at least the "Scaling provable adversarial" paper), some are missing the conference where paper where cited and just cite the arxiv version


Variational inference (VI) methods and especially variational autoencoders (VAEs) specify scalable generative models that enjoy an intuitive connection to manifold learning -with many default priors the posterior/likelihood pair q(z|x)/p(x|z) can be viewed as an approximate homeomorphism (and its inverse) between the data manifold and a latent Euclidean space. However, these approximations are well-documented to become degenerate in training. Unless the subjective prior is carefully chosen, the topologies of the prior and data distributions often will not match. Conversely, diffusion maps (DM) automatically infer the data topology and enjoy a rigorous connection to manifold learning, but do not scale easily or provide the inverse homeomorphism. In this paper, we propose a) a principled measure for recognizing the mismatch between data and latent distributions and b) a method that combines the advantages of variational inference and diffusion maps to learn a homeomorphic generative model. The measure, the locally bi-Lipschitz property, is a sufficient condition for a homeomorphism and easy to compute and interpret. The method, the variational diffusion autoencoder (VDAE), is a novel generative algorithm that first infers the topology of the data distribution, then models a diffusion random walk over the data. To achieve efficient computation in VDAEs, we use stochastic versions of both variational inference and manifold learning optimization. We prove approximation theoretic results for the dimension dependence of VDAEs, and that locally isotropic sampling in the latent space results in a random walk over the reconstructed manifold. Finally, we demonstrate our method on various real and synthetic datasets, and show that it exhibits performance superior to other generative models.

The paper proposes a new generative model for unsupervised learning, based on a diffusion random walk principle inspired by the manifold learning literature. The basic idea is to (probabilistically) map points to a latent space, perform a random walk in that space, and then map back to the original space again. Learning of the suitable maps is achieved by casting the problem in a variational inference framework.
The paper is generally well-written, and clearly states out its goals and motivation. Sections 1 - 3 in particular give a nice overview of the broader context of the paper, and its aim of borrowing ideas from manifold learning and variational autoencoders. The particular aim on using concepts from manifold learning to avoid mode collapse - corresponding to the underlying homeomorphism losing its bijectivity - is in particular intriguing.
The method itself is intuitive at a high level, although I did have some difficulty with Section 4:
- in 4.1, one begins by considering the local evidence. This requires drawing a point from U_x, which is defined to a be set. I presume this means one draws uniformly from this set?
- Eqn 3 does not apparently have the same structure as Eqn 2. In particular, the first term in (2) is a function of x, but for (3) it is not a function of x'. How does conditioning affect the ELBO?
- I was not sure how to interpret the statement that pθ(x'|x) ≈ ψ^{-1}(q(z'|x)). Do you mean the distribution is strongly concentrated around this value? Note also an extra "]" in the latter.
- Eqn 5 should presumably be an equality? Also, it was not clear what the d in |.|_d^2 means, and why one does not use ||.||^2.
- At a higher level, given that x' ~ U_x originally, why do we now draw x' ~ p(x'|x)?
- Arriving at 4.2, it was not clear what "The sampling procedure" refers to, i.e., which of the steps in 4.1 it is seeking to specify or augment. It would be useful to clearly lay out the objective function that is being optimised, and how this section fits into that.
- In 4.3, it seemed as if the discussion of the neighbourhood reconstruction error would be better placed in 4.1 itself. It appears to be a justification of the already-derived Eqn 5.
- Algorithm 2, is there a need to introduce Z_t? It is a bit confusing that, e.g., Z_1 is first written to in iteration 1 by g(Z_0, ε), and then by ψ(X_1) in the second iteration.
The authors also claim a contribution to be the identification of a principled measure to identify mismatch between latent and data distributions. This "bi-Lipschitz" property is only introduced in Sec 5.1, and the discussion is not too approachable to someone unfamiliar with the area. In particular:
- it is not clear how precisely the discussion in this section relates to the VDAE algorithm described in the previous section.
- precisely what quantity we are to compute so as to verify this condition remains elusive. The abstract and introduction made me expect that the property is practically verifiable, but it was not clear from this section whether it is so.
- the conclusion or key takeaway of this subsection was unclear. I gather that Jones et al. established the existence of a neighbourhood wherein one can define a bi-Lipschitz mapping to R^d for suitable d. But how does this relate to latent and data space mismatch?
The experiments show that the proposed method can generate meaningful samples for synthetic manifold data, as well as on the MNIST dataset. I would've preferred more discussion of the results in Sec 4.1. I also was hoping for a clearer illustration of mode-collapse problems on standard benchmarks for GANs, with comparison of results to, e.g., those of Wasserstein-GANs (beyond those in Sec 6.1) or other proposals that aim to mitigate mode collapse.
Minor comments:
- Fig 1, the text in the middle panel is hard to read in black and white.
- SpectralNet is mentioned a few times but never formally introduced.
- notationally, Section 4 is a little heavy. I would suggest considering to omit the subscript θ's in the function ψ and its inverse.
- when mentioning the "reparametrisation trick", please provide a citation.
- what is \mathbb{X} in Theorem 2?
- "completementary" -> "complementary"
- " rejection sampling Azadi et al. (2018); Turner et al. (2018)." -> " rejection sampling (Azadi et al. 2018; Turner et al. 2018)."
- some form of Conclusion would be appropriate.


Spike-sorting is of central importance for neuroscience research. We introduce a novel spike-sorting method comprising a deep autoencoder trained end-to-end with a biophysical generative model, biophysically motivated priors, and a selfsupervised loss function to training a deep autoencoder. The encoder infers the action potential event times for each source, while the decoder parameters represent each source's spatiotemporal response waveform. We evaluate this approach in the context of real and synthetic multi-channel surface electromyography (sEMG) data, a noisy superposition of motor unit action potentials (MUAPs). Relative to an established spike-sorting method, this autoencoder-based approach shows superior recovery of source waveforms and event times. Moreover, the biophysical nature of the loss functions facilitates interpretability and hyperparameter tuning.
Overall, these results demonstrate the efficacy and motivate further development of self-supervised spike sorting techniques.

This paper describes a machine learning model for learning spiking representations from EMG (electromyography) like data. The basic model is an autoencoder, where the bottleneck layer is designed to project the waveform inputs into "spike" representations. Specifically, there is a DSD (Deep Spike Decoder) encoder and decoder, where the encoder is implemented as either a DenseNet[1]-like or Tiramisu[2]-like architecture. The bottleneck seems to be implemented by a linear layer binarized with a Gumbel-Softmax activation. The decoder is a linear layer. Several losses are considered, including a L4-norm (!), a sparsity loss, a parsimony loss, a uniqueness loss, and a refractory period loss. The model is validated qualitatively on real EMGs, and quantitatively on synthetic data.
The strengths of the paper are the following:
- Spiking models are very interesting class of models, which if attained would have a great impact on several other areas of machine learning.
- I like the straightforward design that is fit to the purpose of generating spiking representations. A Gumbel-Softmax has proven its validity and is a logical fit to the problem setup.
- I like the simpler setup the paper chooses.
  -- First, the paper chooses not to take the route of trying to learn backpropagation through infinite time steps, as often happens in spiking methods. This is a beast on each own, which would be nice to set as an ultimate goal (in my opinion). For now, however, it's ok if we forego this requirement.
  -- Second, the paper chooses not to assume that signals come asynchronously, which again makes things unnecessarily complex, given the state of the field.
The weaknesses of the paper:
- While the problem setup is simple, perhaps it makes some assumptions that are too strong. In my opinion, the strongest one is that of batch-learning (for the lack of a better name). Often, spiking models are studied in an online setup where the data comes online (continuously), and then spikes are generated when sufficient changes in the inputs warrant a delta-difference (spike). When in batch mode, however, it should be quite much easier to obtain spikes that lead to good reconstructions, as there is no much need for a "memory" mechanism. While one could argue whether an online setup is necessary or not, in my opinion it is necessary and would make a spiking model challenging and interesting to learn. Otherwise, it looks like a conventional autoencoder, only with spikes instead of dense representations.
- The model is unclear and writing vague.
- First of all, after reading the abstract and the introduction I get the feeling that the model is probabilistic, as there is mention of priors and autoencoders. Also, later on a Gumbel-Softax is mentioned for binarization. Gumbel-Softmax is a continuous function and binarization makes sense when sampling, that is when assuming a generative process. However, the rest of the paper seems not to explain a probabilistic or generative model. There is no explicit prior discussed. There is no sampling procedure discussed. The losses are explained but not within a particular probabilistic or stochastic framework. If the model is not stochastic, one way or the other, how are the discrete spikes obtained and how is the system is trained?
- All the loss functions, albeit logical when taken one by one, they do look ad hoc and appearing out of the blue. This perhaps relates to the previous point, where it is not clear if the model is stochastic or deterministic. If it is deterministic, it is ok to have all the loss functions appearing like that, but I would expect some more explanation on what purpose do they fill w.r.t. the spiking model. In the end, what does the deep spiker model try to achieve? Learn spikes as representations? Recover the original spikes? Be sparse? If yes, why and how sparse? Be energy-efficient?
- Some design choices are quite unclear. Generally, it is fair to say that the experimental and design setups are rather simple: multiple 1-D waveforms and not much noise (from what I get). In that context, it is not clear why DenseNets, or even Tiramisu-Nets are used as an encoder; especially when the decoder is a simple linear model.
- Also, phrases like "what good combinations are useful ... the hyperparametres of DSD" do not add to the clarity. There is no exploration of hyperparameters in the experiments and no individual examination of the contribution of each loss (unless I missed it somewhere).
- Similarly, what does "For the DSD ... between 10 minutes to an hour depending on ...".  Such statements should be more precise, for instance plotting wall clock time vs training loss.
All in all, while I like the motivation and the original direction, I believe there exist a lot of questions  unanswered before acceptance.


We propose a probabilistic generative model for unsupervised learning of structured, interpretable, object-based representations of visual scenes. We use amortized variational inference to train the generative model end-to-end. The learned representations of object location and appearance are fully disentangled, and objects are represented independently of each other in the latent space. Unlike previous approaches that disentangle location and appearance, ours generalizes seamlessly to scenes with many more objects than encountered in the training regime. We evaluate the proposed model on multi-MNIST and multi-dSprites data sets.

This paper introduces a compositional generative model of images, where the image is described by a variable number of latent variables. Moreover, the latent variables are disentangled, in the sense that they represent different parts of the scene, and where appearance and location are described separately. While the generative model is very similar to AIR [1], the central claim of the paper is that the proposed inference scheme can generalize to a much higher number of objects than seen during training, which is demonstrated empirically, and with which AIR struggles. Better generalization is achieved by removing the recurrent core of AIR and replacing it with a fully-convolutional inference network, which predicts appearance vectors at every location in the feature map. The appearance vectors are then stochastically sampled without replacement according to a location distribution. Unlike AIR, this approach does not model the object scale.
I recommend REJECTing this paper. While the improved generalization performance is a useful property, it has been achieved previously and in a very similar fashion by SPAIR [2], which follows a similar model design. SPAIR still uses an RNN, while the proposed approach does not, but this is only a small simplification and does not warrant publication at a top tier conference. There are no other contributions in this paper. Additionally, on the one hand, the experimental evaluation is insufficient: the proposed approach is compared only against a fully-convolutional VAE, while very similar models like AIR [1], SPAIR [2], SuPAIR [3] are not considered. On the other hand, the experimental section focuses on the disentanglement of representations, which is a) evident from the model construction and b) achieved in all previous models.
The paper is clearly written, and the presented generative model is interesting. Having said that, the problem that this paper addresses is mostly solved in [2] and [4]; also both [2] and [4] scale the general approach introduced in [1] to much higher number of objects (in the hundreds) and more general settings (real images, atari games).
[1] Eslami et. al., “Attend, Infer, Repeat:...”, NIPS 2016.
[2] Crawford and Pineau, “Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks”, AAAI 2019.
[3] Stelzner et. al., “Faster Attend-Infer-Repeat with Tractable Probabilistic Models”, ICML 2019.
[4] Jiang et. al., “Scalable Object-Oriented Sequential Generative Models”, arXiv 2019.


Recent developments in natural language representations have been accompanied by large and expensive models that leverage vast amounts of general-domain text through self-supervised pre-training. Due to the cost of applying such models to down-stream tasks, several model compression techniques on pre-trained language representations have been proposed (Sun et al., 2019a;Sanh, 2019). However, surprisingly, the simple baseline of just pre-training and fine-tuning compact models has been overlooked. In this paper, we first show that pre-training remains important in the context of smaller architectures, and fine-tuning pre-trained compact models can be competitive to more elaborate methods proposed in concurrent work. Starting with pre-trained compact models, we then explore transferring task knowledge from large fine-tuned models through standard knowledge distillation. The resulting simple, yet effective and general algorithm, Pre-trained Distillation, brings further improvements. Through extensive experiments, we more generally explore the interaction between pre-training and distillation under two variables that have been under-studied: model size and properties of unlabeled task data. One surprising observation is that they have a compound effect even when sequentially applied on the same data. To accelerate future research, we will make our 24 pre-trained miniature BERT models publicly available.

This submission revisits the student-teacher paradigm and shows through extensive experiments that pre-training a student directly on masked language modeling is better than distillation (from scratch). It also shows that the best is to combine both and distill from that pre-trained student model.
My rating is Weak Accept. I think the submission highlights a very useful observation about knowledge distillation that I imagine is overlooked by many researchers and practitioners. The decision of Weak as opposed to a Strong accept is because the submission does not introduce anything truly novel, but simply points out observations and offers a recommended training strategy. However, I do argue for its acceptance, because it does a thorough job and presents many interesting findings that can benefit the community.
Comparison with prior work:
The submission focuses on comparison with Sun et al. and Sanh. These comparisons are important, but not the most compelling part of the paper. Comparison with more prior work that show large benefits would make the paper even stronger.
Interesting experiments:
The paper presents many interesting experiments useful for anyone trying to develop a compressed model. First, it shows that distillation (from scratch) by itself may be overrated, since simply repeating the pre-training+fine-tuning procedure on the small model directly is effective. However, distillation remains relevant since it also shows that pre-training the student, then distilling against a teacher, is a potent combination. In the case when the transfer set is the same size as the pre-training set, it surprisingly still has some benefits. This is not experimentally explained, but I suspect there are optimization benefits that are hard to pin down exactly. The paper hypothesizes that the two methods learn different “linguistic aspects,” but I think it is a bit too speculative to put it in such terms.
The experiments are thorough, with many student sizes, transfer set sizes, transfer set/task set correlation, etc. It also compares against the truncation technique, where the student is initialized with a truncated version of the teacher. There are no error bars in the plots, but there are so many plots with clear trends, that this is not a big concern. I can’t think of any experiments that are obviously missing.
Misc:
- The introduction says that the pre-training+fine-tuning baseline has been overlooked. It would be great to point out papers that has actually overlooked this baseline. Including this in the results would be even better.
- During my first read-through, I got confused because I didn’t realize “pre-training” in most of the paper refers to “student pre-training” (as opposed to simply training the teacher). Making this a bit more explicit here and there can avoid this confusion.


Despite the impressive success of graph convolutional networks (GCNs) on numerous applications, training on large-scale sparse networks remains challenging. Current algorithms require large memory space for storing GCN outputs as well as all the intermediate embeddings. Besides, most of these algorithms involves either random sampling or an approximation of the adjacency matrix, which might unfortunately lose important structure information. In this paper, we propose Chordal-GCN for semi-supervised node classification. The proposed model utilizes the exact graph structure (i.e., without sampling or approximation), while requires limited memory resources compared with the original GCN. Moreover, it leverages the sparsity pattern as well as the clustering structure of the graph. The proposed model first decomposes a large-scale sparse network into several small dense subgraphs (called cliques), and constructs a clique tree. By traversing the tree, GCN training is performed clique by clique, and connections between cliques are exploited via the tree hierachy. Furthermore, we implement Chordal-GCN on large-scale datasets and demonstrate superior performance.

This paper leverages the clique tree decomposition of the graph and design a new variant of GCN which does graph convolution on each clique and penalize the inconsistent prediction made on separators of each node and its children. Experiments on citation networks and the reddit network show that the proposed method is efficient.
Overall, this paper could be a significant contribution on improving GCN, with the caveat for some clarifications on the model and experiments. Given these clarifications in an author response, I would be willing to increase the score.
Pros:
1, I like the idea of exploiting graph decomposition. In my opinion, it may not only improve the scalability but also help the model learn representations which better capture the structure or speed up the learning process. It would be great if authors could show some evidence along this line.
2, The examples in Figure 2 and 3 are very helpful in understanding the concepts related to the clique tree decomposition.
3, The summarization of time and memory complexity is very helpful in comparing different models.
4, I read the detailed questions and responses in the open review. It helps me understand more details about the experiments. Besides the typo of Table 2, I tend to believe that the experimental setup is reasonable and results are convincing although I did not run the code by myself.
Cons & Questions:
1, The main motivation of exploiting the graph decomposition is to save memory such that GCN could be applied to large scale graphs without sacrificing the structural information. However, the scale of the largest experiments is still less impressive. To strengthen the paper, it would be great to try larger graph datasets which have been used in the literature.
2, I am confused by the writing on the final prediction made by the model. In particular, do you only keep the prediction of residual or do you average the predictions on the separators? It may be interesting to explore different ways of making predictions based on this decomposition based inference. In general, it would be great to separate the writing of loss (learning) and prediction (inference).
3, Why does Chordal-GCN take significant more epochs than GCN on Reddit and less epochs on all other datasets?
Suggestion:
1, I think the clique tree is very similar if not the same with the junction tree given the node ordering (see section 2.5.2 of [1]). It would be great to discuss the relationship between your chordal graph representation and the tree decomposition used by the probabilistic inference algorithms of graphical models. From the perspective of complexity, the junction tree method and yours both highly depend on the tree-width. Also, linking to probabilistic inference could help better motivate the method since tree-based inference algorithm is shown to converge faster in the literature.
2, It would be great to discuss and or compare with [2] as it uses graph partition algorithms to get clusters and apply GNN with a propagation schedule which alternates between within-cluster and between-cluster. It is closely related to the chordal-GCN as it uses the decomposition of graph clustering directly rather than the clique tree. Decomposition like multiple overlapping spanning trees are also studied in [2].
[1] Wainwright, M.J. and Jordan, M.I., 2008. Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1(1–2), pp.1-305.
[2] Liao, R., Brockschmidt, M., Tarlow, D., Gaunt, A.L., Urtasun, R. and Zemel, R., 2018. Graph partition neural networks for semi-supervised classification. arXiv preprint arXiv:1803.06272.
======================================================================================================
Thank authors for the thorough reply! After I read authors' rebuttal and other reviewers' comments, I would like to keep my original rating. Again, I like this idea and believe better exploiting structure in the propagation could improve the inference in many ways. I hope authors could keep improving it, e.g., better motivating the proposed method (memory saving is just one angle which sometimes may need more engineering work to fully verify) and change the experiments accordingly.


Widely used recurrent units, including Long-short Term Memory (LSTM) and the Gated Recurrent Unit (GRU), perform well on natural language tasks, but their ability to learn structured representations is still questionable. Exploiting reduced Tensor Product Representations (TPRs) -distributed representations of symbolic structure in which vector-embedded symbols are bound to vector-embedded structural positions -we propose the TPRU, a simple recurrent unit that, at each time step, explicitly executes structural-role binding and unbinding operations to incorporate structural information into learning. A gradient analysis of our proposed TPRU is conducted to support our model design, and its performance on multiple datasets shows the effectiveness of our design choices. Furthermore, observations on a linguistically grounded study demonstrate the interpretability of our TPRU.

This paper proposes a novel model of recurrent unit for RNNs which is inspired from tensor product representation (TPR) introduced by Smolensky et al. in 1990. The authors claim that this allows one to better incorporate structural information into learning and easier interpretability for the learned representations. The proposed approach is motivated by a theoretical analysis showing that using TPR in this context acts as a sort of pre-conditioner and stabilizes learning. Experiments on entailment tasks (given two statement, decide whether the first implies the second) are provided to validate the approach.
I find the paper not easy to follow, with a non-negligible amount of typos in the notations and results. The advantage in terms of accuracy of the proposed approach seems marginal in the experiment, and the analysis of the interpretability of the learned representations could be improved: loosely speaking, particular examples of interpretability are given but sometimes without contexts or baselines to compare to (see the two last comments below). I know "interpretability" is a difficult property to assess but I think there may be more principled ways to showcase the approach.
I think this paper is not yet ready for publication: the proposed model is interesting and relevant but its validity could be better assessed and the paper needs some thorough proof-reading.
* Questions / Comments *
- page 1: the authors write U^TR = I, but I believe this is only possible if the TPR dimension d is bigger than the number of roles N. Is this always the case? This should be clarified.
- related to the previous point:  if U^TR = I then shouldn't U^Tb_{t-1} simply be f_{t-1} in Eq. 1?
- before Eq.1, f should be from R^d\times R^d' to R^d, not from R^d \times R^d
- In Eq. 1, b_{t-1} and x_t are not of the same dimension, so the cannot be multiplied by the same matrix U (this is why the matrices V_x and V_b are introduced later on).
- there seems to be a problem with Eq. (7): db_t/d_{b_{t-1}} appears on both sides of the equality...
- Modification 1: what is \tilde{vb_t}? I don't remember seeing this notation introduced before.
- Table 1: constants should not be included in big O notation! To compare constants, one should give the exact number of operations needed for inference.
- POS tagging: Aren't there many other reasons that could lead to this correlation (beside the informal argument that "TPR captures structured information")? Maybe the authors should compare with something else, for example the PMI between values of hidden neurons in a learned RNN and POS tags. Out of context, the numbers in Table 5 are not informative.
- Polysemy: Only a very specific cherry picked example is given here. A more principled or in depth analysis of this phenomenon is needed to make a stronger case.
* Typos *
- " The number of parameter matrices *is* the same as that of..."
- page 4 "stables" -> "stabilizes" (but rephrasing the sentence altogether would be better).
- page 5: BiDAF misses the capital letters ("bidaf").
- "dev set" -> "validation set" or "development set".
- page 8: "provides research*ers with* an intuitive..."?


The complexity of large-scale neural networks can lead to poor understanding of their internal details. We show that this opaqueness provides an opportunity for adversaries to embed unintended functionalities into the network in the form of Trojan horse attacks. Our novel framework hides the existence of a malicious network within a benign transport network. Our attack is flexible, easy to execute, and difficult to detect. We prove theoretically that the malicious network's detection is computationally infeasible and demonstrate empirically that the transport network does not compromise its disguise. Our attack exposes an important, previously unknown loophole that unveils a new direction in machine learning security.

This paper proposed a novel an very interesting attacking scenario (the authors called it the Trojan horse attack) that aims to embed a secret model for solving a secret task into a public model for solving a different public task, through the use of weight permutations, where the permutations can be considered as a key in the crypto setting. The authors prove the computational complexity (NP-completeness) of detecting such as a secret model. Experimental results show that it is possible to secretly embed multiple and different secret models into one publish model via joint training with permutation, while the performance of each model is similar to the individually trained models.
Overall, the trojan horse attacking scenario considered in this paper is novel and provides new insights to the research in adversarial machine learning. The finding that permutation along is able to embed multiple models is highly non-trivial. While I agree with the authors' explanations on the difference between trojan horse attack versus multi-task learning (shared data or not), my main concern is the lack of comparison and discussion to another secrecy-based attack scheme, the "Adversarial Reprogramming of Neural Networks" published in 2019. In their adversarial reprogramming attack, the model weights also remain unchanged (and un-permuted). To train the secret "model", Elsayed et al. used a trainable input perturbation to learn how to solve the secret task. Although Elsayed et al. did not consider the case of reprogramming for multiple secret tasks, I believe the proposed method and adversarial reprogramming share common goals and their attacks are both stealthy in the sense final model weights are unchanged. I would like to know the authors' thoughts on the proposed attack v.s. adversarial reprogramming to better motivate the importance of the considered attack scenario. In my perspective, they have the same threat model but adversarial reprogramming seems to be even stealthier as it does not use the secret data to jointly train the final model. Some discussion and numerical comparisons will be very useful for clarifying the advantage of the proposed method over adversarial reprogramming in terms of "attacks". I am happy to increase my rating if the authors address this main concern.
*** Post-rebuttal comments
I thank the authors for the clarification. I actually quite like the idea and believe the threat model is valid if addressed fin a clearer manner. I look forward to a future version of this work.
***


We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model. Assuming that both the adversary and victim model fine-tune a large pretrained language model such as BERT (Devlin et al., 2019), we show that the adversary does not need any real training data to successfully mount the attack. In fact, the attacker need not even use grammatical or semantically meaningful queries: we show that random sequences of words coupled with task-specific heuristics form effective queries for model extraction on a diverse set of NLP tasks including natural language inference and question answering. Our work thus highlights an exploit only made feasible by the shift towards transfer learning methods within the NLP community: for a query budget of a few hundred dollars, an attacker can extract a model that performs only slightly worse than the victim model. Finally, we study two defense strategies against model extraction-membership classification and API watermarking-which while successful against some adversaries can also be circumvented by more clever ones.

This paper studies the effectiveness of model extraction techniques on large pretrained language models like BERT. The core hypothesis of the paper is that using pretrained language models, and pretrained contextualized embeddings, has made it easier to reconstruct models using model stealing/extraction methods. Furthermore, this paper demonstrates that an attacker needn't have access to queries from the training set, and that using random sequences of words as a query to the "victim" model is an effective strategy. They authors also show that their model stealing strategies are very cost effective (based on Google Cloud compute cost).
The basic set up of their experiments has a fine-tuned BERT model as the victim model, and a pre-trained BERT model as a the attacker model. The attacker model is assumed to not have access to the training set distribution and the queries are randomly generated. There are 2 strategies for query generation (with additional task specific heuristics): 1) randomly selecting words from WikiText-103, and 2) randomly selecting sentences or paragraphs from WikiText-103. The victim model is passed a generated query and the attacker model is fine-tuned using the output from the victim model. Overall, this paper find that this simple strategy for query generation is effective on 4 different datasets: SST2, MNLI, SQuAD 1.1 and BooolQ. The method is also cost-effective, a few hundred dollars depending on the dataset and the number of queries used to train the attacker model.
The paper also present some analysis. They find that queries with higher agreement across victim models (5 BERTs with different random seeds) also leda to better results for the attacker model. The authors also run some experiments with humans to test the interpretability of the queries they generated. They collect annotations on SQuAD using questions that were generated with the WIKI and RANDOM strategies (they also compare highest agreement and lowest agreement queries), and also collect a control with the original SQuAD questions. While this is an interesting analysis to present, showing that most of the generated queries are nonsensical to humans and there is low inter-annotator agreement, I have an issue with the experimental procedure here: the victim model is fine-tuned on the original data, therefore it has picked up some of the data heuristics used to generate the queries, the annotators are not trained on, or shown any of the original examples (there is a control run, but these are presumably a separate set of annotators).  Through interviews, the authors learn that the annotators were using word overlap heuristics, but perhaps training the annotators on a small set of the original data would draw a closer example to the victim model. Either way, while this is an interesting result, it seems a bit misplaced in this paper. I'm not sure this human annotation experiment is contributing in any real way to the core thesis of the paper.
The authors also test the results of having a mismatch between the victim and attacker model. They consider the mismatch of BERT-base and BERT-large models. They conclude that "attackers can maximize their accuracy by fine-tuning more powerful language models, and that matching architectures is a secondary concern." This conclusion feels like a bit of a stretch. I would suggest that the authors add another few rows of experiments comparing less similar model architectures.
The paper's finding that  a model trained from scratch, QANet on SQuAD, suffers significantly without access to the training set inputs is strong supportive evidence for their hypothesis that using pretrained language models has made model extraction easier.
The authors also present a few defense strategies, membership inference, implicit membership classification, and watermarking. They also discuss the limitations of these strategies and do not claim to have solved the problem at hand.
Overall, I think this paper makes a useful  contribution to the field and I would accept this paper. While I have a couple of issues with some of the experiments (human evaluation and architecture mismatch), I think this paper is thorough and the experiments are well presented. This is the first paper, to the best of my knowledge, showing the efficacy of model extraction of large pretrained language models using rubbish/nonsensical inputs.


This paper consists of two parts. Firstly, we specify a general framework for constructing systems capable of learning distributed representations of graphs in an unsupervised manner. We show how systems such as Deep Graph Kernels, Graph2Vec and Anonymous Walk Embeddings can be formulated under this framework. In the second part of this paper, we exemplify the construction of an instance of this framework by proposing a model which extends Graph2Vec, called G2DR. This model extends Graph2Vec's current implementation to enable application on unlabelled graphs and tackles issues of diagonal dominance through pruning of the substructure vocabulary. These extensions help G2DR achieve state of the art results in downstream application of learned embeddings in graph classification tasks over datasets with small labeled graphs in binary classification to multiclass classification on large unlabelled graphs using an off-the-shelf support vector machine.

Strength:
-- The paper is well written and easy to follow
--  Learning the unsupervised graph representation learning is a very important problem
-- The proposed approach seems effective on some data sets.
Weakness:
-- The novelty of the proposed approach is very marginal
-- The experiments are very weak.
This paper studied unsupervised graph representation learning. The authors combined the techniques for Deep Graph Kernels and Graph2Vec, which essential extract substructures as words and the whole graph as documents and use doc2vec for learning the representations of both graphs and substructures. Experimental results on a few data sets prove the effectiveness of the proposed approach.
Overall, the paper is well written and easy to follow. Learning unsupervised graph representation learning is a very important problem, especially for predicting the chemical properties of molecular structures. However, the novelty of the proposed method is very marginal. Comparing to the Deep Graph kernel methods, the authors simply changed from the word2vec style methods to doc2vec style methods. The paper could be better fit to a more applied conference. Moreover,  I have some concerns on the experiments.
(1) The data sets used in this paper are too small. For unsupervised pretraining methods, much larger data sets are expected.
(2) The results in Table 1 are really weird. Why do the performance of your method have a much lower standard deviation? It is really hard to believe the proposed methods have much stable performance compare to other methods.  Can you explain this?


Performing controlled experiments on noisy data is essential in thoroughly understanding deep learning across a spectrum of noise levels. Due to the lack of suitable datasets, previous research have only examined deep learning on controlled synthetic noise, and real-world noise has never been systematically studied in a controlled setting. To this end, this paper establishes a benchmark of realworld noisy labels at 10 controlled noise levels. As real-world noise possesses unique properties, to understand the difference, we conduct a large-scale study across a variety of noise levels and types, architectures, methods, and training settings. Our study shows that: (1) Deep Neural Networks (DNNs) generalize much better on real-world noise. (2) DNNs may not learn patterns first on real-world noisy data. (3) When networks are fine-tuned, ImageNet architectures generalize well on noisy data. (4) Real-world noise appears to be less harmful, yet it is more difficult for robust DNN methods to improve. (5) Robust learning methods that work well on synthetic noise may not work as well on real-world noise, and vice versa. We hope our benchmark, as well as our findings, will facilitate deep learning research on noisy data.

===========
Summary:
This paper introduces two real-world noisy datasets collected from google search based on two existing datasets: Mini-ImageNet and Stanford Cars. The author then conducted a series of experiments comparing 6 existing noisy label learning methods in two training settings: 1) from scratch, and 2) finetuning. Parameters were tuned when necessary. Based on the results, they made a few claims that challenges some of the previous believes in this field.
===========
My major concerns:
Collection new data using google search then run existing methods has limited contribution. And the new data are related to noisy LABELS only, which does not cover any input noise such as low resolution, abnormal size, black/blank/carton background etc.
The web data searched are of moderate or rather small scales, which doesn’t seem to contribute much to the community. As mentioned in the paper, we already got some real-world datasets such Clothing-1M.
How the new datasets are collected, labeled and mixed are not clear. The numbers do not seem to add up. In text, the authors said they collected 94906/51687 images with annotations. However, in Table 1, there are 39000/8144 in datasets Red Mini-ImageNet/Red Standford Cars. The last paragraph on Page 3, how exactly the negative examples come from? How the “negative” is defined, since you got more than 1 nanotations for each image. “ Following the construction of synthetic datasets, we replace the training images in the original dataset”, refers to which original dataset? The Mini-ImageNet/Red Standford Cars or the web-searched? I don’t quite get it which dataset is creating here. Are the Red noise datasets a mixture of Mini-ImageNet/Red Standford Cars and web-search images, or are they only web-searched images.
The findings are questionable, due to the specific way how the web data are collected. Since the web datasets are created by similarity search based on a pool of seed images (5000), all the searched images are subject to the small number of seed images and the similarity algorithm of google search. Although some filtering was done to reduce test vs training duplication, the entire dataset can be treated as a duplicate of the 5000 seed images, which is roughly 10% of Mini-ImageNet -- the intrinsic noise rate is low. Having this in mind, some of the findings become no surprise at all, for example, 1) DNNs are robust to real-world noise, 2) performance does not decrease much as training progresses, 3) real-world noise is less harmful, and 4) DNN performance does not change much with different robust methods. These are all phenomena of low noise rates.


Learning from a few examples is a key characteristic of human intelligence that inspired machine learning researchers to build data efficient AI models. Recent progress has shown that few-shot learning can be improved with access to unlabelled data, known as semi-supervised few-shot learning(SS-FSL). We introduce an SS-FSL approach, dubbed as Prototypical Random Walk Networks(PRWN), built on top of Prototypical Networks (PN) (Ren et al., 2018). We develop a random walk semi-supervised loss that enables the network to learn representations that are compact and well-separated. Our work is related to the very recent development on graph-based approaches for few-shot learning. However, we show that compact and well-separated class representations can be achieved by modeling our prototypical random walk notion without needing additional graph-NN parameters or requiring a transductive setting where collective test set is provided (e.g., Kim et al. (2019)). Our model outperforms prior art in most benchmarks with significant improvements in some cases. For example, in a mini-Imagenet 5-shot classification task, we obtain 69.65% accuracy to the 64.59% state-of-theart. Our model, trained with 40% of the data as labelled, compares competitively against fully supervised prototypical networks, trained on 100% of the labels, even outperforming it in the 1-shot mini-Imagenet case with 50.89% to 49.4% accuracy. We also show that our model is resistant to distractors, unlabeled data that does not belong to any of the training classes, and hence reflecting robustness to labelled/unlabelled class distribution mismatch. We also performed a challenging discriminative power test, showing a relative improvement on top of the baseline of ≈14% on 20 classes on mini-Imagenet and ≈60% on 800 classes on Omniglot. Our code will be released upon acceptance.

This paper develops a random walk-based method on top of prototypical networks to address the semi-supervised few-shot learning, i.e. when each classification task can access only a few-shot labeled data but many unlabeled data. This paper defines a specific random walk: it walks from a prototype to an unlabeled sample, then walks for several steps between unlabeled samples, and then walk back to some prototype. They then propose two loss terms: one aims to maximize the probability of returning to the same prototype, and the other tries to make each unlabeled data having equal probability to be visited during the random walk. These two terms are added to the original prototypical network training loss and the resulted training procedure involves both the few-shot labeled data and the unlabeled data. Experiments on mini-ImageNet show that the proposed method outperforms the other baselines.
Intuitively, the main idea makes some sense to me. The writing is clear to understand the main idea but can be improved in various places. My major concerns are the motivation and analysis of the proposed random walk and the training algorithm, whose details cannot be found in the paper. The paper uses a lot of spaces to review the existing works and problem settings, but only spends one page (Section 2.2) on the description of their contribution, and the description lacks any in-depth discussion about “why it is designed in this way but not others” or “how this objective helps the few-shot learning”: it simply lists the procedures without convincing explanation.
Detailed comments:
1. It is called “random walks” but is different from the random walks used in classical methods in many key points and thus might not have all the existing properties of random walks. For example, it is not clear whether the negative of the square of Euclidean distance is a valid similarity for random walks. It is also not clear why the random walk probability matrix is computed by softmax instead of normalized Laplacian. Moreover, it fixes the starting nodes and end nodes, and also fixes the propagation steps: this cannot give any guarantee of any convergence (or asymptotic properties, mixing rate, etc) as the original random walks hold. This could be a serious problem because the random walks can stop at any non-stationary state after tau steps, which cannot be used for training.
2. It is not clear why the two additional loss terms (L_walker and L_visit) can help to improve the prototype training. The claimed motivation of minimizing these two terms cannot be soundly related to the final few-shot classification goal, though each of them independently makes some sense if applied to a different goal. Why does maximizing the probability of a prototype transit to itself give new information of improving few-shot classification? If prototypes are involved in the random walk nodes, why do we need to walk back to itself through many unlabeled nodes rather than simply walk to itself (which has the largest probability)? Why do we need to visit as many unlabeled data as possible? Isn’t the case that we hope to only visit the unlabeled data mostly related to the prototype and its class and rule out other unrelated ones since they introduce noises?
Here is a counterexample, which is a trivial solution that can minimize the two terms but does make any sense here: it is easy to define a model that produces the same prototype for different classes (so L_walker is minimized), and achieve equal distance between all the unlabeled samples (so L_visit is minimized), but this does not help the prototype training at all.
3. Another primary flaw here is that the training objective and the computational graph during training always change with the random walks, since the trajectory of the random walks keeps changing due to the randomness. Hence, the trained model does not have any guarantee and the convergence in the training is not the real convergence (it is not well-defined in the first place). If the random walks guarantee to converge to some stationary distribution, it might be possible to treat the varying computational graph as a stochastic one which has been analyzed in previous works. Unfortunately, due to the fixed number of steps and other issues mentioned above, this cannot hold. In Figure 2(a), we can see that even the largest tau they have tried cannot make the landing probability converge (not talking about the stationary distribution over all the nodes), which verifies my worries above.
4. Is the final performance sensitive to the hyperparameters lambda and tau? How did you choose their values in the experiments?
5. The methodology behind most existing semi-supervised few-shot learning methods, as also indicated in this paper’s introduction, is to apply or extend existing semi-supervised learning techniques on top of existing few-shot learning models. In fact, it is quite straightforward to do so and test all the semi-supervised learning techniques on few-shot learning tasks. Hence, I think it is necessary to compare to baselines that directly use, for example, consistency loss from mean teacher method and progressive training on pseudo labeling, or other SSL techniques mentioned in this paper: https://papers.nips.cc/paper/7585-realistic-evaluation-of-deep-semi-supervised-learning-algorithms.
-------------
Update:
Thanks for the authors' reply! After reading the rebuttal, I still think there are several main issues needed to be clarified or needing further study. Hence, I will keep my rating.


Deep generative modeling using flows has gained popularity owing to the tractable exact log-likelihood estimation with efficient training and synthesis process. However, flow models suffer from the challenge of having high dimensional latent space, same in dimension as the input space. An effective solution to the above challenge as proposed by Dinh et al. (2016) is a multi-scale architecture, which is based on iterative early factorization of a part of the total dimensions at regular intervals. Prior works on generative flows involving a multi-scale architecture perform the dimension factorization based on a static masking. We propose a novel multi-scale architecture that performs data dependent factorization to decide which dimensions should pass through more flow layers. To facilitate the same, we introduce a heuristic based on the contribution of each dimension to the total log-likelihood which encodes the importance of the dimensions. Our proposed heuristic is readily obtained as part of the flow training process, enabling versatile implementation of our likelihood contribution based multi-scale architecture for generic flow models. We present such implementations for several state-ofthe-art flow models and demonstrate improvements in log-likelihood score and sampling quality on standard image benchmarks. We also conduct ablation studies to compare proposed method with other options for dimension factorization.

Summary:
The paper investigates a strategy for factoring out dimensions in multi-scale generative flow models. The strategy relies on a heuristic that employs the availability of per-dimension log-determinant terms (e.g. as affine couplings) to decide which half of the dimensions to factor out. Quantitative experimental results support the proposed strategy’s efficacy.
Recommendation: Weak Reject
The paper misses several important comparisons and baselines - both in terms of placing the proposed method in a broader context (e.g. how could the method be used in multiscale i-ResNet), as well as providing experimental comparisons that would definitively demonstrate the method’s contribution (e.g. is it the heuristic dimension splitting or the unfactored pre-training that provides the benefit.
The applicability of the proposed method appears to be overstated. The paper suggests that the method is applicable to any multi-scale flow architecture. However, it depends on the availability of per-dimension log-determinants. As the paper states, these values are readily available for methods that rely on affine couplings (e.g. RealNVP, Glow), but not for methods such as i-ResNets or FFJORD, that also employ multi-scale architectures to factor out variables.
Major comments:
1. Section 2.1 introduces flows as a mapping from data x to the latent code z, and gives the log-likelihood formula (2) that can be optimized during model training. This presentation is at odds with Section 3, where the authors talk about sampling a latent variable that fixes the “log-latent density” and only leaves the log-determinant term to be optimized (e.g. “[...] the log-latent-density term depends on the choice of the latent distribution and if fixed given the sampled latent variable. [...] once the variable is sampled, maximizing the log-det term results in maximized likelihood.” The two presentations are at odds with each other and should be reconciled, ideally by altering Section 3 since it does not reflect the way flow models are trained.
2. The authors made an implicit assumption that the log-det term is decomposable into per-dimension component. As mentioned earlier in this review, generally speaking this is not true and methods such as i-ResNets or FFJORD do not (directly?) permit such decomposition, while they do employ multi-scale architectures akin to that of RealNVP. This assumption manifests itself in several places in the paper, which are listed below. Either the authors could make this assumption and the limitations imposed by it very clear, or they could provide additional information on how their method extends to methods that do not provide per-dimension log-det contributions.
a) In Section 3 “The total log-det term is nothing by the sum of log-det terms for contributed by each dimension”. This is obvious (and possibly holds) only for affine couplings.
b) In Section 3 “Nevertheless, such an analogy can be extended for other flow models which involve a multiscale architecture [...]”. As mentioned, I am not convinced this is true.
c) In Section 4 “Since our method focuses individually on the dimensions using a heuristic which is always available in flow model training, it can prove to be have more versatility in being compatible with generic multi-scale architecture”. Same as above - in fact I believe that the opposite may be true.
d) Section 4 does not mention methods that employ multi-scale architectures whil not allowing for discerning per-dimension contributions of the log-det. Mentioning these methods is important for accurately positioning the authors contribution relative to the existing literature.
e) The method is only applied to the RealNVP architecture. The claim of it being generally applicable to multi-scale architectures would be much stronger if the comparison would include other multi-scale architectures. Here, I strongly recommend i-ResNets or a similar architecture based on log-det approximation to the entire coupling layer.
3. Unless I missed it, the authors do not mention how they aggregate the log-det values. They are available per datapoint. Are they aggregated over the entire dataset?
4. In Section 4 the authors imply that the contribution of a dimension towards the total likelihood is dominated by the log determinant (“[...] as the contribution by the variable (dimension) towards the total log-det (~ total log-likelihood)”). This is not obvious, and I am not sure it is always true - it probably depends on the choice of the base distribution p(z).
5. The authors show experimentally that the use of their variable factorisation heuristic improves log-likelihoods over the baseline RealNVP model; they further perform ablation studies to confirm that preferential factorisation of low log-det dimensions provides best log-likelihoods and qualitative results on the CelebA dataset. However, several points remain unclear.
a) What are the relative contributions of pre-training without dimension factorisation and the proposed factorisation heuristic? Do the experiments shown in Table 2 all employ the pre-training scheme? If so, this should be clearly stated. If not, this set of experiments should be performed.
b) The networks used in the experiments use between m=1 and 4 factorisation layers. So in total, there are 2^m possible decisions of which half of the variables at those layers to factorise. These numbers are not so large as to prevent us from training all possible combinations of the networks. How does the proposed heuristic compare to just training all (or a few sampled) factorisations decided a priori? This comparison should also investigate the effect of using pre-training.
6. In the qualitative comparison (Section 5.2) the authors argue two points (i) RealNVP with the proposed method generates finer details on CelebA; and (ii) provides a reasonable latent space that can be interpolated in.
a) With regards to (i), I personally find it difficult to argue about fine details in what are effectively low-resolution images. The differences (which are already hard to see and easy to imagine) could be present due to minute changes in the training code/procedure/random seed/etc. I would be more confident about these differences if the authors could confirm that the methods were trained using the same code base, random seeds, dataset shuffle and framework versions.
b) How do the qualitative interpolation results compare to RealNVP without the proposed factorisation heuristic?
Minor comments:
- In Section 2.1 “Let x be a high-dimensional random vector with unknown true distribution x ~ p(x) [...]”. Here x appears to refer to the data or a sample from the data distribution, and not just a random vector. This could be made clearer.
- In Section 2.1 the “jacobian” should be written with a capital “J”
- In Section 2.1 and elsewhere, when denoting outputs of intermediate flows f_i as y_i, it is mentioned that y_0 = x, but not that y_k = z. If the latter is true, mentioning this would improve clarity.
- Section 3 “In a multi-scale architecture, it is apparent that the variables getting exposed to more layers of flow will be more expressive in nature [...]”. I do not find this to be an obvious fact. Perhaps the authors could provide an intuition for why this is true or demonstrate this on a toy example? My confusion may stem from the fact that I am unsure what it means for a variable (i.e. a dimension we factor out) to be more expressive.
- Some phrasing is a little unusual (e.g. “less (more) log-det tems” instead of “smaller (larger) log-det term”; “to be have”).
- In Section 5.1 “code length” is introduced without context. The connection between code length, log-likelihood and compression can be made more clear.
------------------------------------------------
Post rebuttal update
------------------------------------------------
I’d like to thank the authors for additional elaboration on their baseline experiments that consider their heuristic-based scheme for factoring out dimensions (the low-low-low scheme) and the “anti-heuristic” (which considers the high-high-high scheme). I agree with the authors that both schemes would have been within the 2^m possibilities I encourage them to consider, however I do not fully agree with some of the conclusions the authors draw for these experiments.
First, given that the method proposed by the authors is a heuristic, it is not obvious that the high-high-high and the low-low-low schemes indeed correspond the worst and best possible of the 2^m schemes. Running the 2^m experiments could demonstrate this empirically and strengthen the manuscript.
Second, if we speak about the high-high-high and low-low-low factorizations, then we’ve already gone through the computationally expensive effort of pre-training a model that does not factor out any variables. It is precisely because of this that I’d like to see a baseline that doesn’t do any pre-training, but simply enumerates the 2^m schemes and selects the one with the best validation score. It is not obvious that this approach would computationally more expensive than the pre-training step -- another point I believe is important to cover.
To summarize, I do not believe that the baselines currently present in the paper are a substitute for the ones requested.
Including new results requires a review of those new results, so I would not be in favour of adding new results at the camera ready stage. I therefore retain my score.


Generative adversarial networks have been very successful in generative modeling, however they remain relatively challenging to train compared to standard deep neural networks. In this paper, we propose new visualization techniques for the optimization landscapes of GANs that enable us to study the game vector field resulting from the concatenation of the gradient of both players. Using these visualization techniques we try to bridge the gap between theory and practice by showing empirically that the training of GANs exhibits significant rotations around LSSP, similar to the one predicted by theory on toy examples. Moreover, we provide empirical evidence that GAN training seems to converge to a stable stationary point which is a saddle point for the generator loss, not a minimum, while still achieving excellent performance.

Summary:
This paper proposes visualization techniques for the optimization landscape in GANs. The primary tool presented in this paper is a quantity called path-angle, which looks at the angle between the game vector field and the linear path between a point away from a stationary point and a point near a stationary point. The paper present examples of the visualization for dynamics with pure attraction, pure rotation, and a mix of attraction and rotation. Along with this, the authors propose to look at the eigenvalues of the game Jacobian and the individual player Hessian’s to evaluate convergence in GANs. The paper presents application of the tools on GANs trained with NSGAN and WGAN-GP objectives on a mixture of Gaussians, MNIST, and CIFAR10. The primary observation is that the generator performance is good, but the algorithms converge to non-Nash stable attractors. Moreover, it is shown using the path-angle plots that GANs exhibit rotational behavior around stable points.
Review:
There has been a lot of work in the past few years (and ongoing) on principled training approaches for GANs. The objective of the algorithms is typically to converge a differential Nash equilibrium and/or to reach a stable point of the dynamics quickly. In my view, this work fills some of the gap on the empirical side of things with respect to each goal.
Notably, a main idea to speed up convergence in GANs is to change the gradient play dynamics so rotational components are neutralized. The path angle visualization provides a novel tool to evaluate the empirical ability of any dynamics proposed for GANs to cancel out rotational components. Since it is generally known that gradient play dynamics are susceptible to cycling, I would have been interesting in seeing the path angle plots for some recently proposed algorithms such as consensus, symplectic gradient adjustment, stable opponent shaping, local symplectic surgery, etc to see how they compare. This would have made the experiments using the path angle visualization stronger in my view. Nonetheless, the path angle tool is useful and I can foresee it being commonly used in the future.
Aside from neutralizing rotational components, dynamics have been proposed with the goal of avoiding non-Nash stable attractors and converging only to differential Nash equilibria. However, to my knowledge, there has not been much, if any, evaluation in GANs to see if the methods are in fact converging to Nash equilibria as theory may predict. While simple, I found it interesting to evaluate the eigenvalues of the relevant quantities at convergence. I am curious why the authors evaluate the top-k eigenvalues in terms of magnitude? The scipy package referenced in the appendix can compute the largest and smallest real eigenvalues, which is what it seems like you would want to evaluate the definiteness of the game Jacobian and the individual player Hessians. The most interesting empirical result in the paper to me was that it is common to converge to non-Nash stable attractors using standard training techniques and at such stable points the generator performance is strong. This is an important observation and  may cause some consideration of what points should be sought in GANs. I am not fully convinced this is always what the dynamics would always converge to depending on the network, learning rates, optimization methods, etc, but showing that it can be the case is useful.
Overall, I think this paper introduces some useful tools to interpret the performance in GANs and to help understand the behavior of training dynamics. The main tool introduced was the path angle visualization and the primary empirical result was that standard GAN methods may reach non-Nash stable attractors and perform well. The paper probably be condensed in the first 4 pages, so that more experimental results could be presented and this would make the paper stronger.
Post Response: Thanks for the response. I believe this paper should be accepted.


Convolutional neural networks (CNNs) have often been treated as "black-box" and successfully used in a range of tasks. However, CNNs still suffer from the problem of filter ambiguity -an intricate many-to-many correspondence relationship between filters and features, which undermines the models' interpretability. To interpret CNNs, most existing works attempt to interpret a pre-trained model, while neglecting to reduce the filter ambiguity hidden behind. To this end, we propose a simple but effective strategy for training interpretable CNNs. Specifically, we propose a novel Label Sensitive Gate (LSG) structure to enable the model to learn disentangled filters in a supervised manner, in which redundant channels experience a periodical shutdown as flowing through a learnable gate varying with input labels. To reduce redundant filters during training, LSG is constrained with a sparsity regularization. In this way, such training strategy imposes each filter's attention to just one or few classes, namely class-specific. Extensive experiments demonstrate the fabulous performance of our method in generating sparse and highly class-related representation of the input. Moreover, comparing to the standard training strategy, our model displays less redundancy and stronger interpretability.

Contributions: The paper proposes a novel Label Sensitive Gate (LSG) structure to enable the model to learn disentangled filters in a supervised manner. The novelty of the paper is to introduce the Label-Sensitive Gate path during the training, on top of the standard training path. This encourages the filters to extract class-sensitive features.
The highlight of the experimental results is that by incorporating the LSG structure, the author is able to achieve better localization with CAM because the filters are trained to extract more relevant features. Additionally, filters from different classes are kind of "orthogonal" to each other.
However, I have a few questions for the author:
a) In Figure 5, the author mentions "TP/FN" samples in the caption. However, for the plot at the right, it is titled with "False Positive". Is this a typo or do you talk about something else?
b) In Figure 7, the author shows the correlation matrix of filters. Since the filters belonging to one class are highly correlated (block matrices in plot d), why do not train even less filters for each class? If we train less filters for each class, will that hurt the performance? It might be a good direction on how to interpret filters from one class and how to make intra-class filters more interpretable.
Overall, the novelty of the paper is limited by the scope of the experimental results. Why would people care about interpretability of CNN filters is not explained clearly. The paper shows interpretable filters could help with localization. However, this is the only example in the paper I find useful. It would be better if the author could show any other applications. Moreover, I wish the author could spend more paragraph discussing why filters belonging to different classes tend to be orthogonal to each other. This is not clearly written (Eq(3)?). A more theoretical/intuitive explanation would also make the paper stronger as well.
If the paper could omtroduce more real applications of the proposed LSG model and give more theoretical/intuitive explanations of why the inter-class filters are orthogonal, I would be willing to raise up my scores.


Generative adversarial networks (GANs) have achieved great success at generating realistic samples. However, achieving disentangled and controllable generation still remains challenging for GANs, especially in the high-resolution image domain. Motivated by this, we introduce AC-StyleGAN, a combination of AC-GAN and StyleGAN, for demonstrating that very limited supervision is sufficient to benefit the generic disentanglement learning of high-resolution images. Specifically, by only using 1% of the labelled data, the disentanglement quality is very close to the fully supervised case and significantly outperforms the unsupervised alternative. Inspired by the observed separation of fine and coarse styles in Style-GAN, we then extend AC-StyleGAN to a new image-to-image model called FCStyleGAN for semantic manipulation of fine-grained factors in a high-resolution image. In experiments, we show that FC-StyleGAN performs well in only controlling fine-grained factors, with the use of instance normalization, and also demonstrate its good generalization ability to unseen images. Finally, we create two new datasets -Falcor3D and Isaac3D with higher resolution, more photorealism, and richer variation, as compared to existing disentanglement datasets.

Summary:
The paper is reasonably clear (though see some of my detailed comments on writing below).
The idea seems well-motivated and somewhat new though not revolutionary,
and the new data sets are nice (though see comments below about how I'm not qualified to evaluate them),
but I don't understand why the proposed algorithm wasn't evaluated on existing data sets as well,
and I don't understand why it wasn't compared against other algorithms that purport to do the same thing.
Barring certain exceptions (again see detailed comments), it seems like those two things are things
we ask of essentially all machine learning paper submissions, and I don't see why this paper is different?
I am open to being persuaded, but as of now I can't recommend acceptance.
Detailed Comments:
> More importantly, only using 5% of the labelled data significantly improves the disentanglement quality.
Hard to parse.
Does this mean using only 5% of data is better for disentangling than using 100%?
>  Generative adversarial networks (GANs) (Goodfellow et al., 2014) have achieved great success at generating realistic images, such as StyleGAN
Strange sentence - styleGAN is not itself a realistic image.
>  the controllable generation of high-resolution images is possible
Nit: i would cut the 'the'
> which characterizes how significant that the model can change a factor.
What does this mean?
> present a state-ofthe-art challenge
What does it really mean for a challenge to be state of the art?
> that enables conditional generation of high-fidelity images
Pretty minor nit, but I don't think it makes sense to refer to samples themselves as high fidelity.
High fidelity to what?
It makes a little more sense to refer to a trained generator as having high fidelity to the training data set,
but TBH I still don't even like the phrase in that context.
Fig 1 is good.
I felt like I could understand what's going on mostly from looking at the figure, which is nice.
The section surrounding eqs 1 and 2 is a little hard to read for me.
Lots of single letter variable names and it's hard to keep them all in my head when I read the equation.
I'd replace e.g. c_r with \text{code}_r and so forth.
> Note that AC-StyleGAN reduces to an InfoGAN variant in the special case
This is helpful.
> AC-StyelGAN
>  FCStyleAGN
> by symmetry
I don't follow this part.
> while its high-resolution blocks accounts for fine styles
I feel like it's worth making a distinction between high frequency bits of an image and non-essential (or nuisance variables or whatever) bits.
They often are the same, but not always (it really just depends how close up your photo is, right?) but this technique is really separating high-freq from low-freq, IIUC.
>  handling complex high-fidelity images
Again, I just don't feel like this phrase makes any sense.
Re eq 3:
I guess you can call this the 'interpolation variance'
if you want, but it's not really a new thing.
You're just getting discretized measurements of the Jacobian of the mapping from code to predicted code, and I don't really think in a way that fits with the
intuition you describe.
There are a few things I can think of that are weird about this measurement, but here's just one:
I might have a dimension of the code that causes one really peaky change at one point, and then the change quickly reverts.
If i measure the variance of the output, I won't see much (in fact, depending on how you discretize, maybe I'll miss it altogether),
even though the fact that the change is peaky doesn't say anything about the 'importance' of the change.
I think you want something more like a line integral.
It's possible I'm misunderstanding something about the description of this, however.
Re: the experiments:
I like the new data sets, although I'm not familiar enough w/ the robotics literature to know whether they add something really new.
What I don't understand is why there's not really any attempt to do either of:
a) compare the models from the paper to existing models
b) evaluate the models from the paper on existing data-sets?
I understand that sometimes one comes up with a thing that does something totally new,
and then reviewers will (maybe unjustifiably) still insist that the some kind of comparison be made even though it doesn't make sense,
but that doesn't seem to be what has happened here?
This paper proposes a new algorithm for synthesizing images and controlling various attributes of the images, but this has certainly been done in the past,
so why can't your technique be compared to those prior techniques?
Similarly, there are lots of data sets that people are already familiar with that you could have evaluated these models on, right?


Owing to the susceptibility of deep learning systems to adversarial attacks, there has been a great deal of work in developing (both empirically and certifiably) robust classifiers, but the vast majority has defended against single types of attacks. Recent work has looked at defending against multiple attacks, specifically on the MNIST dataset, yet this approach used a relatively complex architecture, claiming that standard adversarial training can not apply because it "overfits" to a particular norm. In this work, we show that it is indeed possible to adversarially train a robust model against a union of norm-bounded attacks, by using a natural generalization of the standard PGD-based procedure for adversarial training to multiple threat models. With this approach, we are able to train standard architectures which are robust against ∞ , 2 , and 1 attacks, outperforming past approaches on the MNIST dataset and providing the first CIFAR10 network trained to be simultaneously robust against ( ∞ , 2 , 1 ) threat models, which achieves adversarial accuracy of 46.1% against the union of ( ∞ , 2 , 1 ) perturbations with radius = (0.03, 0.5, 12).

This paper adversarially trains models against l_p norms where p is of there different values. They then propose a method which does somewhat better than the obvious way of adversarially training against more than one l_p perturbation.
The motivation for the paper is limited, in that they suggest previous works have suggested adversarial training itself "overfits" to the given l_p norm. This isn't surprising that it works, since the straightforward baseline works. They make it seem surprising by suggesting that ABS suggested adversarial training is doomed and cannot provide robustness to l_1, l_2, l_\infty norms simultaneously. The other motivation is that this is a step toward studying an expanded threat model, but the authors have not demonstrated that the learned representations are any bit more robust to common corruptions (could the authors show the generalization performance on CIFAR-10-C or generalization to unforeseen corruptions?). Without further evidence, we are left to believe this only helps for this narrow threat model. Overall the paper is deficient in creativity and generality, so I vote for rejection.
Small comments:
> take more time than a single norm,  it is a step closer towards the end goal of truly robust models, with adversarial robustness against all perturbations.
Please show model performance on CIFAR-10-C since if the model is more robust, it should hopefully be more robust to stochastic adversaries.
> has claimed that adversarial training “overfits” to the particular type of perturbation used to generate the adversarial examples
Wouldn't this be that l_\infty training fits specifically to l_\infty examples, not that robust optimization cannot handle more than one norm at a time? Who is claiming that?
> First, we show that even simple aggregations of different adversarial attacks can achieve competitive universal robustness against multiple perturbations models without resorting to complex architectures.
I am not sure this was in doubt. The phrase "universal robustness" is misleading.
How were the budgets chosen for l_2 and l_1? Those values seem small.


Learning to represent videos is a very challenging task both algorithmically and computationally. Standard video CNN architectures have been designed by directly extending architectures devised for image understanding to include the time dimension, using modules such as 3D convolutions, or by using two-stream design to capture both appearance and motion in videos. We interpret a video CNN as a collection of multi-stream convolutional blocks connected to each other, and propose the approach of automatically finding neural architectures with better connectivity and spatio-temporal interactions for video understanding. This is done by evolving a population of overly-connected architectures guided by connection weight learning. Architectures combining representations that abstract different input types (i.e., RGB and optical flow) at multiple temporal resolutions are searched for, allowing different types or sources of information to interact with each other. Our method, referred to as AssembleNet, outperforms prior approaches on public video datasets, in some cases by a great margin.

Summary:
This paper aims to adapt the standard neural architecture search scheme to search a two-input convolutional neural network for video representations. To this end, the paper formulates a direct acyclic graph with two input nodes (for RGB image and optical flow), where each node represents some pre-composed layers and edge represents the data flow with a trainable weight. The searching policy is a modified evolutionary algorithm, which is guided by the trainable weights on the edge, and a set of graph limitations are in-place to avoid over-complicated graphs. The best-selected model outperforms previous baselines and achieves a new state-of-the-art on two video datasets.
Overall, this paper presents a concrete application of neural architecture search for video CNN with interesting results. Edge-weight guided evolutionary algorithm also demonstrates a small improvement in the ablation study. My concern, as detailed later, is if the comparison only with previously human-designed models is necessary. Nevertheless, this paper presents an interesting application of NAS and discover a feasible way to conduct an evolutionary algorithm within a reasonable cost (less than 100 sampled architectures).
Strength:
+ Writing in good shape, easy to follow and understand.
+ Motivation is clear and timely, reformulate neural architecture search for video representation is novel.
+ Clear experimental settings and reasonable convincing results.
Weakness:
- Lack of comparison with previous neural architecture search algorithms
Although the results yield that the proposed new search space is meaningful, considering each model has a similar dimension comparing to ResNet-50, it is still unknown if only comparing to human-designed model is a truly fair baseline. In my perspective, since this paper is built on top of NAS strategy with minor adaptation, could the author add one comparison experiment that, the proposed new search space is superior to those previous NAS spaces? For example, one could based on the earlier two-stream ResNet-50 with RGB+F modality, switch the backbone model into a NAS-based one and search with the same evolutionary algorithm (removing the edge weights adaptation). Otherwise, the improvement shown in the paper is not that surprising.


Training deep neural networks (DNNs) has achieved great success in recent years. Modern DNN trainings utilize various types of training techniques that are developed in different aspects, e.g., activation functions for neurons, batch normalization for hidden layers, skip connections for network architecture and stochastic algorithms for optimization. Despite the effectiveness of these techniques, it is still mysterious how they help accelerate DNN trainings in practice. In this paper, we propose an optimization principle that is parameterized by γ > 0 for stochastic algorithms in nonconvex and over-parameterized optimization. The principle guarantees the convergence of stochastic algorithms to a global minimum with a monotonically diminishing parameter distance to the minimizer and leads to a O(1/γK) sub-linear convergence rate, where K is the number of iterations.
Through extensive experiments, we show that DNN trainings consistently obey the γ-optimization principle and its theoretical implications. In particular, we observe that the trainings that apply the training techniques achieve accelerated convergence and obey the principle with a large γ, which is consistent with the O(1/γK) convergence rate result under the optimization principle. We think the γ-optimization principle captures and quantifies the impacts of various DNN training techniques and can be of independent interest from a theoretical perspective.

This paper proposes a "gamma principle" for stochastic updates in deep neural network optimization. First, the authors propose if Eq. (2) is satisfied then convergence is guaranteed (Thm. 1). Second, they use experimental results of Alexnet and Resnet-18 on cifar10/100 to show that Eq. (2) is satisfied by SGD, SGD with momentum, and Adam, with different activations and tricks like batch normalization, skip connection.
Pros:
1. This paper is well written and the presentation is clear.
2. The experiments are extensive.
Cons:
1. Before Eq. (2), it is assumed that over-parameterized NNs are used as \theta. But there is no quantization how many parameters are enough? Are the Alexnet/Resnet-18 in experiments enough over-parameterized and how can we tell that? Some quantitative conditions should be provided to show what kind of models this principle hold.
2. The connection between Eq. (2) and Thm. 1 is too obvious and the gamma is just to characterize the progress of each update. Of course, large progress corresponds to fast convergence. Eq. (2) is a strong assumption rather than a theoretical contribution.
3. Experiments are used to show that Eq. (2) is a "principle". However, the experiments are problematic as follows.
First, "we set \theta^* to be the network parameters produced in the last training iteration", then how do we make sure \theta in the last training iteration is \theta^*, even if the loss is close to (but not exactly) zero? For this point, I suggest using a teacher-student setting, where the optimal \theta^* is already known.
Second, using \theta in the last training iteration makes the experiments show the following simple fact, that methods/tricks/activations with faster convergence to certain parameter will have larger every update progress to that parameter, which is, of course, true and does not reveal an optimization principle of deep learning.
Third, it is difficult to claim that this is a principle for general deep learning by using experiments on two datasets.
Overall, I found the theory not inspiring and experiments not convincing.
========Update=========
Thanks for the rebuttal.
I have read it and using teacher-student setting is an improvement to resolve my question with respect to \theta^*.
However I would maintain my rating since
1) the theoretical contribution is actually marginal;
2) the argument that this "gamma principle" holds for over-parameterized NNs is vague in the sense (and the author did not resolve my concern of this) that for what kinds of over-parameterized NNs this would work and for what kinds of NNs it does not hold. In particular, mentioning other theory work of over-parameterized NNs is not enough, because usually in these work, the numbers of parameters in NNs are poly(n), like O(n^4), O(n^6), where n is number of training data. There is an obvious gap between poly(n) and the number of parameters in experiments. From this perspective, the experiments cannot verify the claim that this "principle" holds for over-parameterized NNs that mentioned by authors in the rebuttal.
3) experiments on two datasets are not enough to claim this is a general "principle".
Considering this claim is for general over-parameterized NN optimization, I think it lacks of specifying types of NNs for which this claim would hold (of course it cannot hold for any NNs, but it possibly can hold for some NNs, and what are these NNs?), and experiments are not enough to show the generality of this claim.