ROBUST CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS
Conditional generative adversarial networks (cGAN) have led to large improvements in the task of conditional image generation, which lies at the heart of computer vision. The major focus so far has been on performance improvement, while there has been little effort in making cGAN more robust to noise. The regression (of the generator) might lead to arbitrarily large errors in the output, which makes cGAN unreliable for real-world applications. In this work, we introduce a novel conditional GAN model, called RoCGAN, which leverages structure in the target space of the model to address the issue. Our model augments the generator with an unsupervised pathway, which promotes the outputs of the generator to span the target manifold even in the presence of intense noise. We prove that RoCGAN share similar theoretical properties as GAN and experimentally verify that our model outperforms existing state-of-the-art cGAN architectures by a large margin in a variety of domains including images from natural scenes and faces.

General:
In general, this is a well-written paper. This work focuses on the robustness of conditional GAN(RoC-GAN) when facing the noise. The authors claim the generator of RoC-Gan will span the target manifold, even in the presence of large amounts of noise. The main contribution of the paper is to introduce a two-pathway model, where one of them is used to perform regression as ordinary GAN while the other one helps the whole model span the target domain.
Strength:
1. The idea is simple and straightforward. The authors provide necessary theoretical analysis and empirical validation for their model.
2. The proposed method seems technically correct to me. i.e. Although I am not very sure how well it works in practice, the idea is fine.
Possible Improvements:
1. I agree adding another auto-encoder as a helper may give better generation results by spanning the whole target space, but I don't think this constraint is strong enough in practice.
2. In section 3.3, the time complexity of computing 'L_deconv' seems extremely large. From the perspective of numerical optimization, optimizing such a matrix will cause trouble if the dimension of weight matrices are large. i.e. optimizing the high-dimensional covariance matrix seems a problem to me.
3. The experiments looks good. The experiments could be more convincing if using more complex data sets(e.g. CIFAR10, ImageNet) besides CelebA. My concern for using such data sets(the resolution of images is low and the distribution is simple)  is that: although the noise seems to corrupt most of the image, the distribution of the image is not complex, so the generative model can recover it easily. Since this is a more empirical paper, the experiments should be more convincing.
Conclusion:
The author(s) are thoughtful and they put lots of work on this paper. The proposed method is simple. For novelty and significance, I think the idea is not very fancy to me. I am not very convinced by the method proposed in the paper. Although the paper demonstrates the robustness of their model with different experiments, most of them were not performed on deep neural networks and complicated data sets. As a conclusion, I vote for weak rejection.
Minor suggestion:
Increase the resolution of the figures.
------------------------------- After Rebuttal ---------------------------------
I am very satisfied with the authors' response, so I will change my vote from rejection to acceptance.


CapsNet (Capsule Network) was first proposed by Sabour et al. (2017) and later another version of CapsNet was proposed by Hinton et al. (2018). CapsNet has been proved effective in modeling spatial features with much fewer parameters. However, the routing procedures (dynamic routing and EM routing) in both papers are not well incorporated into the whole training process, and the optimal number for the routing procedure has to be found manually. We propose Generalized GapsNet (G-CapsNet) to overcome this disadvantages by incorporating the routing procedure into the optimization. We implement two versions of GCapsNet (fully-connected and convolutional) on CAFFE (Jia et al. (2014)) and evaluate them by testing the accuracy on MNIST & CIFAR10, the robustness to white-box & black-box attack, and the generalization ability on GAN-generated synthetic images. We also explore the scalability of G-CapsNet by constructing a relatively deep G-CapsNet. The experiment shows that G-CapsNet has good generalization ability and scalability.

The paper deals with the idea to generalize the CapsNet architecture from Sabour. Under generalization the authors mean, to define a routing procedure without an iteration parameter.
In general, your paper has a good length, is well explained and good organized. To be honest, I don’t like your writing style. It seems to be a bit too casual and not formal enough for a scientific work. Additionally, note that:
-	You are not staring with Fig. 1 in the introduction…the counting should start by one.
-	AC-GAN is the abbreviation for?
-	Eq. 2 is outside the page space.
I have several concerns about the contribution. My major concern is that it isn’t new in general. If I break down your method, it is just the basic Dynamic Routing procedure with:
-	the number of iterations defined to be one;
-	trainable initial routing coefficients;
-	no softmax normalization over routing coefficients.
The usage of trainable initial routing coefficients was already mentioned by Sabour. Thus, the only thing which is new in your method is that you skip the normalization and I’m not sure that this has a positive effect on the process.
Minor concerns/minor mistakes:
1.	You mentioned that the code is public available. Where is the link to a respective repository?
2.	Page 1: “[…] so it makes sense to believe that CapsNet has a better generalization ability.” Compared to what?
3.	Page 2: “The routing iterations is a meta-parameter that needs to be set manually which limits the scalability of CapsNet […].” Why it should limit the scalability? It has no effect on the model size, etc. It’s just a parameter which has to be defined.
4.	Page 3: Are you sure that a linear transformation of a hyper-cube is defined in that way in general?
5.	Page 4: What is T_k?
6.	Page 7: “Hinton et al. (2018) claimed that CapsNets […]” Are you aware that Hinton worked on Matrix Capsules and not on the CapsNet architecture of Sabour?
7.	Page 8: How you can guarantee the convergence of your method? Moreover, the convergence to what?
8.	Could you add some histograms plots of your c_ij values after the training?
9.	Why are your performance values so bad compared to CapsNet and Matrix Capsules?
10.	Could you add to your tables the inference, training times? If you remove the iteration parameter I would assume that your method should be faster, or?
11.	Is the parameter lambda in Eq. 2 the same as in Eq. 6? How you tune that parameter?


In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching through the entire output space to find the best output with respect to this reward function is typically intractable. In this paper, we instead use efficient truncated randomized search in this reward function to train structured prediction energy networks (SPENs), which provide efficient test-time inference using gradient-based search on a smooth, learned representation of the score landscape, and have previously yielded state-of-the-art results in structured prediction. In particular, this truncated randomized search in the reward function yields previously unknown local improvements, providing effective supervision to SPENs, avoiding their traditional need for labeled training data.

# Summary
This paper proposes search-guided training for structured prediction energy networks (SPENs). SPENs are structured predictors that learn an input-dependent, non-linear energy function that scores candidate output structures. Many methods have recently been proposed for training SPENs. One in particular, rank-based training, has the advantage of supporting training from weak supervision in the form of a reward function. By performing gradient descent on this reward function, rank-based training generates output, improved output pairs that become margin-based constraints on the learning objective. Each constraint specifies a pair of outputs for a given input, and penalizes the current weights if the improved output is not scored higher than the other output by a certain margin.
This paper addresses a limitation of rank-based training, that this gradient descent procedure for finding output pairs may get stuck in plateaus. In search-guided training, truncated randomized searches are performed starting at an initial output to find an improved output. The paper says that the random search procedure is informed by the reward function, but it is not specific. Are steps in the search space performed uniformly at random? The paper only says that the returned improved example must score higher in the reward function by some margin \delta that is "based on the features of the reward function (range, plateaus, jumps)" but it is not discussed how to identify these features of the reward function or how to set \delta accordingly.
Experiments are conducted multi-label classification, citation field extraction, and shape parsing. On multi-label classification search-guided SPENs (SG-SPENs) outperform structural SVM training of SPENs. Why is it not compared with rank-based training (R-SPENs)? On citation field extraction, SG-SPENs improves accuracy by two percentage points over R-SPENs. On shape parsing, R-SPENs fail because it cannot produce valid parsing programs as improved outputs. SG-SPENs perform well relative to other methods like iterative beam search and neural shape parsing.
# Strengths
SG-SPENs are better across the experiments than other SPEN training methods, though I do not know why they are not compared against R-SPENs on multi-label classification.
# Weaknesses
The work seems incremental without any major new insights beyond the work on R-SPENs. The idea seems to reduce to doing random search instead of gradient descent on a reward function in order to produce output pairs.
As mentioned above, the paper is also light on details about how the experiments were conducted, such as setting \delta and creating the space of operators to use when searching for improved outputs.


Obtaining reliable uncertainty estimates of neural network predictions is a long standing challenge. Bayesian neural networks have been proposed as a solution, but it remains open how to specify their prior. In particular, the common practice of a standard normal prior in weight space imposes only weak regularities, causing the function posterior to possibly generalize in unforeseen ways on inputs outside of the training distribution. We propose noise contrastive priors (NCPs) to obtain reliable uncertainty estimates. The key idea is to train the model to output high uncertainty for data points outside of the training distribution. NCPs do so using an input prior, which adds noise to the inputs of the current mini batch, and an output prior, which is a wide distribution given these inputs. NCPs are compatible with any model that can output uncertainty estimates, are easy to scale, and yield reliable uncertainty estimates throughout training. Empirically, we show that NCPs prevent overfitting outside of the training distribution and result in uncertainty estimates that are useful for active learning. We demonstrate the scalability of our method on the flight delays data set, where we significantly improve upon previously published results.

The paper considers the problem of obtaining reliable predictive uncertainty estimates. The authors propose noise contrastive priors — the idea being to explicitly encourage high uncertainties for out of distribution (OOD) data through a loss in the data space.  OOD data is simulated by adding noise to existing data and the model is trained to maximize the likelihood wr.t. training data while being close in the KL sense to a (wide) conditional prior p(y | x) on the OOD responses (y).  The authors demonstrate that the procedure leads to improved uncertainty estimates on toy data and can better drive active learning on a large flight delay dataset.
The paper is well written and makes for a nice read. I like the idea of using “pseudo” OOD data for encouraging better behaved uncertainties away from the data. It is nice to see that even simple schemes for generating OOD data (adding iid noise) lead to improved uncertainty estimates.
My main concern about this work stems from not knowing how sensitive the recovered uncertainties are to the OOD data generating mechanism and the parameters thereof. The paper provides little evidence to conclude one way or the other.  The detailed comments below further elaborate on this concern.
Detailed Comments:
a) I like the sensitivity analysis presented in Figure 4, and it does show for the 1D sine wave the method is reasonably robust to the choice of \sigma_x. However, it is unclear how problem dependent the choice of sigma_x is. From the experiments, it seems that \sigma_x needs to be carefully chosen for different problems, \sigma^2_x < 0.3 seems to not work very well for BBB + NCP for the 1D sine data, but for the flight delay data \sigma^2_x is set to 0.1 and seems to work well. How was \sigma_x chosen for the different experiments?
b) It is also interesting that noise with a shared scale is used for all 8 dimensions of the flight dataset. Is this choice mainly governed by convenience — easier to select one hyper-parameter rather than eight?
c) Presumably, the predictive uncertainties are also strongly affected by both the weighting parameter \gamma and the prior variance sigma^2_y . How sensitive are the uncertainties to these and how were these values chosen for the experiments presented in the paper?
d) It would be really interesting to see how well the approach extends to data with more interesting correlations. For example, for image data would using standard data-augmentation techniques (affine transformations) for generating OOD data help over adding iid noise. In general, it would be good to have at least some empirical validation of the proposed approach on moderate-to-high dimensional data (such as images).
==============
Overall this is an interesting paper that could be significantly strengthened by addressing the comments above and a more careful discussion of how the procedure for generating OOD data affects the corresponding uncertainties.


The Variational Auto Encoder (VAE) is a popular generative latent variable model that is often applied for representation learning. Standard VAEs assume continuous valued latent variables and are trained by maximization of the evidence lower bound (ELBO). Conventional methods obtain a differentiable estimate of the ELBO with reparametrized sampling and optimize it with Stochastic Gradient Descend (SGD). However, this is not possible if we want to train VAEs with discrete valued latent variables, since reparametrized sampling is not possible. Till now, there exist no simple solutions to circumvent this problem. In this paper, we propose an easy method to train VAEs with binary or categorically valued latent representations. Therefore, we use a differentiable estimator for the ELBO which is based on importance sampling. In experiments, we verify the approach and train two different VAEs architectures with Bernoulli and Categorically distributed latent representations on two different benchmark datasets.

In computing the gradient of the ELBO, the main challenge lies in computing the gradient of the reconstruction loss with respect to the encoder parameters. VAEs traditionally rely on reparameterization in order to obtain a low-variance estimate, but there are a number of other gradient estimators that one can apply. The authors here proprose to use a trick that is known, but perhaps not widely known: If we introduce an importance sampling distribution, then we can use samples from this distribution to compute an importance-weighted estimate of the gradient. The idea is now that we can compute the gradient w.r.t. the encoder parameters as a simple importance-sampling estimate, which obviates then need for reparameterization, or likelihood-ratio estimators. The authors then apply this trick to train VAEs with discrete latent variables.
While I think that the idea that the authors present in this paper is worth further exploration, the paper in its current form is not sufficiently mature to appear at ICLR. The two areas where this paper would benefit from improvement are
1. Discussion of related work.
While the authors seem to suggest that there has been no work on VAEs with discrete latent variables, there has in fact been quite a lot of work, including work on VAEs that contain both discrete and continuous variables (e.g. [8-10], but I'm almost certainly missing further references). There has also been a large body of work on continuous relaxations of discrete variables that are amenable to reparameterization (e.g. [6-7], and references therein). There has also been a line of work relating importance sampling to variational objectives (see [1-3] as key references). Finally, there is also related work on reweighted-wake-sleep style objectives (see [4]) which similarly don't require reparameterization. From what I can tell, none of these references are cited or discussed as related work. In order to place this work in context, I would rewrite 2 to discuss approaches to gradient estimation in this space, which then makes it much easier to explain how this approach differs.
2. Empirical evaluation.
The authors only evaluate on MNIST and F-MNIST, and don't compare to any existing approaches. More than a couple of reconstructions, what I would like to see is an analysis of gradient variances, asymptotic ELBO estimates. I would also like to see a larger set of problems. Finally I would like to see a clear comparison to other methods based on, e.g., continuous relaxations.
References
[1] Y. Burda, R. Grosse, and R. Salakhutdinov, “Importance Weighted Autoencoders,” arXiv:1509.00519 [cs, stat], Sep. 2015.
[2] T. Rainforth et al., “Tighter Variational Bounds are Not Necessarily Better,” arXiv:1802.04537 [cs, stat], Feb. 2018.
[3] G. Tucker, D. Lawson, S. Gu, and C. J. Maddison, “Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives,” arXiv:1810.04152 [cs, stat], Oct. 2018.
[4] T. A. Le, A. R. Kosiorek, N. Siddharth, Y. W. Teh, and F. Wood, “Revisiting Reweighted Wake-Sleep,” arXiv:1805.10469 [cs, stat], May 2018.
[5] A. Mnih and D. J. Rezende, “Variational inference for Monte Carlo objectives,” arXiv:1602.06725 [cs, stat], Feb. 2016.
[6] G. Tucker, A. Mnih, C. J. Maddison, J. Lawson, and J. Sohl-Dickstein, “REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models,” in Advances in Neural Information Processing Systems, 2017, pp. 2624–2633.
[7] W. Grathwohl, D. Choi, Y. Wu, G. Roeder, and D. Duvenaud, “Backpropagation through the Void: Optimizing control variates for black-box gradient estimation,” arXiv preprint arXiv:1711.00123, 2017.
[8] J. T. Rolfe, “Discrete Variational Autoencoders,” arXiv:1609.02200 [cs, stat], Sep. 2016.
[9] E. Dupont, “Learning Disentangled Joint Continuous and Discrete Representations,” arXiv:1804.00104 [cs, stat], Mar. 2018.
[10] B. Esmaeili et al., “Structured Disentangled Representations,” arXiv:1804.02086 [cs, stat], Apr. 2018.


A DIRECT APPROACH TO ROBUST DEEP LEARNING USING ADVERSARIAL NETWORKS
Deep neural networks have been shown to perform well in many classical machine learning problems, especially in image classification tasks. However, researchers have found that neural networks can be easily fooled, and they are surprisingly sensitive to small perturbations imperceptible to humans. Carefully crafted input images (adversarial examples) can force a well-trained neural network to provide arbitrary outputs. Including adversarial examples during training is a popular defense mechanism against adversarial attacks. In this paper we propose a new defensive mechanism under the generative adversarial network (GAN) framework. We model the adversarial noise using a generative network, trained jointly with a classification discriminative network as a minimax game. We show empirically that our adversarial network approach works well against black box attacks, with performance on par with state-of-art methods such as ensemble adversarial training and adversarial training with projected gradient descent.

Summary: The paper proposes a GAN-based approach for dealing with adversarial instances, with the training of a robust discriminator that is able to identify adversaries from clean samples, and a generator that produces adversarial noise for its given input clean image in order to mislead the discriminator. In contrast to the state-of-the-art “ensemble adversarial training” approach, which relies on several pre-trained neural networks for generating adversarial examples, the authors introduce a way for dynamically generating adversarial examples on-the-fly by using a generator, which they along with their clean counterparts are then consumed for training the discriminator.
Quality: The paper is relatively well-written, although a little sketchy, and its motivations are clear. The authors compare their proposed approach with a good of variety of strong defenses such as “ensemble adversarial training” and “PGD adversarial training”, supporting with convincing experiments their approach.
Originality: Xioa et al. (2018) used very similar technique for generating new adversarial examples (generator attack), then used for training a robust discriminator. Likewise, Lee et al. (2018) also used GANs to produce perturbations for making images misclassified. Given this, what is the main novelty of this approach comparing to the (Xioa et al., 2018) and (Lee et al., 2018)? These references should be discussed in details in the paper.
Moreover, limited comparison with different attacks: Why did not compare against targeted attacks such as T-FGS, C&W or GAN-attack?
It is really surprising that undefended network is working better (showing more robustness) than the defended network “adversarial PGD” on black-box attacks, why this is happening?
References:
- Xiao, C., Li, B., Zhu, J. Y., He, W., Liu, M., & Song, D. (2018). Generating adversarial examples with adversarial networks. arXiv preprint arXiv:1801.02610.
- Lee, H., Han, S., & Lee, J. (2017). Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN. arXiv preprint arXiv:1705.03387.


DISCOVERY OF NATURAL LANGUAGE CONCEPTS IN INDIVIDUAL UNITS OF CNNS
Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret. Especially, little is known about how they represent language in their intermediate layers.
In an attempt to understand the representations of deep convolutional networks trained on language tasks, we show that individual units are selectively responsive to specific morphemes, words, and phrases, rather than responding to arbitrary and uninterpretable patterns. In order to quantitatively analyze such an intriguing phenomenon, we propose a concept alignment method based on how units respond to the replicated text. We conduct analyses with different architectures on multiple datasets for classification and translation tasks and provide new insights into how deep models understand natural language.

========== Edit following authors' response  ==========
Thank you for your detailed response and updated version. I think the new revision is significantly improved, mainly in more quantitative analyses and details in several places. I have updated my evaluation accordingly.
See a few more points below.
1. Thank you for clarifying your definition of concepts. I still think that the word "concept" has a strong semantic connotation, while the linguistic elements your analyses capture may do other things. The results in appendix E do show that some semantic clusters arise. It's especially interesting to see the blocks in some of the heat maps, where similar "concepts" are clustered together (like the sports terms in AG); consider commenting on this.
2. The new quantitative analyses are helpful. One other suggestion that I mentioned before is to connect detected concepts to external resources like WordNet or ConceptNet. That would help show that "concepts" are indeed semantic objects.

3. The motivation for replicating as normalizing for length does make sense, although the input would still be unnatural. The comparison to "one instance" is helpful, but it's interesting that the differences between it and replication in figure 2 are not large. It would be good to show results that substantiate your assumption that without replication there will be a bias towards lengthy concepts. Does "one instance" detect more lengthy concepts than replication?
4. The results on frequency and loss difference in 4.5 are very interesting. There is another angle to consider frequency: words that appear frequently often carry less semantic content (e.g. function words), so one might conjecture that they would require less units. It may be interesting to look at which concepts are detected at each frequency bin.
5. Minor points: section 2.2 still mentions "regression" where it should be "classification".
6. A few remaining grammar issues:
- "one concept has a less activation value.." - rephrase
- end of section 3.3: "this experiments" -> "these experiments"
========== Original review follows ==========
Summary:
=======
This paper analyzes individual units in CNN models for text classification and translation tasks. It defines a measure of sensitivity for a unit and evaluates how sensitive each unit is to "concepts" in the input text, where concepts are morphemes, words, and phrases. The analysis shows that some units seem to learn semantic concepts, while others capture linguistic elements that are frequent or relevant for the end task. Layer-wise results show some correspondence between layer depth and linguistic element size.
The paper studies an important question that is relatively under-studies in NLP compared to the computer vision community. The motivation for the work is quite convincing.
I found some of the results and analysis interesting, but overall felt that the work can be made much stronger by more quantitative evaluations. I am also worried that the notion of "concept" is misleading here. See below for this and other comments. I am willing to reconsider my evaluation pending response to the below issues.
Main comments:
=============
1. Concepts:
- morphemes, words, and phrases - are these "concepts"? They are indeed "fundamental building blocks of natural language" (2.2), but "concepts" has a more semantic connotation that I'm not sure these units target at.
- Some of the results do suggest that units learn concepts, as the analysis in 4.2 shows a "unit detecting the meaning of certainty in knowledge" and later units that have similar sentiments. It would be informative to quantify this in some way, for example by matching detected concepts to WordNet synsets, sentiment lexicons, etc., or else tagging and classifying them with various NLP tools. This could also reveal if units learn more syntactic or semantic concepts, and so on.
2. Generally, many of the analyses in the paper are qualitative and on a small scale. The results will be more convincing with more automatic aggregate measures.
3. The structure of the paper is confusing. Section3 starts with the approach but then mentions datasets and tasks (3.1). Section 4 is titled experiments, but section 4.1 starts with defining the concept selectivity. I would suggest reorganizing sections 3 and 4, such that section 3 describes all the methods and metrics, while dataset-specific parts are moved to section 4.
4. section 3.2 should provide more details on the sentence representation and how its obtained in the CNN models. A mathematical derivation and/or figure could be helpful. It is also not clear to me what's the motivation for mean-pooling over the l entries of the vector.
5. section 3.3: the use of replicated text for "concept alignment" is puzzling. This is not a natural input to the model, and I think more justification and motivation åre needed for this issue, as well as perhaps comparison with other approaches.
6. I found section 4.4 very interesting. It shows some intuitive results of larger linguistic elements learned at higher layers, but then some results that do not show such a trend. Then, hypothesizing that the middle layers are sufficient AND validating the hypothesis by retraining the model is excellent. It's a very nice demonstration that the analysis can lead to model improvements.
7. Figure 2 seems to be almost caused by construction of the different options for S_+. Is it surprising that the replicate set has the highest sensitivity? Is there a better control setup than comparing with a random set?
8. One concern that I have is the effect of confounding factors like frequency on the results. The papers occasionally attributes importance to concepts (e.g. in 4.2), but I wonder if instead we may be seeing more frequent words. Controlling for the effect of frequency would be useful.
Minor comments:
==============
- Section 2.2, first paragraph: regression should be changed to classification
- The related work is generally relevant, although one could mention a few other papers that analyzed individual neurons in NLP tasks [1, 2]
- section 4.1: the random set may perhaps be denoted by something more neutral, not S_+ as the replicate and inclusion sets.
- section 4.3, last paragraph: listing examples showing that units in Europarl focus on key words would be good.
- Figure 5, y axis label: should this be number of units instead of concepts?
- Appendix A has several interesting points but there is no reference to them from the main paper.
Writing, grammar, etc.:
======================
- Introduction: among them - who is them?
- 2.1: motivated from -> motivated by; In computer vision community -> In the computer vision community
- 2.1: quantifying characteristics of representations in layer-wise -> rephrase
- 3.2: dimension of sentence -> dimension of the/a sentence
- 4.1: to which -> remove "which"
- 4.2: in the several encoding layer -> in several encoding layers
- 4.3: aliged -> aligned
- Capitalize titles in references
- A.2: with following -> with the following; how much candidate -> how much a candidate; consider following -> consider the following
- A.3: induces similar bias -> induces a bias; such phrase -> such a phrase; on very -> on a very
- C: where model -> where the model; In consistent -> Consistent; where model -> where the model
References
==========
[1] Qian et al., Analyzing linguistic knowledge in sequential model of sentence
[2] Shi et al., Why Neural Translations are the Right Length


NADPEX: AN ON-POLICY TEMPORALLY CONSISTENT EXPLORATION METHOD FOR DEEP REINFORCEMENT LEARNING
Reinforcement learning agents need exploratory behaviors to escape from local optima. These behaviors may include both immediate dithering perturbation and temporally consistent exploration. To achieve these, a stochastic policy model that is inherently consistent through a period of time is in desire, especially for tasks with either sparse rewards or long term information. In this work, we introduce a novel on-policy temporally consistent exploration strategy -Neural Adaptive Dropout Policy Exploration (NADPEx) -for deep reinforcement learning agents. Modeled as a global random variable for conditional distribution, dropout is incorporated to reinforcement learning policies, equipping them with inherent temporal consistency, even when the reward signals are sparse. Two factors, gradients' alignment with the objective and KL constraint in policy space, are discussed to guarantee NADPEx policy's stable improvement. Our experiments demonstrate that NADPEx solves tasks with sparse reward while naive exploration and parameter noise fail. It yields as well or even faster convergence in the standard mujoco benchmark for continuous control.

The authors propose a new on-policy exploration strategy by using a policy with a hierarchy of stochasticity. The authors use a two-level hierarchical distribution as a policy, where the global variable is used for dropout. This work is interesting since the authors use dropout for policy learning and exploration.  The authors show that parameter noise exploration is a particular case of the proposed policy. The main concern is the gap between the problem formulation and the actual optimization problem in Eq 12. I am very happy to give a higher rating if the authors address the following points.
Detailed Comments
(1) The authors give the derivation for Eq 10. However, it is not obvious that how to move from line 3 to line 4 at Eq 15.
Minor:  Since the action is denoted by "a",  it will be more clear if the authors use another symbol to denote the parameter of q(z) instead of "\alpha" at Eq 10 and 15.
(2) Due to the use of the likelihood ratio trick, the authors use the mean policy as an approximation at Eq 12. Does such approximation guarantee the policy improvement? Any justification?
(3) Instead of using the mean policy approximation in Eq 12, the authors should consider existing Monte Carlo techniques to reduce the variance of the gradient estimation. For example, [1] could be used to reduce the variance of gradient w.r.t. \phi. Note that the gradient is biased if the mean policy approximation is used.
(4) Are \theta and \phi jointly and simultaneously optimized at Eq 12?  The authors should clarify this point.
(5) Due to the mean policy approximation, does the mean policy depend on \phi? The authors should clearly explain how to update \phi when optimizing Eq 12.
(6) If the authors jointly and simultaneously optimize \theta and \phi, why a regularization term about q_{\phi}(z)  is missing in Eq 12 while a regularization term about \pi_{\theta|z} does appear in Eq 12?
(7) The authors give the derivations about \theta such as the gradient and the regularization term about \theta (see, Eq 18-19). However, the derivations about \phi are missing.  For example, how to compute the gradient w.r.t. \phi? Since the mean policy is used, it is not apparent that how to compute the gradient w.r.t. \phi.
Minor, 1/2 is missing in the last line of Eq 19.
Reference:
[1] AUEB, Michalis Titsias RC, and Miguel Lázaro-Gredilla. "Local expectation gradients for black box variational inference." In Advances in neural information processing systems, pp. 2638-2646. 2015.


Predicting structured outputs such as semantic segmentation relies on expensive per-pixel annotations to learn strong supervised models like convolutional neural networks. However, these models trained on one data domain may not generalize well to other domains unequipped with annotations for model finetuning. To avoid the labor-intensive process of annotation, we develop a domain adaptation method to adapt the source data to the unlabeled target domain. To this end, we propose to learn discriminative feature representations of patches based on label histograms in the source domain, through the construction of a disentangled space. With such representations as guidance, we then use an adversarial learning scheme to push the feature representations in target patches to the closer distributions in source ones. In addition, we show that our framework can integrate a global alignment process with the proposed patch-level alignment and achieve state-of-the-art performance on semantic segmentation. Extensive ablation studies and experiments are conducted on numerous benchmark datasets with various settings, such as synthetic-to-real and cross-city scenarios.

This paper introduces a domain adaptation approach for structured output data, with a focus here on semantic segmentation. The idea is to model the structure by exploiting image patches, but account for the fact that these patches may be misaligned, and thus not in exact correspondence. This is achieved by defining new patch classes via clustering the source patches according to the semantic information, and making use of an adversarial classifier on the predicted patch-class distributions.
Strengths:
- Modeling the structure via patches is an interesting idea.
- The proposed method achieves good results on standard benchmarks.
Weaknesses:
Method:
- The idea of relying on patches to model the structure is not new. This was achieved by Chen et al., CVPR 2018, "ROAD: Reality Oriented Adaptation...". In this work, however, the patches were assumed to be in correspondence, which leaves some novelty to this submission, although reduced.
- In essence, the patch-based adversarial alignment remains global; this can be thought of as working at a lower resolution and on a different set of classes, defined by the clusters, than the global alignment. The can be observed by comparing Eq. 3 and Eq. 6, which have essentially the same form. This is fine, but was not clear to me until I reached Eq. 6. In fact, what I understood from the beginning of the paper was an alternative formulation, where one would essentially assign each patch to a cluster and aim to align the distributions of the output (original classes) within each cluster. I suggest the authors to clarify this, and possibly discuss the relation with this alternative approach.
- I am not convinced by the claimed relationship to methods that learn disentangled representations. Here, in essence, the authors just perform clustering of the semantic information. This is fine, but I find the connection a bit far-fetched and would suggest dropping it.
Experiments:
- The comparison to the state of the art is fine, but I suggest adding the results of Chen et al, CVPR 2018, which achieves quite close accuracies, but still a bit lower. The work of Saito et al., CVPR 2018, "Maximum Classifier Discrepancy..." also reports results on semantic segmentation and should be mentioned here. I acknowledge however that their results are not as good as the one reported here.
- While I appreciate the ablation study of Section 4.2, it only provides a partial picture. It would be interesting to study the influence of the exact values of the hyper-parameters on the results. These hyper-parameters are not only the weights \lambda_d, \lambda^g_{adv} and \lambda^l_{adv}, but also the number of clusters and the size of the patches used.
Summary:
I would rate this paper as borderline. There is some novelty in the proposed approach, but it is mitigated by the relation to the work of Chen et al., CVPR 2018. The experiments show good results, but a more thorough evaluation of the influence of the hyper-parameters would be useful.


Effectively capturing graph node sequences in the form of vector embeddings is critical to many applications. We achieve this by (i) first learning vector embeddings of single graph nodes and (ii) then composing them to compactly represent node sequences. Specifically, we propose SENSE-S (Semantically Enhanced Node Sequence Embedding -for Single nodes), a skip-gram based novel embedding mechanism, for single graph nodes that co-learns graph structure as well as their textual descriptions. We demonstrate that SENSE-S vectors increase the accuracy of multi-label classification tasks by up to 50% and link-prediction tasks by up to 78% under a variety of scenarios using real datasets. Based on SENSE-S, we next propose generic SENSE to compute composite vectors that represent a sequence of nodes, where preserving the node order is important. We prove that this approach is efficient in embedding node sequences, and our experiments on real data confirm its high accuracy in node order decoding.

This paper presents two approaches: one called SENSE-S for embedding nodes in attributed networks; the other one called SENSE for embedding a sequence of nodes. SENSE-S follows the structure of Skip-gram model. The main difference is that SENSE-S considers both node and words in node content as input and output for learning their embedding. For generating embedding vector for a sequence of nodes, SENSE takes summation of cyclically shifted unit-vectors constructed by SENSE-S on nodes in a sequence.
The paper is well written with a clear definition of the studied problem and a clear introduction of the presented methods. Evaluation was conducted on two real-world data sets (Wikipedia and citation network). It is an interesting idea to represent a sequence by the summation of cyclically shifted unit-vectors of nodes in a sequence. However, there are several concerns about the work presented in this paper.
1) the evaluation of SENSE-S is not sufficient. The baseline methods used in comparison are the simple ones that take concatenation of vectors induced from text and graph, or use one for initializing the learning of the other.  There existing several approaches that learn node embedding vectors from attributed graph (considering both the node content text and graph topology structure), such as TADW [1], HSCA [2], PLANE [3],GAE[4], AANE[5], ANRL [6]. SENSE-S should be compared with these methods for showing its effectiveness.
2) the embedding vector of a node sequence is evaluated by showing the decoding accuracy. It would be more interesting to show how these vectors can be used for some real applications. And, to have high decoding accuracy, the embedding dimension for sequences of 10 nodes should be up to 1024, which is quite expensive for computing and for storage, making the presented method unpractical in real-world applications.
[1] C. Yang, Z. Liu, D. Zhao, M. Sun, E. Y. Chang, Network representation learning with rich text information. IJCAI, 2015
[2] D. Zhang, J. Yin, X. Zhu, C. ZHang, Homophily, structure, and content augmented network representation learning.  ICDM 2016.
[3] T. M. V. Le and H. W. Lauw. Probabilistic latent document network embedding.  ICDM, 2014.
[4] Thomas N Kipf, Max Welling. Variational Graph Auto-Encoders. NIPS Workshop on Bayesian Deep Learning.  2016
[5] Xiao Huang, Jundong Li, Xia Hu. Accelerated attributed network embedding. SDM 2017.
[6] Zhen Zhang, Hongxia Yang, Jiajun Bu, Sheng Zhou, Pinggang Yu, Jianwei Zhang, Martin Ester, Can Wang. ANRL: Attributed Network Representation Learning via Deep Neural Networks. IJCAI, 2018


DIVERSITY IS ALL YOU NEED: LEARNING SKILLS WITHOUT A REWARD FUNCTION
Intelligent creatures can explore their environments and learn useful skills without supervision. In this paper, we propose "Diversity is All You Need"(DIAYN), a method for learning useful skills without a reward function. Our proposed method learns skills by maximizing an information theoretic objective using a maximum entropy policy. On a variety of simulated robotic tasks, we show that this simple objective results in the unsupervised emergence of diverse skills, such as walking and jumping. In a number of reinforcement learning benchmark environments, our method is able to learn a skill that solves the benchmark task despite never receiving the true task reward. We show how pretrained skills can provide a good parameter initialization for downstream tasks, and can be composed hierarchically to solve complex, sparse reward tasks. Our results suggest that unsupervised discovery of skills can serve as an effective pretraining mechanism for overcoming challenges of exploration and data efficiency in reinforcement learning.

This paper proposes a method for learning skills in absence of a reward function. These skills are learned so that the diversity of the trajectories produced by each skill is maximised. This is achieved by having a discriminator attempting to tell these skills apart. The agent is rewarded for visiting states that are easy to distinguish and the discriminator is trained to better infer the skills from states visited by the agent. Furthermore, a maximum entropy policy is used to force the skills to be diverse. The proposed method is general and any RL algorithm with entropy maximisation in the objective can be used, the implementation in the paper uses the Soft Actor Critic method.
The problem that they are tackling is interesting and is of clear value for obtaining more generalisable RL algorithms. The paper is overall clear and easy to follow, the results are interesting and potentially useful, although I have some reservations regarding how they assess this usefulness in the current version of this paper.
Structure-wise, I would say that the choice of writing the paper in the form of a Q&A, with very brief explanations and details was more distracting and at times unnecessary than I liked (e.g. Question 7 could move to Appendix as it is quite trivial).
I really appreciated how much care has been taken to discuss differences with the closest prior work, Variational Intrinsic Control (VIC) by Gregor et al.
One such difference is that their prior distribution over skills is not learnt. While there are good arguments by the authors about why this is appealing (e.g. it prevents collapsing to sampling only a few skills), I feel this could be also quite a limitation of their method. This assumes that you have a good a-priori knowledge and assumptions regarding how many skills are useful or needed in the environment. This is unlikely to be the case in complex environments, where you first need to learn simple skills in order to explore the environment, and later learn to form new more complex skills. During this process, you might want to prune simplistic skills after you learnt more abstract and complex ones, for instance in the context of continual learning. I understand this could be investigated in future work, but I feel they take a rather optimistic take on this problem.
Overall, the use case for the proposed method is slightly unclear to me. While the paper claims to allow diverse set of skills to be learnt, it is highly dependent on learning varied action sequences that help you visit different part of state space, regardless of their usefulness. This means there could be learn a lot of skills that capture part of the state space that is not useful or desirable for downstream tasks. While there is a case made for DIAYN being a stepping stone for imitation learning and hierarchical RL, I don’t find the reported experiments for imitation learning and HRL convincing. In the imitation learning experiment, the distance (KL divergence) between all skills and the expert data is computed and the closest skill is then chosen as the policy imitating the expert. The results are weak and no comparisons with any LfD baselines are reported. The HRL experiments also lack comparisons to any other HRL baseline. I feel that this section is rather weak, especially compared to the rest of the paper, and I am not sure it achieves much.
As a general comment, the choice of reporting the training progress using “hours spent training” is an peculiar choice which is never discussed. I understand that for methods with varying computational costs this might be a fairer comparison but it would be perhaps good to also report progress against number of required environment interactions (including pre-training).
Another assumption made is that the method is valuable in situations where the reward function is expensive to compute and the unsupervised pre-training is free (somewhat easing the large amount of pre-training required). However, it would have been interesting to see examples of such environments in their experiments supporting these claims, as this assumption is not valid for the chosen MuJoCo environments.
Despite these comments, I still feel this is valuable work, that can clearly inspire further relevant work and deserves to be presented at ICLR.
It presents a solid contribution, given its technical novelty, proposed applications and its overall generality.
However, the paper could use more convincing experiments to support its claims.
Additional comments and typos:
- Figure 5 lack error bars across the 5 random seeds and are crucial to assess whether this performance difference is indeed significant given the amount of pre-training required.
- Figure 7’s title and caption is missing...
- typo: page 3, last paragraph “...mutual information between skills and states, **I(S; Z )**” not I(A; Z)
- typo: page 7 paragraph next to Figure 6 “...whereas DIAYN explicitly **learns** skills that effectively partition the state space”
- typo: page 7 above Figure 8 “...make them **exceedingly** difficult for non- hierarchical RL algorithms.”


In this paper, we propose an efficient framework to accelerate convolutional neural networks. We utilize two types of acceleration methods: pruning and hints. Pruning can reduce model size by removing channels of layers. Hints can improve the performance of student model by transferring knowledge from teacher model. We demonstrate that pruning and hints are complementary to each other. On one hand, hints can benefit pruning by maintaining similar feature representations. On the other hand, the model pruned from teacher networks is a good initialization for student model, which increases the transferability between two networks. Our approach performs pruning stage and hints stage iteratively to further improve the performance. Furthermore, we propose an algorithm to reconstruct the parameters of hints layer and make the pruned model more suitable for hints. Experiments were conducted on various tasks including classification and pose estimation. Results on CIFAR-10, ImageNet and COCO demonstrate the generalization and superiority of our framework.

This paper proposes a new framework which combines pruning and model distillation techniques for model acceleration. Though the ``pruning” (Molchanov et al. (2017)) and hint components already exists, the authors claim to be the first to combine them, and experimentally show the benefit of jointly and iteratively applying the two techniques. The authors show better performance of their new framework over baseline, pruning only method and hint only method on a few standard Vision data set.
The motivation is clearly conveyed in the paper. As a framework of combining two existing techniques, I expect the framework can stably improve its two components without too much additional time cost. I have some small questions.
--What is the ``additional cost” of your proposed framework. For example, how many iterations do you typically use. For each data set, what time delta you spent to get the performance improvement comparing to pruning only or hint only models.
--In your iterative algorithm (pseudo code in appendix), the teacher model is only used in the very beginning and final step, though richest information is hidden in the original teacher model. In the intermediate steps, you are fine tuning iteratively without accessing the original teacher model.
--In your reconstruction step, you said due to the randomness, you do not always use the learned new W. How much your algorithm benefit from this selection strategy?


Community detection in graphs is of central importance in graph mining, machine learning and network science. Detecting overlapping communities is especially challenging, and remains an open problem. Motivated by the success of graphbased deep learning in other graph-related tasks, we study the applicability of this framework for overlapping community detection. We propose a probabilistic model for overlapping community detection based on the graph neural network architecture. Despite its simplicity, our model outperforms the existing approaches in the community recovery task by a large margin. Moreover, due to the inductive formulation, the proposed model is able to perform out-of-sample community detection for nodes that were not present at training time.

This paper proposes Deep Overlapping Community detection model (DOC), a graph convolutional network (GCN) based community detection algorithm for network data. The model is a simple combination of GCN and existing framework for community detection. The proposed algorithm is compared to baselines on various datasets, and demonstrated to be accurate in many cases.
I think the paper does not deal with one of the most important aspects of network modeling - the degree heterogeneity of nodes. Many works reported that lack of degree corrections would result in bad estimates of community structures [1,2,3]. Probably including the degrees as feature of nodes would be helpful.
Regarding the stochastic gradient descent by edge subsampling, I think the authors should mention [4], where the idea of edge subsampling in stochastic gradient descent setting was introduced before this work. Also, it is worth noting that we may lose some important distributional properties in graphs if we naively subsample from it [5]. For instance, sampling from positive and negative pairs to balance the class contribution may distort the sparsity and degree distributions of subsampled graphs.
If we choose to use Bernoulli-Poisson link function, we can reduce the time complexity of likelihood and gradient computation to O(N + E), where N is the number of nodes and E is the number of edges, with the auxiliary variable trick introduced in [6]. In that case we don't really have to worry about subsampling. Why didn't you consider applying this to your model?
Regarding the experiments, I think some important baselines are missing [3, 6]. Also, I wonder whether the proposed algorithm would scale to the graphs with more than 100,000 nodes.
References
[1] B. Karrer and M. E. J. Newman. Stochastic blockmodels and community structure in networks. Physical Review E, 83(1):016107, 2011.
[2] P. K. Gopalan, C. Wang, and D. Blei. Modeling overlapping communities with node popularities. NIPS 2013.
[3] A. Todeschini, X. Miscouridou and F. Caron. Exchangeable Random Measures for Sparse and Modular Graphs with Overlapping Communities. CoRR 2016.
[4] J. Lee, C. Heakulani, Z. Ghahramani, L. F. James, and S. Choi. Bayesian inference on random simple graphs with power law degree distributions. ICML 2017.
[5] P. Orbanz. Subsampling large graphs and invariance in networks. CoRR 2017.
[6] M. Zhou. Infinite edge partition models for overlapping community detection and link prediction. AISTATS 2015


THE LAPLACIAN IN RL: LEARNING REPRESENTA- TIONS WITH EFFICIENT APPROXIMATIONS
The smallest eigenvectors of the graph Laplacian are well-known to provide a succinct representation of the geometry of a weighted graph. In reinforcement learning (RL), where the weighted graph may be interpreted as the state transition process induced by a behavior policy acting on the environment, approximating the eigenvectors of the Laplacian provides a promising approach to state representation learning. However, existing methods for performing this approximation are ill-suited in general RL settings for two main reasons: First, they are computationally expensive, often requiring operations on large matrices. Second, these methods lack adequate justification beyond simple, tabular, finite-state settings. In this paper, we present a fully general and scalable method for approximating the eigenvectors of the Laplacian in a model-free RL context. We systematically evaluate our approach and empirically show that it generalizes beyond the tabular, finite-state setting. Even in tabular, finite-state settings, its ability to approximate the eigenvectors outperforms previous proposals. Finally, we show the potential benefits of using a Laplacian representation learned using our method in goalachieving RL tasks, providing evidence that our technique can be used to significantly improve the performance of an RL agent.

Summary: This paper proposes a method to learn a state representation for RL using the Laplacian. The proposed method aims to generalize previous work, which has only been shown in finite state spaces, to continuous and large state spaces. It goes to approximate the eigenvectors of the Laplacian which is constructed using a uniformly random policy to collect training data. One use-case of the learnt state representation is for reward-shaping that is said to accelerate the training of standard goal-driven RL algorithms.
In overall, the paper is well written and easy to follow. The idea that formulates the problem of approximating the Laplacian engenfunctions as constraint optimization is interesting. I have some following major concerns regarding to the quality and presentation of the paper.
- Though the idea of learning a state representation seems interesting and might be of interest within the RL research, the authors have not yet articulated the usefulness of this learnt representation. For larger domains, learning such a representation using a random policy might not be ideal because the random policy can not explore the whole state space efficiently. I wish to see more discussions on this, e.g. transfer learning, multi-task learning etc.
- In terms of an application of the learnt representation, reward-shaping looks interesting and promising. However I am concerned about its sample efficiency and comparing experiments. It takes a substantial amount of data generated from a random policy to attain such a reward-shaping function, so the comparisons in Fig.5 are not fair any more in terms of sample efficiency. On the other hand, the learnt representation for reward-shaping is fixed to one goal, can one do transfer learning/multi-task learning to gain the benefit of such an expensive step of representation learning with a random policy.
- The second equation, below the text"we rewrite the inequality as follows" in page 5, is correct? this derivation is like E(X^2) = E(X) E(X)?
- About the performance reported in Section 5.1, I wonder if the gap can be closer to zero if more eigenfunctions are used?
================
After rebuttal:
Thanks the authors for clarification. I have read the author's responses to my review. The authors have sufficiently addressed my concerns. I agree with the responses and decide to change my overall rating


The ability of modeling the other agents, such as understanding their intentions and skills, is essential to an agent's interactions with other agents. Conventional agent modeling relies on passive observation from demonstrations. In this work, we propose an interactive agent modeling scheme enabled by encouraging an agent to learn to probe. In particular, the probing agent (i.e., a learner) learns to interact with the environment and with a target agent (i.e., a demonstrator) to maximize the change in the observed behaviors of that agent. Through probing, rich behaviors can be observed and are used for enhancing the agent modeling to learn a more accurate mind model of the target agent. Our framework consists of two learning processes: i) imitation learning for an approximated agent model and ii) pure curiosity-driven reinforcement learning for an efficient probing policy to discover new behaviors that otherwise can not be observed. We have validated our approach in four different tasks. The experimental results suggest that the agent model learned by our approach i) generalizes better in novel scenarios than the ones learned by passive observation, random probing, and other curiositydriven approaches do, and ii) can be used for enhancing performance in multiple applications including distilling optimal planning to a policy net, collaboration, and competition. A video demo is available at https://www.dropbox.com/ s/8mz6rd3349tso67/Probing_Demo.mov?dl=0.

This paper presents a method for interactive agent modeling that involves learning to model a demonstrator agent not only through passively viewing the demonstrator agent, but also through interactions from a learner agent that learns to probe the environment of the demonstrator agent so as to maximally change the behavior of the demonstrator agent. The approximated demonstrator agent is trained through standard imitation learning techniques and the learning or probing agent is trained using reinforcement learning. The mind of the demonstrating agent is modeled as a latent space representation from a neural net. This latent space representation is used as the reinforcement learning signal for the learner (probing) agent similar to the curiosity driven techniques where larger changes in the representation of mind are sought out since they should lead to larger differences in demonstrator agent behavior. The authors test this in several gridworld environments as well as a sorting task and show that their method achieves superior performance and generalizes better to unseen states and task variations compared to several baseline methods.
General comments, in no particular order:
1. The authors should provide more details on how the hand-crafted demonstrator agents were made. I assume something similar to an a* algorithm was probably used for the passing task, but what about the maze navigation task?
2. The demonstrated tasks are (gridworld and algorithmic) which are very simple RL taks with low-dimensional (non-visual) state-spaces.  It's unclear how this would scale to more complex tasks with higher-dimensional state spaces such as Atari, Starcraft II or if this would work with tasks with continuous state and action spaces such as mujoco.
3. The core premise behind training the learner agent with RL is using a curiosity driven approach to train a probing policy to incite new demonstrator behaviors by maximizing the differences between the latent vectors of the behavior trackers at different time steps. Because the latent vector is modeled as a non-linear function, distances between latent vector representations do not necessarily correspond to similar distances between behavior policies (for example, KL distances between two policy distributions). Since this is for ILCR, I think the authors should have taken a deeper dive into examining those latent representations and potentially visualizing those distances and how they correspond to different policy behaviors.
4. The biggest flaw that I see in this method is the practicality of it's use. This method relies on the ability to obtain or gain access to a demonstration agent to learn from. In very simple tasks, such as the one presented here, the authors were able to hard-code their own demonstration agent. However, in harder tasks, this will not be feasible. If you are able to obtain or code your own agent, then you've already solved the task and there is no need to do any sort of imitation learning in the first place.  In reality, for sufficiently difficult tasks, a human would be the demonstration agent (as is done in most robotics tasks). In practice, imitation learning from a human works well since the learning can be done offline (i.e., post-hoc after a set of demonstrations are collected from the human). However, this task requires the learning to be interactive and thus the demonstrator needs to be present during the learning.  Interactively learning from a human becomes a problem if the learning takes tens of thousands of episodes of training since a human cannot reasonably be expected to be present for that amount of time. Thus, the question is 1) how well will this method work with a human acting as the demonstrator? and 2) how can this method work if you are not able to have access to a demonstrator long periods of time (or even at all)?
5. My previous comment relates mainly to the application of improved imitation learning. However, I do think this is still very useful in the context of multi-agent reinforcement learning for collaborative and competitive tasks (sections 4.6 and 4.7). I think this method demonstrates a method for improved collaborative and/or competitive performances given the fact that you already have a single agent with a learned policy.
Overall, I think the paper presents a really nice idea of how to improve modeling of agents. essentially, a learner agent learns how to probe a demonstrator agent to provide more information about what's being demonstrated and prevent over-fitting to a set of fixed demonstrations.   This work sounds novel to me from a reinforcement learning perspective, however, I'm not well versed on theory of mind research.


LEARNING TO SCREEN FOR FAST SOFTMAX INFER- ENCE ON LARGE VOCABULARY NEURAL NETWORKS
Neural language models have been widely used in various NLP tasks, including machine translation, next word prediction and conversational agents. However, it is challenging to deploy these models on mobile devices due to their slow prediction speed, where the bottleneck is to compute top candidates in the softmax layer. In this paper, we introduce a novel softmax layer approximation algorithm by exploiting the clustering structure of context vectors. Our algorithm uses a light-weight screening model to predict a much smaller set of candidate words based on the given context, and then conducts an exact softmax only within that subset. Training such a procedure end-to-end is challenging as traditional clustering methods are discrete and non-differentiable, and thus unable to be used with back-propagation in the training process. Using the Gumbel softmax, we are able to train the screening model end-to-end on the training set to exploit data distribution. The algorithm achieves an order of magnitude faster inference than the original softmax layer for predicting top-k words in various tasks such as beam search in machine translation or next words prediction. For example, for machine translation task on German to English dataset with around 25K vocabulary, we can achieve 20.4 times speed up with 98.9% precision@1 and 99.3% precision@5 with the original softmax layer prediction, while state-of-the-art (Zhang et al., 2018) only achieves 6.7x speedup with 98.7% precision@1 and 98.1% precision@5 for the same task.

This paper proposes a novel method to speedup softmax computation at test time. Their approach is to partition the large vocabulary set into several discrete clusters, select the cluster first, and then do a small scale exact softmax in the selected cluster. Training is done by utilizing the Gumbel softmax trick.
Pros:
1. The method provides another way that allows the model to learn an adaptive clustering of vocabulary. And the whole model is made differentiable by the Gumbel softmax trick.
2. The experimental results, in terms of precision, is quite strong. The proposed method is significantly better than baseline methods, which is a really exciting thing to see.
3. The paper is written clearly and the method is simple and easily understandable.
Cons:
1. I’d be really expecting to see how the model will perform if it is trained from scratch in NMT tasks. And I have reasons for this. Since the model is proposed for large vocabularies, the vocabulary of PTB (10K) is by no terms large. However, the vocabulary size in NMT could easily reach 30K, which would be a more suitable testbed for showing the advantage of the proposed method.
2. Apart from the nice precision results, the performance margin in terms of perplexity seems not as big as that of precision. And according to earlier discussions in the thread, the author confirmed that they are comparing the precision w.r.t. original softmax, not the true next words. This could raise a possible assumption that the model doesn’t really get the probabilities correct, but somehow only fits on the rank of the words that was predicted by the original softmax. Maybe that is related to the loss? However, I believe sorting this problem out is kind of beyond the scope of this paper.
3. In another scenario, I think adding some qualitative analysis could better present the work. For example, visualize the words that got clustered into the same cluster, etc.
In general, I am satisfied with the content and enjoys reading the paper.


Learning representations of data is an important issue in machine learning. Though GAN has led to significant improvements in the data representations, it still has several problems such as unstable training, hidden manifold of data, and huge computational overhead. GAN tends to produce the data simply without any information about the manifold of the data, which hinders from controlling desired features to generate. Moreover, most of GAN's have a large size of manifold, resulting in poor scalability. In this paper, we propose a novel GAN to control the latent semantic representation, called LSC-GAN, which allows us to produce desired data to generate and learns a representation of the data efficiently. Unlike the conventional GAN models with hidden distribution of latent space, we define the distributions explicitly in advance that are trained to generate the data based on the corresponding features by inputting the latent variables that follow the distribution. As the larger scale of latent space caused by deploying various distributions in one latent space makes training unstable while maintaining the dimension of latent space, we need to separate the process of defining the distributions explicitly and operation of generation. We prove that a VAE is proper for the former and modify a loss function of VAE to map the data into the pre-defined latent space so as to locate the reconstructed data as close to the input data according to its characteristics. Moreover, we add the KL divergence to the loss function of LSC-GAN to include this process. The decoder of VAE, which generates the data with the corresponding features from the pre-defined latent space, is used as the generator of the LSC-GAN. Several experiments on the CelebA dataset are conducted to verify the usefulness of the proposed method to generate desired data stably and efficiently, achieving a high compression ratio that can hold about 24 pixels of information in each dimension of latent space. Besides, our model learns the reverse of features such as not laughing (rather frowning) only with data of ordinary and smiling facial expression.

* Pros
- addresses an interesting problem
- gives a nice approach to the problem
- attempts to give some theoretical justification for the approach
* Cons
- I generally understand the approach, but details were not clear to me (specifics given below)
- Sections 3.2.1 and 3.2.2 (the theoretical section), I found particuarly hard to follow.
- The visual results are not particularly compelling, tbhough I suppose the panel liked them better than the competitor methods (Table 1).  For example "inverse pale skin" and "pale skin" in figure 5 do not convince me that the model understands skin. The skin and background seem to be changing colors together.  Might be worth including examples of the competitor approaches to show that they are even worse.
* Comments and Questions
- Throughout, you seem to assume binary-valued features, without ever explicitly stating this.  Would be helpful to state explicitly.
- Would be useful to specify the codomain of the discriminator D(x) -- from the objective function, seems to be a value in (0,1).
- In Section 3.2, you say: "Besides, we add the encoder for LSC-VAE into LSC-GAN to make sure that the generated data actually have the desired features.  The encoder projects back to latent space so as to be trained to minimize the difference between latent space where data is generated and the space where the compressed data is projected."  This seems like a fundamental change to training, much more than just initializing a GAN with a VAE-GAN. I think this should be elaborated on.  For example, what happens if you don't include the term with the encoder in (5).  Moreover, what goes wrong if you just try to train everything jointly using (5) without the VAE-GAN initialization step?
- I have a very difficult time understanding 3.2.1, both the text and the equations.  e.g. what are the z_i's in (8)?  In 3.2.1, you say "we pre-train G with the decoder of LSC-VAE" -- this "pretraining" is what you refer to as initialization previously, I think?  Which seems also what section 3.1 is about?
- I think some clarification would be nice in Section 3.2.2.  The conclusion of the section is that the "proposed learning process is valid and efficient."       What do you mean by "valid" and "efficient"?  Perhaps you can explain that a bit more in words at the beginning of the section.  It's not entirely clear where your theory section connects to the objective function in (5). I don't really follow your argument for LSC_VAE being a good initializer in this section. In what sense is theorem 2 demonstrating "efficiency"?
- In Theorem 1, you write p_data \approx p_G.  I guess p_data is some unknown data generating distribution, rather than the empirical distribution of a training set?   I've also never seen \approx 0 used in formal mathematical statements and proofs especially when we're talking about \approx 0 at infinitely many points.  Can this be elaborated on?
- In the end of section 3 intro paragraph you say "The decoder of LSC-VAE is used in the generator (G) of LSC-GAN in the second phase." By "used" do you mean used as the initialization of the generator G, when we switch to the LSC_GAN training?  Seems like it, but could be made more clear.
- In Section 3, second paragraph, you say "In the first phase, LSC-VAE is trained to project data into a specific location of latent space according to its features".  It's not clear whether or not this "projection step" (which I guess is also called encoding step or the inference step depending on the context?) uses the explicit feature values in this step, or can only use the input (e.g. the image).  I really have the same question for the decoder/generator: does it explicitly use the feature values, or does it depend only on the latent variables?  My guess is that in both cases the feature values are not depended on directly, but I think this could be made more clear, one way or the other.
- how did results vary when you deviate from using 20 latent dimension per feature?
- You say "As shown in Fig. 4, the change between images is natural so that we can say that the latent space of LSC-GAN is a manifold." --- maybe a linear manifold?
- Footnote 2 on page 4: This is confusing.  You are [basically arbitrarily] defining the conditional distribution on the latent space for any feature setting.  How can any particular distribution be "correct".
- In the equations in (5), you're taking expectations over z_i, but don't you need to have an expectation over i (the feature assigments) as well?  Do you use the same feature distributions as you have in the training data?  Should be clarified.
- Also, in (5), the expectation over z_i applies to the first term, as well as the z_i in G(z_i) in the second KL divergence term, right?
- In VAE the encoder typically produces the parameters of the Q distribution on the latent space.  What distribution does Q have and how are you parameterizing it?  Indendent Gaussians on each coordinate, each with its own mean and standard deviation?  or what? If you are allowing the encoder to take the feature values as input (which I don't think you are, but am not entirely sure of), does the encoder have to learn the means for each feature setting, or are you explicitly building those feature-based offsets into the encoder?


ON SELF MODULATION FOR GENERATIVE ADVER- SARIAL NETWORKS
Training Generative Adversarial Networks (GANs) is notoriously challenging. We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings. Intuitively, self-modulation allows the intermediate feature maps of a generator to change as a function of the input noise vector. While reminiscent of other conditioning techniques, it requires no labeled data. In a large-scale empirical study we observe a relative decrease of 5% − 35% in FID. Furthermore, all else being equal, adding this modification to the generator leads to improved performance in 124/144 (86%) of the studied settings. Selfmodulation is a simple architectural change that requires no additional parameter tuning, which suggests that it can be applied readily to any GAN.

The paper examines an architectural feature in GAN generators -- self-modulation -- and presents empirical evidence supporting the claim that it helps improve modeling performance. The self-modulation mechanism itself is implemented via FiLM layers applied to all convolutional blocks in the generator and whose scaling and shifting parameters are predicted as a function of the noise vector z. Performance is measured in terms of Fréchet Inception Distance (FID) for models trained with and without self-modulation on a fairly comprehensive range of model architectures (DCGAN-based, ResNet-based), discriminator regularization techniques (gradient penalty, spectral normalization), and datasets (CIFAR10, CelebA-HQ, LSUN-Bedroom, ImageNet). The takeaway is that self-modulation is an architectural feature that helps improve modeling performance by a significant margin in most settings. An ablation study is also performed on the location where self-modulation is applied, showing that it is beneficial across all locations but has more impact towards the later layers of the generator.
I am overall positive about the paper: the proposed idea is simple, but is well-explained and backed by rigorous evaluation. Here are the questions I would like the authors to discuss further:
- The proposed approach is a fairly specific form of self-modulation. In general, I think of self-modulation as a way for the network to interact with itself, which can be a local interaction, like for squeeze-and-excitation blocks. In the case of this paper, the self-interaction allows the noise vector z to interact with various intermediate features across the generation process, which for me appears to be different than allowing intermediate features to interact with themselves. This form of noise injection at various levels of the generator is also close in spirit to what BigGAN employs, except that in the case of BigGAN different parts of the noise vector are used to influence different parts of the generator. Can you clarify how you view the relationship between the approaches mentioned above?
- It’s interesting to me that the ResNet architecture performs better with self-modulation in all settings, considering that one possible explanation for why self-modulation is helpful is that it allows the “information” contained in the noise vector to better propagate to and influence different parts of the generator. ResNets also have this ability to “propagate” the noise signal more easily, but it appears that having a self-modulation mechanism on top of that is still beneficial. I’m curious to hear the authors’ thoughts in this.
- Reading Figure 2b, one could be tempted to draw a correlation between the complexity of the dataset and the gains achieved by self-modulation over the baseline (e.g., Bedroom shows less difference between the two approaches than ImageNet). Do the authors agree with that?


DIAGNOSING AND ENHANCING VAE MODELS
Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In this regard, we rigorously analyze the VAE objective, differentiating situations where this belief is and is not actually true. We then leverage the corresponding insights to develop a simple VAE enhancement that requires no additional hyperparameters or sensitive tuning. Quantitatively, this proposal produces crisp samples and stable FID scores that are actually competitive with a variety of GAN models, all while retaining desirable attributes of the original VAE architecture. The code for our model is available at https://github.com/daib13/TwoStageVAE.

Overview:
I thank the authors for their interesting and detailed work in this paper. I believe it has the potential to provide strong value to the community interested in using VAEs with an explicit and simple parameterization of the approximate posterior and likelihood as Gaussian. Gaussianity can be appropriate in many cases where no sequential or discrete structure needs to be induced in the model. I find the mathematical arguments interesting and enlightening. However, the authors somewhat mischaracterize the scope of applicability of VAE models in contemporary machine learning, and don't show familiarity with the broad literature around VAEs outside of this case (that is, where a Gaussian model of the output would be manifestly inappropriate). Since the core of the paper is valuable and salvageable from a clarity standpoint, my comments below are geared towards what changes the authors may make to move this paper into the "pass" category.
Pros:
- Mathematical insights are well reasoned and interesting. Based on the insight from the analysis in the supplementary materials, the authors propose a two-stage VAE which separate learning the a parsimonious representation of the low-dimensional (lower than the ambient dimension of the input space), and the training a second VAE to learn the unknown approximate posterior. The two-stage training procedure is both theoretically motivated and appears to enhance the output quality of VAEs w.r.t. FID score, making them rival GAN architectures on this metric.
Cons:
- The title and general tone of the paper is too broad: it is only VAE models with Gaussian approximate posteriors and likelihoods. This is hardly the norm for most applications, contrary to the claims of the authors. VAEs are commonly used for discrete random variables, for example. Many cases where VAEs are applied cannot use a Gaussian assumption for the likelihood, which is the key requirement for the proofs in the supplement to be valid (then, the true posterior is also Gaussian, and the KL divergence between that and the approximate posterior can be driven to zero during optimization--clearly a Gaussian approximate posterior will never have zero KL divergence with a non-Gaussian true posterior).
- None of the proofs consider the approximation error garnered by only having access to empirical samples through a sample of the ground truth population. (The ground-truth distribution must be defined with respect to the population rather just the dataset in hand, otherwise we lose all generalizability from a model.) Moreover, the proofs hold asymptotically. Generalization bounds and error from finite time approximations are very pertinent issues and these are ignored by the presented analyses. Such concerns have motivated many of the recent developments in approximate posterior distributions. Overall, the paper contains little evidence of familiarity with the recent advances in approximate Bayesian inference that have occurred over the past two years.
- A central claim of the paper is that the two-stage VAE obviates the need for highly adaptive approximate posteriors. However, no comparison against those models is done in the paper. How does a two-stage VAE compare against one with, e.g., a normalizing flow approximate posterior? I acknowledge that the purpose of the paper was to argue for the Gaussianity assumption as less stringent than previously believed, but all of the mathematical arguments take place in an imagined world with infinite time and unbounded access to the population distribution. This is not really the domain of interest in modern computational statistics / machine learning, where issues of generalization and computational efficiency are paramount.
- While the mathematical insights are well developed, the specifics of the algorithm used to implement the two-stage VAE are a little opaque. Ancestral sampling now takes place using latent samples from a second VAE. An algorithm box is badly needed for reproducibility.
Recommendations / Typos
I noted a few typos and omissions that need correction.
- Generally, the mathematical proofs in section 7 of the supplement are clear. At the top of page 11, though, the paragraph correctly begins by stating that the composition of invertible functions is invertible, but fails to establish that G is also invertible. Clearly it is so by construction, but the explicit reasons should be stated (as a prior sentence promises), and so I assume this is an accidental omission.
- The title of Section 8.1 has a typo: clearly is it is the negative log of p_{theta_t} (x) which approaches its infimum rather than p_{theta_t} (x) approaching negative infinity.
- Equation (4): the true posterior has an x as its argument instead of the latent z.
- Missing parenthesis under Case 2 and wrong indentation. This analysis also seems to be cut off. Is the case r > d relevant here?
* EDIT: I have read the authors' detailed response. It has clarified a few key issues, and convinced me of the value to the community for publication in its present (slightly edited according to the reviwers' feedback) form. I would like to see this published and discussed at ICLR and have revised my score accordingly. *


INVASE: INSTANCE-WISE VARIABLE SELECTION US- ING NEURAL NETWORKS
The advent of big data brings with it data with more and more dimensions and thus a growing need to be able to efficiently select which features to use for a variety of problems. While global feature selection has been a well-studied problem for quite some time, only recently has the paradigm of instance-wise feature selection been developed. In this paper, we propose a new instance-wise feature selection method, which we term INVASE. INVASE consists of 3 neural networks, a selector network, a predictor network and a baseline network which are used to train the selector network using the actor-critic methodology. Using this methodology, INVASE is capable of flexibly discovering feature subsets of a different size for each instance, which is a key limitation of existing state-of-the-art methods. We demonstrate through a mixture of synthetic and real data experiments that IN-VASE significantly outperforms state-of-the-art benchmarks.

This paper proposes a new instance-wise feature selection method, INVASE. It is closely related to the prior work L2X (Learning to Explain). There are three differences compared to L2X. The most important difference is about how to backpropagate through subset sampling to select features.  L2X use the Gumbel-softmax trick and this paper uses actor-critic models.
The paper is written well. It is easy to follow the paper. The contribution of this paper is that it provides a new way,  compared to L2X, to backpropagate through subset sampling in order to select features. The authors compare INVASE with L2X and several other approaches on synthetic data and show outperforming results. In the real-world experiments, the authors do not compare INVASE with other approaches.
Regarding experiments, instance-wise feature selection is often applied on computer vision or natural language process applications, where global feature selection is not enough. This paper lacks experiments on CV or NLP applications. For the MAGGIC dataset, I expect to see subgroup patterns. The patterns that authors show in Figure 2 are very different for all randomly selected 20 patients. The authors do not explain why it is preferred to see very different feature patterns for all patients instead of subgroup patterns.
I have questions about other two differences from L2X, pointed by the authors. First, the selector function outputs a probability for selecting each feature \hat{S}^\theta(x). In the paper of L2X, it also produces a weight vector w_\theta(x) as described in section 3.4. I think the \hat{S}^\theta(x) has similar meaning as w_\theta(x) in L2X. In the synthetic data experiment, the authors fix the number of selected features for L2X so that it forces to overselect or underselect features in the example of Syn4. Did the author try to relax this constraint for L2X and use w_\theta(x) in L2X to select features as using \hat{S}^\theta(x) in INVASE?
Second, I agree with the authors that L2X is inspired by maximizing mutual information between Y and X_S and INVASE is inspired by minimizing KL divergence between Y|X and Y|X_S. Both intuitions lead to similar objective functions that INVASE has an extra term \log p(y|x) and \lambda ||S(x)||. INVASE is able to add a l_0 penalty on S(x) since it uses the actor-critic models. For the \log p(y|x) term, as the author mentioned, it helps to reduce the variance in actor-critic models. This \log p(y|x) term is a constant in the optimization of S(x). In Algorithm 1, 12, the updates of \gamma does not depend on other parameters related to the predictor network and selector network. Could the authors first train a baseline network and use it as a fixed function in Algorithm 1? I don't understand the meaning of updates for \gamma iteratively with other parameters since it does not depend on the learning of other parameters. Does this constant term \log p(y|x) have other benefits besides reducing variance in actor-critic models?
I have another minor question about scaling. How does the scaling of X affect the feature importance learned by INVASE?
Note: I have another concern about the experiments. Previous instance-wise variable selection methods are often tested on CV or NLP applications, could the authors present those experiments as previous works?


DIRECTED-INFO GAIL: LEARNING HIERARCHICAL POLICIES FROM UNSEGMENTED DEMONSTRATIONS USING DIRECTED INFORMATION
The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging. In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning. In this work, we discover the interaction between sub-tasks from their resulting stateaction trajectory sequences using a directed graphical model. We propose a new algorithm based on the generative adversarial imitation learning framework which automatically learns sub-task policies from unsegmented demonstrations. Our approach maximizes the directed information flow in the graphical model between sub-task latent variables and their generated trajectories. We also show how our approach connects with the existing Options framework, which is commonly used to learn hierarchical policies.

The paper presents a learning-based method for learning the latent context codes from demonstrations along with a GAIL model.
This amounts to learning the option segments and the policies simultaneously.
The main contribution is the model the problem as a time-dependent context and then use a directed information flow loss instead of the mutual information loss.
1. What is the effect of models of the underlying distribution of latent codes.
Can it be categorical only, or can it be continuous?
Could we also model it as multidimensional?
The current results only provide single dimensional categorial distribution as latent codes.
2. The paper missed an important line of work which solves nearly the same problem -- option discovery and policy learning.
Krishnan -- Discovery of Deep Option(1703.08294). This work was used by authors in continuous options and then again for program generation (https://openreview.net/pdf?id=rJl63fZRb).
They explicitly infer the option parameters, along with termination conditions with the Expectation Propagation method.
The results are in very similar domains hence comments, if not a comparison, would be useful.
3. The authors state that the main problem with an InfoGail style method is dependence on the full trajectory as in eq 1. Hence the directed info flow is required to solve the problem. However in the actual model, the authors make a sequence of variational approximations -- (a) reduction of eq2 to eq1 with a variation lower bound on posterior p(c|c,\tau) and then replace the prior p(c) with q(c|c,\tau) in eq 5. But looking at the model diagram in fig 2. the VAE actually makes the Markovian assumption -- i.e. c only depends on c_{t-1} and s_{t}. If that is true then how would this be very different from InfoGAIL mutual information loss.
It appears that to capture the authors' mathematical intuition the VAE should have a recurrent generator which should have a hidden state factor passing in to capture dependence on history until the current time.
3a. In fact the first term in eq 6 looks closer to the actually used model. If that is not true then the authors should clarify.
4. Experiments do capture the notion discovery of options. But the simplicity of data leaves much to be desired.
One of the main difference of this work in comparison to unsupervised segmentation models GMM or BP-AR-HMM is the fact that the options learned are composable. But the authors only show this composability on the circle domain -- which is arguably a toy-domain.
A reasonable confirmation that the model indeed learns composition is to generate a trajectory for a sequence of latent code not seen in data. -- like walking -- normal -- left-right-left can be converted to limping gait -- left-left-right-right. This is only a suggestive example.
5. In appendix eq 8 how is the reduction from line 3 to line 4 of the equation made -- what is the implicit assumption.
joint distribution p(c, \tau) is written out as p (\tau|c) p(c) without an integral.


Wide adoption of complex RNN based models is hindered by their inference performance, cost and memory requirements. To address this issue, we develop AntMan, combining structured sparsity with low-rank decomposition synergistically, to reduce model computation, size and execution time of RNNs while attaining desired accuracy. AntMan extends knowledge distillation based training to learn the compressed models efficiently. Our evaluation shows that AntMan offers up to 100x computation reduction with less than 1pt accuracy drop for language and machine reading comprehension models. Our evaluation also shows that for a given accuracy target, AntMan produces 5x smaller models than the state-ofart. Lastly, we show that AntMan offers super-linear speed gains compared to theoretical speedup, demonstrating its practical value on commodity hardware.

Model Compression is used to reduce the computational and memory complexity of DL models without significantly affecting accuracy. Existing works focused on pruning and regularization based approaches where as this paper explores structured sparsity on RNNs, using predefined compact structures.
They replace matrix-vector multiplications which is the building computational block part of RNNs, with localized group projections (LGP). where LGP divides the input and output vectors into groups where the elements of the output group is computed as a linear combination of those from the corresponding input group. Moreover, they use a permutation matrix or a dense-square matrix to combine outputs across groups. They also combine LGP with low-rank matrix decomposition in order to further reduce the computations.
Strong points:
Paper shows how combining the SVD and LGP can reduce computation. In particular in matrix-vector multiplications Ax, low rank reduces the computation by factorizing A into smaller matrices P and Q, while LGP reduces computation by sparsifying these matrices without changing their dimensions.
The paper discussed that their model target labels alone does not generalize well on test data and they showed teacher-student training helps greatly on retaining accuracy. They use the original uncompressed model as the teacher, and train the compressed model(student) to imitate the output distribution of the teacher, in addition to training on the target labels.
Paper is well written and easy to follow.
This paper would be much stronger if it compared against quantization and latest pruning techniques.
This paper replace matrix-vector multiplications with the lowRank-LGP, but they only consider RNN networks. I am wondering how it affects other models given the fact that matrix-vector multiplications is the core of many deep learning models. It is not clear why their approach should only work for RNNs.
Table 1 shows the reduction in computation and model size over the original matrix-vector multiplications Ax. I think in this analysis the computation of the those approaches are neglected. For example running the SVD alone on A (n by m matrix) takes O(m^2 n+n^3). That is true that if P and Q are given, then the cost would be n(m+n)/r. However, finding P and Q takes O(m^2 n+n^3) that could be very expensive when matrices are large.
Table 2 only shows the LGB-shuffle resuts. What about the combined SVD and LGP? Similarly in Table 4, what is the performance of the LGB-Dense?