The use of deep learning for a wide range of data problems has increased the need for understanding and diagnosing these models, and deep learning interpretation techniques have become an essential tool for data analysts. Although numerous model interpretation methods have been proposed in recent years, most of these procedures are based on heuristics with little or no theoretical guarantees. In this work, we propose a statistical framework for saliency estimation for black box computer vision models. We build a model-agnostic estimation procedure that is statistically consistent and passes the saliency checks of Adebayo et al. (2018b). Our method requires solving a linear program, whose solution can be efficiently computed in polynomial time. Through our theoretical analysis, we establish an upper bound on the number of model evaluations needed to recover the region of importance with high probability, and build a new perturbation scheme for estimation of local gradients that is shown to be more efficient than the commonly used random perturbation schemes. Validity of the new method is demonstrated through sensitivity analysis.

Summary
This paper proposes an attribution method, linearly estimated gradient (LEG) for deep networks in the image setting.
The paper also introduces a variant of the estimator called LEG-TV, which includes a TV penalty, and provides a
theorem on the convergence rate of the estimator. The paper finds that the LEG attributions pass sanity
checks.
My recommendation
Overall, I am recommending this paper as a weak accept. There are several points to address with
regards to the exposition and flow of the paper, which is my biggest issue with this paper. I believe
the authors can address this point and I am willing to raise my point on this basis. The paper also
provide some theoretical analysis of the proposed method, which is typically lacking for most
of the interpretation methods in this domain.
Possible Improvements
- The LEG method is not sufficiently motivated. Here, I am specifically referring to the functional
form of the estimated itself in definition 1. See the question section  for some of the issues
I raised there.
- From figure 4, we see that the method passes the proposed sanity checks which seem like a
key motivation for this work, however, the authors don't give an explanation for why this is the
case.
- The paper notes that LEG can be estimated using an LP; it would have been great for the authors
to completely spell this out in the appendix or somewhere in the text. What is the exact form of the
LP? What are the constraints?
- As the authors know, the two evaluations presented in the paper: sanity checks, and the zeroing
out procedure (in figure 5) don't actually tell us which method is a good method, just rule out a method.
I would encourage the authors to design a toy task where the ground truth attributions are know, then
train a model to be 100 percent or so accurate on this task. You can then obtain LEG-TV estimates
from this model and compare to the ground-truth.
- I found the proof of lemma 1 confusing, the authors say it follows trivially, but I don't see it. For example,
there should be a factor of 2 somewhere after taking the derivative wrt to $vec(g)$, but I don't see it. It is
fine for the authors to spell out the derivation here if possible.
-The paper ends quite abruptly with no conclusion or discussion. It would be great to include a wrap up
section that puts the contributions into context.
- I get the sense that this method should be computationally intensive, though the paper says otherwise.
It is fine for a method to be computationally intensive, but can the authors speak to this issue?
Some Questions
Definition 1: I had a difficult time understanding this definition. What is $g$ here? I assume
it is the gradient based on the reference to the first order Taylor expansion. In addition, why
is the estimand squared? Further, What does it mean to take expectation wrt $F + x_0$. I was
particularly confused by the last point, because F is a continuous distribution, while presumably
$x_0$ is the point of interest. The paper notes in several places that it can sample from $F + x_0$,
is this equivalent to sampling from $F$ and adding point $x_0$?
What is LEG0 in figure 5?
Is $\kappa$ in your theorem 1, the condition number of the covariance matrix of the perturbation?
Conclusion
Overall, the paper provides a nice method along with analysis on convergence rates and other statistical
properties. Several of the key issues/questions I have about the paper are raised above. None of these
should be dealbreakers but would require the authors to flesh out more details and possibly justify
certain choices. In general, I think more effort should be put into the flow and writing of the paper.
Overall, this is an interesting contribution.
## After reading author responses
I believe the authors have clarified and improved the readability of the paper and clarified several of the questions
that I had. I am raising my score to an accept. While I believe this is a valuable contribution to the sea of attribution methods that have now been published, like the authors noted, it is still not clearly if attribution methods as a whole
are useful of decision making or understanding of a model by either a generic end user or the model developer.
This is a huge problem in this area that deserves significant attention. This said, the goal of this current paper is to
take a step towards developing a principled method, so this is a step in that direction perhaps.


Stochastic gradient descent (SGD) forms the core optimization method for deep neural networks. While some theoretical progress has been made, it still remains unclear why SGD leads the learning dynamics in overparameterized networks to solutions that generalize well. Here we show that for overparameterized networks with a degenerate valley in their loss landscape, SGD on average decreases the trace of the Hessian of the loss. We also generalize this result to other noise structures and show that isotropic noise in the non-degenerate subspace of the Hessian decreases its determinant. In addition to explaining SGDs role in sculpting the Hessian spectrum, this opens the door to new optimization approaches that may confer better generalization performance. We test our results with experiments on toy models and deep neural networks.

This paper proves that under certain conditions, SGD on average decreases the
trace of the Hessian of the loss. The paper is overall clear and the topic addressed in the paper is relevant in machine learning but I don’t think this paper is ready for publication. There are numerous assumptions that are made in the paper, but not properly stated or backed up. In my view, the experimental results are also not sufficient to compensate for the shortcomings of the theoretical part.
Lemma 1
This is a corollary of Morse lemma, which is also used in Dauphin et al: https://arxiv.org/pdf/1406.2572.pdf (see Equation 1). I don’t see why you would need to re-derive it in your paper and most importantly this should be clearly stated in your paper.
Page 2: existence stationary state?
What are the conditions for the existence of a stationary state? Is bounded noise sufficient?
Equation 4 and local approximation
Regarding the standard decomposition of the Hessian in eq 4 (sometimes referred to as Gauss-Newton decomposition), the discussion is not precise. The authors claim “Empirically, negative eigenvalues are exponentially small in magnitude compared to major positive ones when a model is close to a minimum“
But clearly from the formula itself, one needs to differentiate between minima with low and high function values. For local minima that are high in the energy landscape, the second term might have a large function value. If of course all **depends on the size of the approximation region**. This is extremely important and it is not properly characterized in the paper. The authors simply claim that the loss can be locally approximated while I think these assumptions should be clearly stated. Note that similar types of analyses are usually done in dynamical systems where the behavior of a system is linearized around a critical point. In some cases, one can however characterize the size of a basin of attraction, see e.g. the book Nonlinear Systems (3rd Edition): Hassan K. Khalil.
Assumption on timescale separation
This section makes numerous claims that are not properly backed up.
1) “This assumption is because there is a timescale separation between
the dynamics of \theta_bar, which relax quickly and the dynamics of \theta_hat, which evolve much more slowly as the minimal valley is traversed.“
Are you claiming the absolute value of the eigenvalue of the non-degenerate space are much smaller than the degenerate space? Why would that be so?
2) Ornstein-Uhlenbeck: OU processes require the noise to be Brownian motion. This assumption needs to be clearly stated. Note that there is actually evidence that the noise of SGD is heavy-tail:
Simsekli, Umut, Levent Sagun, and Mert Gurbuzbalaban. "A tail-index analysis of stochastic gradient noise in deep neural networks." arXiv preprint arXiv:1901.06053 (2019).
Take away
I am unsure what the added value of this paper is. Second-order methods can already be shown to decrease the maximum eigenvalue, see e.g. convergence results derived in Cubic regularization of Newton method and its global performance by Nesterov and Polyak. These methods have also been analyzed in stochastic settings where similar convergence results hold as well. What particular insight do we gain from the results of Theorem 2? The authors claim this could “potentially improve generalization” but this is not justified and no reference is cited.
“This indicates that the trace itself is not sufficient to describe generalization (Neyshabur et al., 2017).“
I do not see what aspect of Neyshabur et al. justifies your claim, please explain.
Experiments
All the experiments performed in the paper are on very small models. Given the rather strong assumptions made in the paper, I feel that the paper should provide stronger empirical evidence to back up their claims.


Central to all machine learning algorithms is data representation. For multi-agent systems, selecting a representation which adequately captures the interactions among agents is challenging due to the latent group structure which tends to vary depending on context. However, in multi-agent systems with strong group structure, we can simultaneously learn this structure and map a set of agents to a consistently ordered representation for further learning. In this paper, we present a dynamic alignment method which provides a robust ordering of structured multiagent data enabling representation learning to occur in a fraction of the time of previous methods. We demonstrate the value of this approach using a large amount of soccer tracking data from a professional league.

*Synopsis*:
  The paper proposes a new algorithm for template discovery and representation learning for unstructured multi-agent data with strong group roles (i.e. sports data). Their algorithm is several orders of magnitude faster because of a key assumption in the underlying structure of the position generating distributions of players conditioned on roles. They also formulate the problem in a novel way, splitting the structuring of the data into a three-step approach with each component requiring a different algorithm. This will allow the community to swap in new algorithms into the different components more simply than before. They finally provide some evidence of their algorithm working on a soccer dataset consisting of 380 games.
  Main Contributions:
  - A key insight into the underlying structure of the data, enabling an algorithm with orders of magnitude speed up.
  - A new framing of the problem.
  *Review*:
  The paper is well written, and seems to be a nice improvement over prior art in terms of runtime. I am not very familiar with the surrounding literature, but had a few concerns over the current paper that I feel should be addressed. Overall, I think this paper could be accepted unless other reviewers more familiar with the literature find critical flaws. I would, however, like to see Q2 addressed by the authors.
  Q1 It seems the key insight is that you are modeling the occupancy of space by an agent as a mixture of independent gaussian distributions. How reasonable is the assumption that each generating distribution conditioned on a role is independent of the others? Wouldn't a roles place on the field be dependent by the placement of other roles?
  Q2 How does the performance (i.e. accuracy) of your algorithm compare to prior approaches (i.e. the Sha or Bialkowski)? Or really any baseline? It is hard to judge this method in isolation (other than the impressive computational gains), especially for someone unfamiliar with this literature. Is there a way you could include a comparison in the final draft? If not, why?
      As a note: It seems you include some comparisons in section 4.3, 4.4, and 4.5, but it is unclear to me what the competitor is (i.e. A.3) and how it relates to the surrounding literature. If this is related to prior art, you should make it clear what algorithm this is in the main text, not relegating all discussion to the appendix.
  C1: It would be nice to include table 1 (appendix) in the main paper, or you could include a simplification of the runtime complexity of the K-Means and GMM algorithms in terms you use for the Hungarian algorithm in the main text. This would make it easer to see the significant gains your approach (i.e. using k-means) gets over prior approaches.
  *Minor Comments not taken into account in the review*
  - You should be more clear what the Hungarian algorithm is. Specifically, the Kuhn 1955 should be cited when the algorithm is first mentioned.
  - "global shape and structure the group" -> "global shape and structure of the group"
  - "'anchor' agent (Mehrasa et al.)" -> year for citation
----------
Post Review
I appreciate the author addressing my concerns and making clarifications here in discussion and in the text. I'm going to keep my score as a 6 primarily because I am not confident in the literature this paper is appealing to to truly advocate for its acceptance.


Graph Neural Networks (graph NNs) are a promising deep learning approach for analyzing graph-structured data. However, it is known that they do not improve (or sometimes worsen) their predictive performance as we pile up many layers and add non-lineality. To tackle this problem, we investigate the expressive power of graph NNs via their asymptotic behaviors as the layer size tends to infinity. Our strategy is to generalize the forward propagation of a Graph Convolutional Network (GCN), which is a popular graph NN variant, as a specific dynamical system. In the case of a GCN, we show that when its weights satisfy the conditions determined by the spectra of the (augmented) normalized Laplacian, its output exponentially approaches the set of signals that carry information of the connected components and node degrees only for distinguishing nodes. Our theory enables us to relate the expressive power of GCNs with the topological information of the underlying graphs inherent in the graph spectra. To demonstrate this, we characterize the asymptotic behavior of GCNs on the Erdős -Rényi graph. We show that when the Erdős -Rényi graph is sufficiently dense and large, a broad range of GCNs on it suffers from the "information loss" in the limit of infinite layers with high probability. Based on the theory, we provide a principled guideline for weight normalization of graph NNs. We experimentally confirm that the proposed weight scaling enhances the predictive performance of GCNs in real data.

In the paper, the authors carry out theoretically analysis on the expressive power for GNN. The analysis focused on the limiting case when the depth of layers goes to infinite. The authors prove that if the weights of the GNN satisfy certain condition based on the graph Laplacian, then the transformed features contain only degree and connected component information. Then the authors study the G_{np} random graph as a special case. Finally, empirical experiments are carried out to corroborates the theoretical results.
Strength:
1. The authors study a very important problem. It has long been observed that additional depth does not help GNN. The authors provide a convincing theoretical explanation for the behavior.
2. The theoretical results are very close to practical models with little assumptions. Most importantly, the non-linearity is kept compared to other analysis that takes essential components out from the model. Moreover, the technique used in the paper could potentially benefit other theoretical analysis for NN.
3. The authors provide supporting empirical experiments for the theoretical analysis. The observation on s in section 6.3 and the perpendicular component experiment on section 6.4 provide interesting insight to the performance of GNN.
Weakness:
1. The analysis is mostly for dense graphs. However, most of the real-world networks are large-scale sparse networks.
2. It would be great if the authors could provide suggestions on how to eliminate the information loss in GNN. For example, it would be super interesting if the authors can generalize their analysis when residual links exists.


Sample inefficiency is a long-lasting problem in reinforcement learning (RL). The state-of-the-art estimates the optimal action values while it usually involves an extensive search over the state-action space and unstable optimization. Towards the sample-efficient RL, we propose ranking policy gradient (RPG), a policy gradient method that learns the optimal rank of a set of discrete actions. To accelerate the learning of policy gradient methods, we establish the equivalence between maximizing the lower bound of return and imitating a near-optimal policy without accessing any oracles. These results lead to a general off-policy learning framework, which preserves the optimality, reduces variance, and improves the sampleefficiency. We conduct extensive experiments showing that when consolidating with the off-policy learning framework, RPG substantially reduces the sample complexity, comparing to the state-of-the-art.

This paper proposes to reparameterize the policy using a form of ranking, and updating the policy gradient update accordingly. This on it's own isn't particularly useful, however the authors use this insight to attempt to convert the RL problem into a supervised learning problem. I think this is an interesting paper and approach, however some issues remain.
High level:
The reward threshold appears to require a-priori knowledge of the correct scale to set it at, and the maximum possible reward, which is unfortunate. Also I'm not sure it would make sense in a sparse reward environment, or an environment with small reward leading up to a single large reward, since the trajectories that get small reward might be discarded erroneously. Maybe there should be an annealing schedule for this parameter? This deserves some discussion at least.
The experimental results are very good, however you are using off-policy replay data and therefore have the issue that using the same data many times can (often) improve performance for any off-policy algorithm. This needs to be controlled and investigated very closely, however I don't even see the batch size being used listed anywhere. The authors need to really demonstrate that the benefit is coming from their algorithm and *not* just the re-use of replay data that the baselines don't get to see as much of.
More minor:
Firstly, there is very bad writing in places, .e.g, abstract: "The state-of-the-art uses action value function to derive policy". Things like this appear in many places.
Q-learning, Q-values sometimes Q is upper case sometimes lower case.
Relative action values are never defined, but if they are what I think they are then they are usually referred to as 'advantage values', and are generally given the notation of A(s,a).
Does the policy in (2) sum to 1?
Assumption 1 is written very confusingly. I would state it as something like:
"Assumption 1. For a state s and action i the set of events {E^i_j}_{j \neq i}, where E^i_j corresponds to action ai being ranked higher than action aj, are conditionally independent, given a MDP and a stationary policy."
Assumption 1: What is is conditioned on?
Thm 1: the statement of the theorem has an equality, but the proof has a clear 'approximate equality', this is very misleading since the statement is not actually true!
I don't follow the proof of corollary 5, especially the sentence explaining (65) -> (66), which doesn't parse. Again the proof is actually approximate but the statement is given using an equality, which is false.
The appendix is very messy and several parts of it just consist of series of equations with no supporting explanation.
Refs:
The original DQN paper (Mnih et al) should be cited after you mention DQN.
I was surprised not to see references to
ACER: https://arxiv.org/abs/1611.01224 and
PGQ: https://arxiv.org/abs/1611.01626 when discussing the off-policy / RL + SL work.
Further, in your discussion about entropy regularized RL "However, the discrepancy between
the entropy-regularized objective and original long-term reward exists due to the entropy term." - this has been corrected by a recent work: https://arxiv.org/abs/1807.09647. Probably worth mentioning it here too.


Despite a lot of theoretical efforts, very little is known about mechanisms of implicit regularization by which the low complexity contributes to generalization in deep learning. In particular, causality between the generalization performance, implicit regularization and nonlinearity of activation functions is one of the basic mysteries of deep neural networks (DNNs). In this work, we introduce a novel technique for DNNs called "random walk analysis" and reveal a mechanism of the implicit regularization caused by nonlinearity of ReLU activation. Surprisingly, our theoretical results suggest that the learned DNNs interpolate almost linearly between data points, which leads to the low complexity solutions in the over-parameterized regime. As a result, we prove that stochastic gradient descent can learn a class of continuously differentiable functions with generalization bounds of the order of O(n −2 ) (n: the number of samples). Furthermore, our analysis is independent of the kernel methods, including neural tangent kernels.

This paper suggests a new technique to analyze the implicit regularization caused by ReLU activations. They bound the generalization error by two terms: 1) one term that represents the distance between the trained network output and a piecewise linear function built based on the set of training points and 2) another term that represents the distance between the piecewise linear approximation and the desired target. The first term is bounded using a random walk type of analysis, which to the best of my knowledge is novel.
I find this technique rather interesting and technically sound, although I do have a number of concerns and I'm at the moment more on the reject side, although I will re-consider my score if the authors can provide satisfactory answers.
Generalization to more complex activation functions
If I understand correctly, the interpolation technique between two points only works for ReLU functions. If one were to try to generalize the analysis to more complex non-linear functions by using a more complex interpolation schemes, wouldn’t you then have a random walk in high-dimensions? If so, wouldn’t that be a problem given the different properties of Brownian motion in high-dimensions?
Generalization to smooth activation functions
Another question related to the previous one is whether one could hope to generalize the analysis to smooth activation functions. I believe this is also a drawback of combinatorial techniques such as Hanin and Rolnick which have to rely on the discrete nature of the breakpoints.
Generalization bound is only derived for 1-d functions
Theorem 2 is derived for each dimension independently while the generalization results in Theorem 4 are for 1-dimensional inputs. Where is the difficulty in generalizing these results to higher dimensions?
Prior work on generalization of SGD
I was really expecting a discussion about how the generalization bound derived in this paper compares to prior work, e.g.
Hardt, Moritz, Benjamin Recht, and Yoram Singer. "Train faster, generalize better: Stability of stochastic gradient descent." arXiv preprint arXiv:1509.01240 (2015).
Kuzborskij, Ilja, and Christoph H. Lampert. "Data-dependent stability of stochastic gradient descent." arXiv preprint arXiv:1703.01678 (2017).
Brutzkus, Alon, et al. "Sgd learns over-parameterized networks that provably generalize on linearly separable data." arXiv preprint arXiv:1710.10174 (2017).
And many others…
For instance the bound derived in Hardt et al. is also of the order O(n^-2). The bounds in Kuzborskij are also data-dependent and so are yours since your generalization bound depends on the density of the training points. Can you comment on this? What specific insights do we gain your analysis?
Noise SGD
My understanding is that the authors assume that the noise of SGD is Gaussian. Although this is commonly used when analyzing SGD, there is evidence that the noise is actually not Gaussian, see e.g.
Daneshmand, Hadi, et al. "Escaping saddles with stochastic gradients." arXiv preprint arXiv:1803.05999 (2018).
Simsekli, Umut, Levent Sagun, and Mert Gurbuzbalaban. "A tail-index analysis of stochastic gradient noise in deep neural networks." arXiv preprint arXiv:1901.06053 (2019).
I feel this is worth pointing out and one could perhaps also extend this analysis to Heavy-tail noise. I would expect that the results would still hold in expectation but perhaps with a slightly worse probability.
Influence step size SGD
Using larger step sizes in the SGD updates increase the influence of the noise. I was expecting this to somehow be captured in your analysis but I fail to see where it appears. Can you comment on this?
Proof Lemma 3
The derivation of Eq. 10 does not seem completely justified in the proof in the appendix. The authors essentially prove that the length of the gradient gap is bounded by |S| but why is the coefficient \omega distributed according to a normal distribution. It seems to me that you need the noise of SGD to be Gaussian for such statement to hold. Can you confirm this? If so, I think this needs to be clearly stated as an assumption since -- as pointed out above -- this is not necessarily true in practice.
Minor: proof Theorem 2
It seems rather trivial but for completeness, you should write the proof of Eq. (6) in Theorem 2.
“A priori estimates”
This is a terminology that is often used in the paper but never defined. What do you mean by “a priori” in this context?
Minor comment
I would move footnote 3 directly in the main step. I think it is important to point out that the steps of the random walk correspond to the breakpoints.


This work presents a two-stage neural architecture for learning and refining structural correspondences between graphs. First, we use localized node embeddings computed by a graph neural network to obtain an initial ranking of soft correspondences between nodes. Secondly, we employ synchronous message passing networks to iteratively re-rank the soft correspondences to reach a matching consensus in local neighborhoods between graphs. We show, theoretically and empirically, that our message passing scheme computes a well-founded measure of consensus for corresponding neighborhoods, which is then used to guide the iterative re-ranking process. Our purely local and sparsity-aware architecture scales well to large, real-world inputs while still being able to recover global correspondences consistently. We demonstrate the practical effectiveness of our method on real-world tasks from the fields of computer vision and entity alignment between knowledge graphs, on which we improve upon the current state-of-the-art.

This paper proposes a two-stage GNN-based architecture to establish correspondences between two graphs. The first step is to learn node embeddings using a GNN to obtain soft node correspondences between two graphs. The second step is to iteratively refine them using the constraints of matching consensus in local neighborhoods between graphs. The overall refining process resembles the classic graph matching algorithm of graduated assignment (Gold & Rangarajan, 1996), but generalizes it using deep neural representation. Experiments show that the proposed algorithm performs well on real-world tasks of image matching and knowledge graph entity alignment.
The paper is interesting and has some good potential but lacks some important evaluations and analyses. My main concerns are as follows.
1) The consensus in the second stage is crucial?
As the title shows, the main technical contribution lies in the second stage of consensus inducing. But, for the real tasks, in the experiments, the gain by the second stage is not significant or often negligible (L=0 vs. L=10 or 20  in Table 1,2,3). The results of the first stage (L=0) already give better results than all the baselines in many cases, so that most gains appear to come from the usage of GNNs for representation. This makes the major contribution of this work less significant.  I hope the authors justify this. And, I guess that's maybe because the consensus information may also be induced in the first stage by matching nodes with relational presentations learned using a GNN. To see this, the authors may run the second stage only without the first stage.
2) Comparison to the graduated assignment (GA) process
As discussed in 3.3, the proposed neighborhood consensus can be viewed as a generalization of GA of Eq.6 with trainable neural modules. But, it's not actually shown what is the gain by this generalization. This needs to be shown experimentally by substituting the second stage by GA process.
3) Robustness to node addition or removal.
All the experiments look assuming only edges are varied. Is this algorithm robust to node addition or removal, occurring in many practical graph matching problems? This needs to be also discussed.
======================================
The rebuttal succeeds in addressing most of my concerns so that I upgrade my initial rating to weak accept. I hope all the points in the rebuttal are included in the final manuscript.


Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core. In particular, standard likelihood training and decoding leads to dull and repetitive outputs (Holtzman et al., 2019). While some post-hoc fixes have been proposed, in particular top-k and nucleus sampling, they do not address the fact that the token-level probabilities predicted by the model are poor. In this paper we show that the likelihood objective itself is at fault, resulting in a model that assigns too much probability to sequences containing repeats and frequent words, unlike those from the human training distribution. We propose a new objective, unlikelihood training, which forces unlikely generations to be assigned lower probability by the model. We show that both token and sequence level unlikelihood training give less repetitive, less dull text while maintaining perplexity, giving superior generations using standard greedy or beam search. According to human evaluations, our approach with standard beam search also outperforms the currently popular decoding methods of nucleus sampling or beam blocking, thus providing a strong alternative to existing techniques.

This paper proposes training losses, unlikelihood objective, for mitigating the repetition problem of the text generated by recent neural language models. The problem is well-motivated by evidence from the existing literature. Specifically, the paper argues that the main cause of the degenerated output is the maximum likelihood objective commonly used to train language models. Their main contribution is to introduce additional objectives to penalize “unlikely” word probabilities. The proposed penalty is derived into 2 objectives: token level (previous words in context) and sentence level (future decoded words). The prior objective is used along with the MLE, while the later and more expensive is used for fine-tuning. They perform experiments on Wikitext-103 and evaluate models on the perplexity of the models, and n-gram statistics such as repetition, and uniqueness of the decoded texts. The proposed training scheme (UL-token+seq) is shown to have the closest statistics to the original corpus while the perplexity slightly suffers. The additional manual analysis shows that human annotators prefer the outputs (sentence completion) of the proposed method over the other baselines.
Overall, this paper tackles a relevant problem and could propose a novel method.
For the unlikelihood objectives, there are a few clarifications on the design decision. There are some “correct” repetitions in the ground-truth text as well. However, the proposed loss minimizes all repetitions regardless. In addition, it is unclear how the proposed method mitigates the token distribution mismatch. Finally, there is a similar work where they attempt to “match” repetition (or n-gram distribution) of a reference corpus (https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16961). A discussion of how this paper distinguishes from previous work would be helpful.
For the experiments, there are some missing detail and concerns:
1. The fine-tuning using UL-seq (eq 7) procedure is not well explained. For example, how many sequences are used per update? How many times that you decode in the 1,500 updates?
2. the stochastic decoding results are related as the paper motivation is built on top of Holtzman et al., 2019, the results also support the claim. However, how do you explain the discrepancy between the experts and the crowd workers?
3. GPT-2 results show that UL-seq does not significantly improve several evaluation metrics from MLE. This is a conflict with the result in Table 2. This could be a sign of the generalization problem of the proposed method. Additional results on different model architectures would be helpful.
Minor question:
1. It is uncertain how these repetition and uniqueness statistics translate to a downstream task (e.g. summarization or NMT). Do you have results regarding this?
2. It appears that UL-token does not do much. What is the increase in training time? Do you recommend using the token loss?


Representation learning has recently been successfully used to create vector representations of entities in language learning, recommender systems and in similarity learning. Graph embeddings exploit the locality structure of a graph and generate embeddings for nodes which could be words in a language, products of a retail website; and the nodes are connected based on a context window. In this paper, we consider graph embeddings with an error-free associative learning update rule, which models the embedding vector of node as a non-convex Gaussian mixture of the embeddings of the nodes in its immediate vicinity with some constant variance that is reduced as iterations progress. It is very easy to parallelize our algorithm without any form of shared memory, which makes it possible to use it on very large graphs with a much higher dimensionality of the embeddings. We study the efficacy of proposed method on several benchmark data sets in Goyal & Ferrara (2018b) and favourably compare with state of the art methods. Further, proposed method is applied to generate relevant recommendations for a large retailer.

Thanks to the authors for their response. There are still significant issues with motivation, writing, and baseline comparisons (the latter noted by R3). I would encourage the authors to continue to polish and investigate their method and submit to a future conference.
=====
This paper proposes an approach to learning embeddings associated with nodes in a graph. Inspired by Hebbian learning, the representations of a node are iteratively updated to be similar to representations of its neighbors. The update is performed by adding a scaled vector sampled from a Gaussian centered on a neighbor's representation to the current node's vector. After learning, the embeddings may be used for tasks such as graph reconstruction, link prediction, or product recommendation.
Contributions include:
* Proposal of an approach to learn embeddings of nodes in a graph in which representations of a node are updated by scaled and Gaussian-perturbed versions of its neighbors' representations.
* Experiments demonstrating reconstruction and link prediction performance of the proposed approach on several datasets, as well as product recommendation perfomance on a large retail dataset.

There are several significant concerns with the paper as it currently stands. The three most pressing issues are as follows:
1. The proposed algorithm is not situated relative to related work.
2. The paper does not provide the reader with enough details or precision to be able to replicate the work.
3. Experiments do not compare to any baselines other than random embeddings.

The paper suggests that the proposed algorithm is a form of Hebbian learning because the representation of nearby nodes in the graph are encouraged to be similar. However, this idea has long been used for learning node embeddings (for example, LLE encourages representations of a node to be predicted as a linear combination of neighboring nodes). The connection seems loose other than a superficial similarity in the update rule and naming the algorithm after Hebbian learning is somewhat misleading.
The algorithm is reminiscent of message-passing inference in a continuous Markov random field with pairwise potentials encouraging nearby nodes to have similar representations. I am not an expert in graph embedding approaches, but I would be surprised if the approach could not be easily related to classical approaches such as MDS or LLE.
There are several notational/clarity issues:
* j is used for both the node index and the embedding of the node itself (equation 1-2). Replacing the j on the left hand side with w_j would resolve ambiguity and bring the equations in line with Algorithm 1.
* Equation 2 is inconsistent with equation 1 because they both specify different distributions over the same embedding. Framing equation 1 as an initialization and equation 2 as providing the conditional distribution p(w_j | w_i) may make the situation clearer.
* Equation 3 is a mixture of Gaussian distributions, yet \delta_j is a vector added to the current node embedding. Instead, first write \tilde{w}_i as a sample from the Gaussian and then let \delta_j be the weighted sum of the samples.
* Equation 3 is not consistent with equations 4-5. Equation 3 suggests that a node is updated by summing over neighbors and then applying the update. But Algorithm 1 suggests that nodes are updated based on only a single neighbor at a time.
* What does it mean when the negative embedding is propagated with a small transition probability? This should be described mathematically.
* It is misleading to call the graph a Gaussian hierarchy, since a hierarchy implies that certain nodes are higher than others.
* How are the "transition probabilities" set for an unweighted graph? Specifically, the GrQc dataset doesn't appear to have edge weights.
* What values of the variance and \tau hyperparameters were used?
* How are reconstructions and link predictions computed?

Experimentally, the proposed approach is not compared to any baselines other than random embeddings. The claim made in the paper that the method compares favorably is thus not backed up by results. The results in section 3.2 should be described in greater detail. If the items are nodes, then how are edges and weights determined?
 Other specific comments:
* What is the connection between the current work and hyperbolic geometry of Nickel & Kiela (2017)? The proposed algorithm does not rely on hyperbolic geometry so this seems like a non sequitur.
* Algorithm 1: Rather than describing the algorithm in terms of the intended application (products), it would be useful to describe it in general terms and then use retail products as specific application.
* Figures 2 and 3 are not particularly useful. The most important information for the reader or practitioner is how various methods compare on the same dataset, not how a single method performs across different datasets.

Questions for the authors:
* How is the proposed algorithm similar/different to related approaches for learning node embeddings?
* What are baseline results for related algorithms on the datasets experimented upon?
* What is the role of the variance scaling? How do the results change if the variance is reduced to 0 immediately after random initialization?


While pre-training and fine-tuning, e.g., BERT (Devlin et al., 2018), GPT-2 (Radford et al., 2019), have achieved great success in language understanding and generation tasks, the pre-trained models are usually too big for online deployment in terms of both memory cost and inference speed, which hinders them from practical online usage. In this paper, we propose LightPAFF, a Lightweight Pre-training And Fine-tuning Framework that leverages two-stage knowledge distillation to transfer knowledge from a big teacher model to a lightweight student model in both pre-training and fine-tuning stages. In this way the lightweight model can achieve similar accuracy as the big teacher model, but with much fewer parameters and thus faster online inference speed. LightPAFF can support different pretraining methods (such as BERT, GPT-2 and MASS (Song et al., 2019)) and be applied to many downstream tasks. Experiments on three language understanding tasks, three language modeling tasks and three sequence to sequence generation tasks demonstrate that while achieving similar accuracy with the big BERT, GPT-2 and MASS models, LightPAFF reduces the model size by nearly 5x and improves online inference speed by 5x-7x.

Summary: This work leverages knowledge distillation both in pre-training and fine-tuning stages to learn a more compact student model that approximates the performance of a teacher model. Extensive experiments with different knowledge distillation loss functions are conducted on a number of representative language representation models including BERT, GPT-2 and MASS to show the benefit of the method.
Strengths:
The numerical results show some promise of the proposed method to achieve competitive performance to the teacher network and outperform the considered baselines. The evaluated 9 downstream tasks cover a wide range of language understanding and generation problems and they do prove the effectiveness of knowledge distillation in boosting the accuracy.
Weaknesses:
My main concern is about the limited novelty of approach. The knowledge distillation method developed in the paper seems not particularly novel in principle. Although extensive numerical results are provided to show the effectiveness of knowledge distillation, the value-added beyond applying different distillation loss functions in different NLP tasks is still insufficient. Particularly, the loss function that combines maximal likelihood loss with KL divergence looks similar to those used in the paper [1]. The connection and difference to that related work need to be clarified.
[1] Yu, Dong, et al. "KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition." 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2013.
=== update after author response ===
Thank you for the response. Although interesting and showing some promise in a bunch of applications, I am still not convinced that the proposed approach is novel enough in principle.  I thus stick to my assessment of this paper as borderline leaning to rejection.


Understanding black-box machine learning models is important towards their widespread adoption. However, developing globally interpretable models that explain the behavior of the entire model is challenging. An alternative approach is to explain black-box models through explaining individual prediction using a locally interpretable model. In this paper, we propose a novel method for locally interpretable modeling -Reinforcement Learning-based Locally Interpretable Modeling (RL-LIM). RL-LIM employs reinforcement learning to select a small number of samples and distill the black-box model prediction into a low-capacity locally interpretable model. Training is guided with a reward that is obtained directly by measuring agreement of the predictions from the locally interpretable model with the black-box model. RL-LIM near-matches the overall prediction performance of black-box models while yielding human-like interpretability, and significantly outperforms state of the art locally interpretable models in terms of overall prediction performance and fidelity.

This paper studies the problem of interpreting predictions of blackbox models. In particular, they study local interpretable models, which are used to study interpretability at the level of one or a few data points. A key challenge is that in order for local models to be interpretable, they need to be simple in form and therefore lower in capacity (i.e. linear); thus, if they are trained on entire datasets they will underfit.
The aim of this work is to address this issue by learning how to select representative samples from a dataset for training local linear models to reproduce predictions of black-boxes. In particular, they propose to use RL to learn to weight instances from datasets; RL is required as they make hard (i.e. non-differentiable) decisions to select a subset of the dataset.
This work is closely related to Ren et al. 2018 [1], which proposes to meta-learn how to weight samples in a batch so as to maximize performance on a validation set. By analogy, in this paper, the samples are taken from the entire dataset (i.e. the dataset is subsampled), while the validation loss is effectively an imitation loss formed by treating the black-box model predictions as a target. The data subsampling operation introduces the added complication of non-differentiability. The contribution of this paper is thus the use of RL for meta-learning how to subsample a larger dataset in order to maximize some validation loss.

* Pros:
	* Considers an interesting dataset subsampling variant of the sample weighting meta-learning problem.
	* Novel application of meta-learning for improving locally-linear models.
	* Extensive quantitative evaluation shows that the method seems to perform better than baselines, though it might be that a differentiable approximation could do as well while being more sample efficient.
	* It is a nice result that the l1 penalty actually works well in reducing the number of samples chosen by the
	* I found the discussion and figures presented in 4.2 to be quite nice and informative.
* Cons:
	* Given the lack of a differentiable approximation baseline, I am not entirely convinced that the use of RL is absolutely necessary/optimal.
		* I.e. if the weighting function is actually high-entropy, randomly sampling a (large) batch and weighting it might work just as well.
	* Though there is discussion of the complexity of the overall method it would be nice to see a discussion and figures related to the sample efficiency of REINFORCE?
		* This would be strongest if given with a comparison to differentiable alternatives (mentioned above) as well.
		* This would help elucidate whether RL is optimal in this setting: fitting a linear model on more data might be cheaper learning to subsample with REINFORCE.
	* While the sample weighting function is fast at inference time, most of the overhead comes at training time. This function needs be updated in settings where the underlying dataset changes.
	* This is a minor issue, but this pushes the burden of interpretability further up to the black-box sample weighting function. While this interpretability problem is less critical, it still exists.
* Other comments/requests:
	* While the use of RL is certainly motivated in order to solve the problem in an unbiased way, it would be nice to see a comparison to a differentiable approximation as a baseline? A few ideas:
		* Randomly sample a (possibly large batch) and learn to weight it (closely related to the straight through estimator)
		* Randomly sample a batch and apply [1]
	* Would be nice to show the sizes of datasets and how many samples end up being used for different values of lambda.
	* Would be nice to understand which samples are chosen and why. This is probably tricky to analyze, but it would be interesting to see if certain samples are often chosen, or if the weighting distribution has an interesting shape (i.e. is low or high-entropy).
I’ve given a weak accept, conditioned on being provided more evidence regarding 1) comparisons to simple differentiable alternatives, 2) sample efficiency of the RL method, and 3) basic analysis of the weighting function.
[1] Learning to Reweight Examples for Robust Deep Learning. Mengye Ren, Wenyuan Zeng, Bin Yang, Raquel Urtasun. https://arxiv.org/abs/1803.09050


Certified defenses against adversarial examples are very important in safety-critical applications of machine learning. However, existing certified defense strategies only safeguard against perturbation-based adversarial attacks, where the attacker is only allowed to modify normal data points by adding small perturbations. In this paper, we provide certified defenses under the more general threat model of unrestricted adversarial attacks. We allow the attacker to generate arbitrary inputs to fool the classifier, and assume the attacker knows everything except the classifiers' parameters and the training dataset used to learn it. Lack of knowledge about the classifiers parameters prevents an attacker from generating adversarial examples successfully. Our defense draws inspiration from differential privacy, and is based on intentionally adding noise to the classifier's outputs to limit the attacker's knowledge about the parameters. We prove concrete bounds on the minimum number of queries required for any attacker to generate a successful adversarial attack. For a simple linear classifiers we prove that the bound is asymptotically optimal up to a constant by exhibiting an attack algorithm that achieves this lower bound. We empirically show the success of our defense strategy against strong black box attack algorithms.

Although this paper's title contains "certified defense" and "unrestricted adversarial attack",  what I believe this paper is doing is analyzing the query complexity of query-based black-box attacks under simple linear models such as logistic regressions (or kernelized versions). The authors considered a binary classifier with the additional capability of giving "no response" when the confidence is low.  In addition, the output of the classifier has to be perturbed by a random Gaussian vector. The authors then define several metrics including defensibility and query privacy to develop the query complexity on the considered model. The authors tested the query performance on two attacks: (1) the sign attack proposed by the authors and (2) the simba attack proposed by Guo et al.
I have several concerns regarding this paper:
1. In my perspective, the title is very misleading and does not properly justify the claims made in this paper. "Certified defense" usually refers to consistent top-1 prediction of a perturbed data sample under a defined threat model. The paper reads like the authors are actually certifying the defined defensibility metric but without a threat model to certify. In addition, the attack setting is limited to black-box attacks (i.e. zero-order adversary), whereas in certified defense the attack assumption is white-box.
2. It is also very unclear how unrestricted attack plays a role in the studied problem.  In the introduction, the authors' definition of adversarial examples is "any input is considered a valid adversarial example as long as it induces the classifier to predict a different label than an oracle classifier." But what is the oracle classifier? How do we justify the credibility of the "adversarial examples" in the experiments?
3. Only two black-box attacks were compared in this paper, one is the sign attack proposed by the authors, the other is the simba attack proposed by Guo et al. To my knowledge, simba attack paper has not been published at any peer-reviewed venue. In other words, both attacks are not widely recognized attacks or methods from published papers. Therefore, the performance evaluation is not fully justified. Since there are many black-box attacks from published papers, why not do performance analysis on those attacks?
4. Similar to 3, the classifier setting is also uncommon. Although I am happy to see classifiers have the ability to give no-response,  admittedly this type of classifier is rarely used in practice, not to mention the analysis is tied with Gaussian perturbation on the output. The technical contributions can be limited if the main contribution of this paper is characterizing the query complexity (or defensibility) of an uncommon classifier with Gaussian perturbation on the output. I believe providing more insights on how the analysis can be useful to mainstream classifiers are critical and necessary.
***Post-rebuttal comments
I thank the authors for the response. I hope the comments areuseful for preparing a future version of this work.
***


The capability of reliably detecting out-of-distribution samples is one of the key factors in deploying a good classifier, as the test distribution always does not match with the training distribution in most real-world applications. In this work, we propose a deep generative classifier which is effective to detect out-of-distribution samples as well as classify in-distribution samples, by integrating the concept of Gaussian discriminant analysis into deep neural networks. Unlike the discriminative (or softmax) classifier that only focuses on the decision boundary partitioning its latent space into multiple regions, our generative classifier aims to explicitly model class-conditional distributions as separable Gaussian distributions. Thereby, we can define the confidence score by the distance between a test sample and the center of each distribution. Our empirical evaluation on multi-class images and tabular data demonstrate that the generative classifier achieves the best performances in distinguishing out-of-distribution samples, and also it can be generalized well for various types of deep neural networks.

Summary:
Unlike the softmax classifier, the authors considered the generative classifier based on Gaussian discriminative analysis and showed that such deep generative classifiers can be useful for detecting out-of-distribution samples. For various benchmark tasks, the proposed method outperforms baselines based on the softmax classifier.
Detailed comments:
The novelty of this paper is not significant due to the following reasons:
1. The main message (i.e. the concept of the deep generative classifier can be useful for detecting out-of-distribution samples) is not really new because it has been explored before [Lee' 18]. Even though this paper considers training a deep generative classifier directly unlike [Lee' 18], the proposed method looks like a simple variant of [Lee' 18].
2. Missing baselines for training the deep generative classifier: training the deep generative classifier directly has been studied by [Guerriero' 18] and [Pang' 18] but the authors did not compare the proposed training method with such baselines. Because of that, it is hard to say that contributions from proposing a training method are significant.
Questions:
1. Could the authors consider a case without an identity covariance assumption? Most training methods for deep generative classifier assumes the identity covariance matrix because optimizing the log determinant is not easy. So, it would be interesting if the authors can handle this issue.
2. Even though the authors assume the identity covariance matrix, the covariance matrix of pre-trained features can not be an identity matrix. Could the authors report the performance of Mahalanobis detector using the proposed deep generative classifier?
[Lee' 18] Lee, K., Lee, K., Lee, H. and Shin, J., 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems (pp. 7167-7177).
[Guerriero' 18] Samantha Guerriero, Barbara Caputo, Thomas Mensink, DeepNCM: Deep Nearest Class Mean Classifiers, ICLR workshop 2018.
[Pang' 18] Pang, T., Du, C. and Zhu, J.,  Max-mahalanobis linear discriminant analysis networks. In ICML, 2018.


Among deep generative models, flow-based models, simply referred as flows in this paper, differ from other models in that they provide tractable likelihood. Besides being an evaluation metric of synthesized data, flows are supposed to be robust against out-of-distribution (OoD) inputs since they do not discard any information of the inputs. However, it has been observed that flows trained on FashionMNIST assign higher likelihoods to OoD samples from MNIST. This counterintuitive observation raises the concern about the robustness of flows' likelihood. In this paper, we explore the correlation between flows' likelihood and image semantics. We choose two typical flows as the target models: Glow, based on coupling transformations, and pixelCNN, based on autoregressive transformations. Our experiments reveal surprisingly weak correlation between flows' likelihoods and image semantics: the predictive likelihoods of flows can be heavily affected by trivial transformations that keep the image semantics unchanged, which we call semantic-invariant transformations (SITs). We explore three SITs (all small pixel-level modifications): image pixel translation, random noise perturbation, latent factors zeroing (limited to flows using multi-scale architecture, e.g. Glow). These findings, though counter-intuitive, resonate with the fact that the predictive likelihood of a flow is the joint probability of all the image pixels. So flows' likelihoods, modeling on pixel-level intensities, is not able to indicate the existence likelihood of the high-level image semantics. We call for attention that it may be abuse if we use the predictive likelihoods of flows for OoD samples detection.

Summary:
The paper studies likelihood-based models of images, such as Glow and PixelCNN. The paper shows empirically that image transformations that preserve semantics (e.g. translations by a few pixels) produce images that have lower probability (density) under such models.
Decision:
The paper is studying an important topic, which is how to use likelihood-based models correctly, and it warns against the misuse of such models. I agree with the paper's conclusion that we should be careful when using likelihood-based models.
Nonetheless, in my opinion the paper is a clear reject. The paper is full of flaws, incorrect statements, poorly constructed arguments, speculative explanations, and superficial descriptions of previous work.
Broadly, the main issue with the paper is the following:
(a) It begins with flawed assumptions about how a likelihood-based model is expected to behave.
(b) It tests two likelihood-based models experimentally and finds that they don't behave according to the assumptions.
(c) It concludes that we need to be cautious when using likelihood-based models.
As I said, I agree that we should be careful when using likelihood-based models, but I worry that the way the paper reaches this conclusion can mislead and misinform readers. In what follows, I elaborate on specific issues with the paper in more detail.
Issue #1:
The main flaw of the paper is the assumption that semantic-preserving transformations shouldn't reduce the likelihood of the model (beginning of section 4). This is incorrect. To see why, consider a semantic-preserving transformation x' = T(x). As defined in the paper, a semantic-preserving transformation is one that doesn't change the label y of an image x. This can be formalized as:
p(y | x') = p(y | x)
By Bayes' rule, from the above it follows that:
p(x' | y) p(y) / p(x') = p(x | y) p(y) / p(x)
=> p(x' | y) / p(x') = p(x | y) / p(x)
Clearly p(x') can be different from p(x), as long as p(x' | y) is different from p(x | y) by the same factor. Hence, it doesn't follow that if p(y | x') = p(y | x) then p(x') = p(x), which is what the paper incorrectly assumes. To be clear, in the above expressions, p() refers to the true data distribution and not to a model that approximates it.
For example, consider images of digits, where the digit is generally in the centre of the image. Moving a digit to the corner will result in a less likely image, because it's unlikely that digits appear in corners. However, it won't change the classification of that digit, since all digits are less likely to appear in corners in exactly the same way. In fact, this is exactly why we see in section 4.1 that the likelihood of the model decreases as the image is translated to the left; the model is behaving exactly as it is supposed to.
Similarly, in section 4.2 where noise is added to the image, the model is again behaving exactly as it is supposed to. Adding Gaussian noise to the image results in a distribution p(x') that is equal to convolving the original distribution p(x) with the noise distribution p(noise) which is an isotropic Gaussian. As a result, p(x') will be a more diffuse version of p(x), hence samples x' will have on average low probability (density) under p(x), exactly as expected, and exactly as the experiment observes.
Issue #2:
The paper incorrectly assumes that out-of-distribution examples should have low probability (density). This is incorrect, and a common misconception that results from confusing high probability with typicality. In fact, out-of-distribution examples can have high probability (density). Here are two examples that illustrate that:
Suppose you flip a bent coin a million times, with 10% probability of the coin coming up heads. The in-distribution samples (the typical set) are those sequences of coin tosses that have roughly 100 thousand heads. However, the most likely outcome is the all-tails sequence. This outcome is clearly atypical, and many people would agree that it's out-of-distribution, but it has the highest probability.
Consider one million independent Gaussian variables, each with mean 0 and variance 1. Due to the law of large numbers, a typical draw of these variables will have average squared value very close to 1, hence the outcome of all variables being zero is very atypical and many people would agree it's out-of-distribution. However, the all-zero outcome is in fact the one with the highest probability density.
Given the above, the following two statements copied from the paper are flawed, and potentially misleading:
"The foundation of using likelihood-based models for OoD detection is that they are supposed to assign much lower likelihoods for OoD samples than in-distribution samples"
"In OoD detection, we assume that a sample with a higher likelihood indicates that it is more likely to be an in-distribution sample"
Fundamentally, I think the issue is that the paper incorrectly assumes that all images with the same semantics (e.g. all images of the digit 3) must be in-distribution. However this is not necessarily true. For example, the true data-generating process of MNIST images (i.e. asking people to write down a digit, scanning it, denoising it, cropping it and centring it) is unlikely to produce images where the digit is not in the centre or the background is noisy. Hence, the images considered in sections 4.1 and 4.2 are indeed out-of-distribution with respect to the true data distribution of MNIST, and are not adversarial examples of the models as the paper suggests.
Further issues:
The paper is ostensibly about flow models, but in fact very little is specific to flow models, and most of the discussion, where correct, applies to likelihood-based models in general. In fact, PixelCNN is not a flow model, even though the paper misleadingly describes it as such. PixelCNN can be used to model discrete random variables, whereas flow models are used for continuous random variables (flows for discrete random variables exist, but they are different from PixelCNN). That said, if PixelCNN is used to model continuous random variables then by reparameterization it can be viewed as a flow model with one layer, but that would be an unusual way to present it. Same for WaveNet.
"It is also believed that flows can be used to detect out-of-distribution(OoD) samples by assigning low likelihoods on them."
Believed by whom? A citation is needed here.
"Flows can roughly be divided into two categories [coupling flows and autoregressive flows]"
There are several flows that fall in neither of these categories, such as linear flows, residual flows, planar flows, radial flows, Sylvester flows, neural ODEs, FFJORD, and many others.
"The autoregressive property of an autoregressive layer is enforced by masking."
There are other ways of enforcing the autoregressive property (e.g. RNNs); masking is just one of them.
Eq. (5) is not correct in general. The bits per dimension should be approximated by:
BPD = (NLL - log|B|) / ((h x w x c) * log2)
where |B| is the quantization volume, provided |B| is small. That is, if each pixel with range [0, 1] is quantized into 10 bins, then |B| = 0.1 ^ (h x w x c).
"This surprising difference can be attributed to the difference of their architectures" (beginning of page 5)
The explanation that follows is speculative, but is not presented as such, which can be misleading.
"We may reasonably suspect that flows’ counter-intuitive likelihood assignment is dominated by the inherent differences of pixel-level statistics associated to the image semantics"
This is also speculation.
"PixelCNN is more sensitive to the noises, because its pixel-wise modeling quickly augment and propagate the influences of the added noise"
Also speculation.
"We find that the semantic object of a test image depends heavily on the last factored latent zL, rather than the preceding factors"
As far as I can see, there isn't evidence in support of that statement in the paper.
"Considering the weak correlation between flows’ likelihoods and image semantics, it is inappropriate to use them for OoD samples detection"
Given the flawed assumption about the role of image semantics, I don't think there is evidence for that.
"In terms of image generation, we expect that every single generated pixel in a image is the most likely one"
This is inaccurate; when generating images from a model, we don't get the most likely pixels, but samples from the joint distribution over pixels.


We present a deep generative model for unsupervised text style transfer that unifies previously proposed non-generative techniques. Our probabilistic approach models non-parallel data from two domains as a partially observed parallel corpus. By hypothesizing a parallel latent sequence that generates each observed sequence, our model learns to transform sequences from one domain to another in a completely unsupervised fashion. In contrast with traditional generative sequence models (e.g. the HMM), our model makes few assumptions about the data it generates: it uses a recurrent language model as a prior and an encoder-decoder as a transduction distribution. While computation of marginal data likelihood is intractable in this model class, we show that amortized variational inference admits a practical surrogate. Further, by drawing connections between our variational objective and other recent unsupervised style transfer and machine translation techniques, we show how our probabilistic view can unify some known non-generative objectives such as backtranslation and adversarial loss. Finally, we demonstrate the effectiveness of our method on a wide range of unsupervised style transfer tasks, including sentiment transfer, formality transfer, word decipherment, author imitation, and related language translation. Across all style transfer tasks, our approach yields substantial gains over state-of-the-art non-generative baselines, including the state-of-the-art unsupervised machine translation techniques that our approach generalizes. Further, we conduct experiments on a standard unsupervised machine translation task and find that our unified approach matches the current state-of-the-art.

In this paper, the authors propose a probabilistic framework for unsupervised text style transfer. Given two non-parallel corpora X,Y in different domains, the authors introduce unobserved corpora \bar{X}, \bar{Y}. These are used as latent variables that control the generation of the observed data. To train models, the paper proposes to optimize the evidence lower bound of the log marginal likelihood. To facilitate training, multiple techniques are suggested, such as parameter sharing, some gradient approximations and initialization with a reconstruction objective. The approach is evaluated on five style transfer tasks, as well as unsupervised machine translation. Models are evaluated with multiple metrics, and generally obtain reasonably strong performance.
I lean towards the acceptance of the paper because the approach is fairly simple and elegant, while obtaining promising results. The connections to back-translation and language models are also potentially interesting. However, while the paper aims to suggest a principled approach to style transfer, using greedy samples biases the reconstruction objective, and as such the method does not really optimize the ELBO.
Casting style transfer as data completion is a straight-forward idea that doesn't introduce unnecessary or too simplistic assumptions. Optimizing the ELBO follows naturally, and can lead to more diverse outputs than the BT+NLL approach, which misses the negative entropy term. Reference BLEU scores on all tasks are competitive, and sometimes clearly better, with strong baselines.
Greedily sampling latent sequences during training should ideally be justified more carefully as it biases the objective function. In particular, an experimental comparison to stochastic sampling, which should more closely approximate the expectation, would be appreciated. Additionally, detailing the similarities and differences between the proposed approach and current UNMT techniques could be helpful to some readers.
Questions:
Could you present the validation and test evidence lower bounds? If so, how is sampling performed?
In footnote 2, you mention tuning the strength of the KL regularizer. As the KL can be decomposed into 2 terms (Eq. 5), would it be beneficial to control each term separately?


We propose a neural network for unsupervised anomaly detection with a novel robust subspace recovery layer (RSR layer). This layer seeks to extract the underlying subspace from a latent representation of the given data and removes outliers that lie away from this subspace. It is used within an autoencoder. The encoder maps the data into a latent space, from which the RSR layer extracts the subspace. The decoder then smoothly maps back the underlying subspace to a "manifold" close to the original inliers. Inliers and outliers are distinguished according to the distances between the original and mapped positions (small for inliers and large for outliers). Extensive numerical experiments with both image and document datasets demonstrate state-of-the-art precision and recall.

After reading all the reviews and the comments, I feel more positive about the paper. I appreciate the feedback of the Authors and I have decided to increase the rating.
============================
The paper proposes using Robust Subspace Recovery in combination with an autoencoder (and possibly GANs) for anomaly detection. The encoder maps input data to the latent space of dimensionality D, which then is linearly projected to a subspace of dimensionality d (d < D). The projection of the latent space then goes to a decoder that reconstructs the input.
A transformation matrix A is trained jointly with the autoencoder. Two additional terms are added to the loss: one to encourage the subspace of A^TA to approximate the latent space z and the second one to force it to be an orthogonal projector.

The paper claims to generalize the existing RSR framework to the nonlinear case. However, the linear RSR is applied to the latent space of the autoencoder. In addition to that, all the following discussion and proofs are limited to the linear case.
Since the proposed method is using RSB as it’s core part, and claims to be a non-linear extension of it, it would be crucial to have a comparison with RSB, at least on those experimental setups, where high-level features are used (Tiny Imagenet with ResNET features, Reuters-21578, and 20 Newsgroups). However, there is no such comparison.
Since autoencoders can potentially learn any, arbitrary entangled latent space, it is not clear why outliers should necessarily have such embedding that is outside of the learned subspace. In the case of the original RSR it happens due to the dimensionality reduction by the orthogonal projector. However, autoencoders already perform dimensionality reduction at each layer down to the bottleneck layer.
The matrix A and the parameters of the AE are trained jointly. So, it can be seen that two processes can occur:
- The AE in order to minimize the reconstruction error would learn such latent space z, that would fit into the subspace of A^TA, so that projection \tilde z =Az doesn’t cause data loss.
- The AE in order to minimize the reconstruction error would learn such A, so that the subspace that z approximates is the best possible.
It is not clear, which of the two cases would take place. If the first one would dominate, then it is not clear if such method would have any discriminating capabilities.

My point is mainly that the presented work is not really a generalization of RSR as it claims to be, but rather it is just using RSR on a leaned embedding of the data.
Some citations are missing, as well as it is missing a comparison to some state-of-the art methods such as OCNN ‘Robust, Deep and Inductive Anomaly Detection’ ECML 2017;   ‘Adversarially Learned One-Class Classifier for Novelty Detection’ CVPR 2017; DSVDD ‘Deep one-class classification.’ ICML, 2018; ODIN  ‘Enhancing The Reliability  Of Out-of-distribution Image Detection  In Neural Networks’ ICLR 2018; ‘Generative Probabilistic Novelty Detection with Adversarial Autoencoders’ NeurIPS 2018.


Automatically feature generation is a major topic of automated machine learning. Among various feature generation approaches, feature crossing, which takes cross-product of sparse features, is a promising way to effectively capture the interactions among categorical features in tabular data. Previous works on feature crossing try to search in the set of all the possible cross feature fields. This is obviously not efficient when the size of original feature fields is large. Meanwhile, some deep learning-based methods combines deep neural networks and various interaction components. However, due to the existing of Deep Neural Networks (DNN), only a few cross features can be explicitly generated by the interaction components. Recently, piece-wise interpretation of DNN has been widely studied, and the piece-wise interpretations are usually inconsistent in different samples. Inspired by this, we give a definition of interpretation inconsistency in DNN, and propose a novel method called CrossGO, which selects useful cross features according to the interpretation inconsistency. The whole process of learning feature crossing can be done via simply training a DNN model and a logistic regression (LR) model. CrossGO can generate compact candidate set of cross feature fields, and promote the efficiency of searching. Extensive experiments have been conducted on several real-world datasets. Cross features generated by CrossGO can empower a simple LR model achieving approximate or even better performances comparing with complex DNN models.

The paper presents a scheme to generate new features as cross-product
of binary features to improve the performance of linear models while
obtaining interpretable models. The candidate set of cross-features
can be exponential and is handled by the proposed scheme by utilizing
the gradient-based importances of the features in a (deep) neural
network. Features with large discrepancies in their local and global
interpretations are used as the seed set of candidate features for
generating new cross-features, and the final step performs a feature
selection to further reduce the final set of cross-features. The
empirical evaluation demonstrates the utility of the proposed scheme
on 8 datasets.
While the proposed scheme does present a way to improve the accuracy
of interpretable models, I am currently recommending a reject for the
following reasons (given the higher standard recommended for papers
over 8 pages):
- While this paper does consider some baselines, it seems to be
  missing some crucial baselines that address the same (or very
  similar) problem. There are some papers [2,3] that learn boolean
  conjunctions (that can be seen as cross-features) to generate
  accurate interpretable models. Moreover, there are some search based
  feature generation schemes [1,4] that significantly improve upon the
  exhaustive feature generation scheme of Kanter & Veeramacheneni,
  2015. This technique can easily be applicable in learning boolean
  cross features with binary features. At the very least, it is important to
  understand where this proposed scheme is positioned relative to the
  aforementioned literature and why a comparison is not required.
- It is very unintuitive (at least to me) to tie the candidate
  generation scheme to a neural network especially given the
  sensitivity of neural network training to different initializations
  and other factors. For the same data and neural network, the local
  vs. global discrepancies can change significantly, thereby changing
  the candidate set of cross features. This can potentially make the
  proposed feature generation scheme somewhat unstable, and the
  interpretations from the subsequent models might not be as
  interpretable as they seem. It would be good to understand what I am
  missing here and why being tied to a neural network model is
  essential and not an issue here.
Clarification:
- Lines 8-10 in Algorithm 1 is not clearly explained.
- The experiment to motivate Assumption 1 needs to be better
  explained.
Minor:
- The notation in equation (1) needs to be clarified better.
[1] Khurana, Udayan, et al. "Cognito: Automated feature engineering
for supervised learning." 2016 IEEE 16th International Conference on
Data Mining Workshops (ICDMW). IEEE, 2016.
[2] Dash, Sanjeeb, Oktay Gunluk, and Dennis Wei. "Boolean decision
rules via column generation." Advances in Neural Information
Processing Systems. 2018.
[3] Wei, Dennis, et al. "Generalized Linear Rule Models." Proceedings
of the 36th International Conference on Machine Learning. 2019.
[4] Khurana, Udayan, Horst Samulowitz, and Deepak Turaga. "Feature
engineering for predictive modeling using reinforcement learning."
Thirty-Second AAAI Conference on Artificial Intelligence. 2018.


Group convolutional neural networks (G-CNNs) can be used to improve classical CNNs by equipping them with the geometric structure of groups. Central in the success of G-CNNs is the lifting of feature maps to higher dimensional disentangled representations in which data characteristics are effectively learned, geometric data-augmentations are made obsolete, and predictable behavior under geometric transformations (equivariance) is guaranteed via group theory. Currently, however, the practical implementations of G-CNNs are limited to either discrete groups (that leave the grid intact) or continuous compact groups such as rotations (that enable the use of Fourier theory). In this paper we lift these limitations and propose a modular framework for the design and implementation of G-CNNs for arbitrary Lie groups. In our approach the differential structure of Lie groups is used to expand convolution kernels in a generic basis of B-splines that is defined on the Lie algebra. This leads to a flexible framework that enables localized, atrous, and deformable convolutions in G-CNNs by means of respectively localized, sparse and non-uniform B-spline expansions. The impact and potential of our approach is studied on two benchmark datasets: cancer detection in histopathology slides in which rotation equivariance plays a key role and facial landmark localization in which scale equivariance is important. In both cases, G-CNN architectures outperform their classical 2D counterparts and the added value of atrous and localized group convolutions is studied in detail.

The paper proposes an (approximately) equivariant neural network architecture for data lying on homogeneous spaces of Lie groups. In contrast to the Gauge equivariant and Fourier approaches that have recently appeared, here the authors simply put a B-spline basis on local patches of the homogeneous space and move the basis elements around explicitly by applying the group action.
The approach is appealing in its simplicity and generality. No need to worry about irreducible representations and Fourier transforms, the formalism works for virtually any Lie group, no problem with non-compact groups. However, there is a constant need for interpolation. What is more more significant is that both the homogeneous space and the group need to be discretized and in general that cannot be done in a regular manner (no notion of a uniform grid on SO(3) for example). The authors assure us that "we find that it is possible to find approximately uniform B-splines... e.g. by using a repulsion model". I am not sure that it is so simple. This is one of those things where the idea is straightforward but the devil is in the details.
Theorem 1 seems important but it is a bit cryptic. What is the statement "a kernel satisfying such and such properties gives rise to an equivariant CNN"? Or "A CNN is equivariant if and only the kernel satisfies such and such properties"?
Concerningly, the paper is closely related to a few other papers using the spline CNN idea or at least the idea of taking a fixed set of functions and moving it around on the homogeneous space by acting on it with select group elements, most notably "Roto-translational convolutional neural networks for medical image analysis" by Bekkers et al.. The main difference of the present paper relative to that one is that the idea is fleshed out in a little more detail and is generalized from SE(2) to arbitrary Lie groups. However, conceptually there is little that is new.
In such a situation it would be important to present convincing experiments. Unfortunately in the present paper, results are only presented on 2 datasets, and the algorithm is basically only compared to different versions of itself, rather than state of the art competitors.
The paper is clearly written but the intuitive nature of the core ideas could be better conveyed e.g. by fancy diagrams.


Computational methods that infer the function of proteins are key to understanding life at the molecular level. In recent years, representation learning has emerged as a powerful paradigm to discover new patterns among entities as varied as images, words, speech, molecules. In typical representation learning, there is only one source of data or one level of abstraction at which the learned representation occurs. However, proteins can be described by their primary, secondary, tertiary, and quaternary structure or even as nodes in protein-protein interaction networks. Given that protein function is an emergent property of all these levels of interactions in this work, we learn joint representations from both amino acid sequence and multilayer networks representing tissue-specific protein-protein interactions. Using these hybrid representations, we show that simple machine learning models trained using these hybrid representations outperform existing network-based methods on the task of tissue-specific protein function prediction on 13 out of 13 tissues. Furthermore, these representations outperform existing ones by 14% on average.

This paper introduces a method to incorporate both sequence information and graph information to learn the protein representations. The idea is very straightforward. Basically, it used the embedding from OhmNet [Marinka et al, 2017] for the graph information and used the sequence information from UniRep [Ethan et al, 2019] or SeqVec [Michael et al, 2019]. It uses one experiment to show the performance of the combination of the two pieces of information.
This paper should be rejected for the following reasons:
(1) The paper is obviously in the preliminary form without too much polish.
 (2) The simple combination of the results from two published articles is not that interesting
(3) the presentation of the paper and idea is not in an acceptable form (the authors should at least draw a figure to show the big idea of the paper).
(4) the experiment is not convincing (there is only one experiment and it is not compared with the other state-of-the-art methods; since an embedding of a protein can be of broad usage, the authors should give its performance on four tasks: protein function prediction (GO term) [Maxat et al, 2018], enzyme function prediction (EC number) [Yu et al, 2018], protein secondary structure prediction [Sheng et al, 2016], protein contact map prediction [Jinbo Xu, 2019])
(5) The learned embedding is not well discussed. The author should at least visualize the embeddings and check the physical and biological meaning of those embeddings, if possible.
Since this manuscript would be for sure and have to be largely rewritten in the future, I would not give too many detailed suggestions but some high-level suggestions if the authors would like to refine this manuscript further and submit it somewhere else or ICLR next year:
(1) Further improve the idea of combining different sources of information. Combining different pieces of information will definitely be helpful but the authors should figure out a way to use them in a more natural way.
(2) Compared with other methods, which can combine different sources of information.
(3) Run more experiments on various tasks instead of one: protein function prediction (GO term), enzyme function prediction (EC number), protein secondary structure prediction, protein contact map prediction
(4) Refine the representation of the paper.
References:
[Marinka et al, 2017] Predicting multicellular function through multi-layer tissue networks, 2017, https://arxiv.org/abs/1707.04638
[Ethan et al, 2019] Unified rational protein engineering with sequence-based deep representation learning, 2019, Nature Methods
[Michael et al, 2019] Modeling the Language of Life – Deep Learning Protein Sequences, 2019, https://www.biorxiv.org/content/10.1101/614313v2
[Maxat et al, 2018] DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier, 2018, Bioinformatics
[Yu et al, 2018] DEEPre: sequence-based enzyme EC number prediction by deep learning, 2018, Bioinformatics
[Sheng et al, 2016] Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields, 2016, Scientific Reports
[Jinbo Xu, 2019] Distance-based protein folding powered by deep learning, 2019, PNAS


In many reinforcement learning settings, rewards which are extrinsically available to the learning agent are too sparse to train a suitable policy. Beside reward shaping which requires human expertise, utilizing better exploration strategies helps to circumvent the problem of policy training with sparse rewards. In this work, we introduce an exploration approach based on maximizing the entropy of the visited states while learning a goal-conditioned policy. The main contribution of this work is to introduce a novel reward function which combined with a goal proposing scheme, increases the entropy of the visited states faster compared to the prior work. This improves the exploration capability of the agent, and therefore enhances the agent's chance to solve sparse reward problems more efficiently. Our empirical studies demonstrate the superiority of the proposed method to solve different sparse reward problems in comparison to the prior work.

This paper studies the problem of exploration in reinforcement learning. The key idea is to learn a goal-conditioned agent and do exploration by selecting goals at the frontier of previously visited states.  This frontier is estimated using an extension of prior work (Pong 2019). The method is evaluated on two continuous control environments (2D navigation, manipulation), where it seems to outperform baselines.
Overall, I like that the proposed method integrates some notion of novelty with the language of mutual information and density estimation. While this idea has been explored in prior work (as noted in the related work section), the proposed idea seems like a useful contribution to the literature. The use of convolutions to obtain a lower bound on mutual information seems neat. The experimental results are quite strong.
My main concern with the paper is a lack of clarity. I currently have enough reservations and questions (listed below) about the experimental protocol that I am learning towards rejecting this paper. However, if the paper clarified the concerns below, I'd be willing to increase by review.
Questions / Concerns:
* "[Prior work on mutual-information cannot] guarantee that the entire state space can be covered" -- In theory, I think these prior methods should cover the entire state space. Take the DIAYN objective, I(s, z) = H[s] - H[s | z]. This objective is maximized when p(s) is uniform over the entire state space and p(s | z) is a Dirac.
* "It is time consuming to collect enough samples to estimate an accurate state entropy" -- Can you provide a citation/proof for this? Why should we expect the proposed method to require fewer samples?
* "...entropy itself does not provide efficient information to adjust the action at each step." -- Can you provide a citation / proof for this? Also, what does "efficient information" mean?
* It seems like, if S is a finite collection of states in a continuous space, then it has measure zero, so its entropy should be zero. Can you explain why this is not the case?
* If I'm not mistaken, in equation 2, if we take (say) alpha = -0.5, then w_i is proportional to sqrt(p(s_i)), so w_i is an increasing function in p(s_i), not a decreasing function.
* Can you discuss how you might scale a KDE to high dimensions?
* "If the distribution has a larger range, the entropy is larger as well." -- Technically, this is not correct. You can construct distributions with larger ranges but smaller entropies.
* I think that Equation 3 should be the KL divergence between state marginal distributions, not between trajectories. If it were the KL between trajectories, it would include actions and policy terms.
* How are w_int and w_ext chosen? It seems like the method depends critically on the balance between these hyperparameters. Is w_int decayed over time? If not, why does the policy stop exploring once it has found the goal?
* What policy is used at convergence? It seems like the policy is conditioned on Z_t, so how is the Z_t chosen for evaluation?
* Fig 5 -- How are entropy and coverage computed? What are the maximum possible values for both of these quantities? What precisely does the X axis correspond to?
* Table 1 -- How did the baseline algorithms perform on this task?
* Fig 7 -- How did the baseline algorithms perform on this task? If the reward were sparse, shouldn't the Y axis be in the interval [0, 1]?
* "using coverage only during training is not suitable" -- Can you provide a citation/proof for this?
* "As a consequence, the entropy of the distribution of these points is also maximized" -- I believe that a finite number of points in a discrete space have measure zero, so they have zero entropy, regardless of the position of the points.
Other comments
* "What is the difference between *S* (in bold) and S_t?
* I would recommend using some notation other than p(s) to denote the smoothed/convolved density.
* "history states" -- I was confused about what this meant until it was introduced two sections later.
* "assimilate the definition of curiosity in psychology" -- I think that others (e.g., Oudeyer 2007, Pathak 2017) have noted the similarities between curiosity in humans and RL agents.
* Check for backwards quotes in the related work section.
* "Self-Goal Proposing" -- Some more related works are [Florensa 2017, Savinov 2018]
* "space associated environment" -- I don't know what this means.
* "disc rewards" -- I'd recommend spelling out discounted
* "truncated Gaussian function with a narrow range" -- Can you explain precisely what this is?
* In equation 2, I think it'd be clearer to write p^(1+\alpha).
* For the experiment on the effect of variance, I'd recommend making a plot instead of just listing the values.
* In Section 4.3, it's unclear whether the physical robot was successful at solving the task.
* "We rewrite the equation…" -- This paragraph is repeated.
* Double check that \citet and \citep are used properly
--------------UPDATE AFTER AUTHOR RESPONSE------------------
Thanks for answering many of my questions. This was helpful for clarifying my understanding. However, since a large fraction of my concerns were not addressed, so I am inclined with stick with my original vote to reject the paper. Nonetheless, I should emphasize that I think this paper is on the right track and the empirical results seems strong. With a bit more work on writing, I think it would be a fantastic paper at the next conference.