State-of-the-art models are now trained with billions of parameters, reaching hardware limits in terms of memory consumption. This has created a recent demand for memory-efficient optimizers. To this end, we investigate the limits and performance tradeoffs of memory-efficient adaptively preconditioned gradient methods. We propose extreme tensoring for high-dimensional stochastic optimization, showing that an optimizer needs very little memory to benefit from adaptive preconditioning. Our technique applies to arbitrary models (not necessarily with tensor-shaped parameters), and is accompanied by regret and convergence guarantees, which shed light on the tradeoffs between preconditioner quality and expressivity. On a large-scale NLP model, we reduce the optimizer memory overhead by three orders of magnitude, without degrading performance.

============= Update after rebuttal
Thanks for the clarifications; I have updated by score and recommend acceptance.
Please do not forget to implement the promise changes in the camera ready version (e.g. about L-BFGS tradeoff work; as well as gap between OCO and non-convex framework).
==============
Motivated from NLP applications where models with billions of parameters are used, this paper proposes a *memory efficient* variant of Adagrad by maintaining a rank-one tensor approximation of the "second-moment" accumulator normally used in Adagrad. From the theoretical side, a simple regret bound is proved which provides an intuitive quantity quantifying the convergence guarantee loss for memory-efficient version (vs. Adagrad), and which is empirically evaluated to be small on a large-scale language modeling task (Section 5.3). From the empirical side, the memory vs. generalization performance tradeoff is evaluated on this language modeling task, showing that similar performance to Adagrad can be obtained with much a smaller (optimization overhead) memory footprint. A convex toy task also indicate a similar result.
I like this paper, I am leaning towards acceptance. The write-up can be improved in a few places (see detailed comment below); but overall, I find the idea refreshing for optimization and the proof is simple and elegant.
The main motivation for the paper is that these large models used in NLP are often making us hit the memory limitation of the hardware just to store the model, and so one then has to tradeoff the size of the model with the memory requirement of the *optimization* algorithm (e.g. one step-size accumulator per dimension for Adagrad). Trying to get the gains of an adaptive preconditioned gradient method but with lower memory footprint thus seems valuable (and Section 5.2 which compares doubling the size of the model + using their memory efficient method vs. original model + Adagrad highlights the gains one can get).
On the other hand, I wish the paper provided a bit more intuition on when the rank-one tensor approximation structure will give good results (e.g. Section 5.3 gives an empirical measure; but what about some simple theoretical examples which give low values, to provide more intuitions?). In particular, the main inequality relating the Adagrad step-size with the "extreme indexing" step-size is arising from the first equation in the proof of Lemma 4.3 on p.5 -- basically one replace one entry of the squared gradient to store with the whole sum over all entries of the squared gradient except a slice. This inequality can be very loose (O(d) in the worst-case), so it is surprising that one could get good approximation ratios much better than the O(sqrt(d)) worst-case (a square-root is taken afterwards), and I wonder what structure yields this.
I also note that Anil et al. (2019) tackles a similar problem with a different approach, and so ideally a more detailed comparison (theoretically and empirically) would be provided (rather than just one sentence as in the current submission), especially since this appeared on arXiv 9 months before the ICLR deadline. But the authors presented it as concurrent work (and it does not appear published anywhere yet) [and a Google search shows the authors had a first version on arXiv only a few weeks after this one, so this seems righs], and so I decided to not hold it against this submission. Disclaimer: I made my general opinion about the paper before doing these Google searches and finding the paper on arXiv.
I am not very familiar with the modern transformer architecture experiments, so cannot evaluate their quality, but they seem sensible from an outsider's perspective.
== Detailed comments ==
- p.1 "the first empirical study of the tradeoff between training convergence and memory in the optimizer." This should be properly qualified by "in the Adagrad setup" or something like that; I am pretty sure it is false *for general optimizers*. For example, I guess there are several papers which empirically studied the memory - performance tradeoff for L-BFGS optimizers...
- Important: I think the paper should be more transparent about the *big gap* between the online *convex* optimization framework and stochastic optimization for a *non-convex* loss. There is only one sentence at the end of Section 2.1 mentioning vaguely that Agarwal et al. (2018) provide a reduction from stochastic non-convex setting to the online convex setting, but this is buried in the appendix of the said paper, with several caveats, and for a modified algorithm (i.e. as far as I understand, no convergence guarantee would be given for Algorithm 1 to find a stationary point on a non-convex loss; but one could apply Algorithm 1 on a series of convex problems to obtain overall guarantees on the non-convex loss [but this is a different algorithm!]). I suggest that either the argument of Agarwal et al. (2018) is summarized in a few sentences at the end of Section 2.1 to clarify the real link; or give more caveats [also just before 4.1] (e.g., my current guess is that we use the convex analysis setup to gain insights on the behavior in a controlled setup; and then just hope that some of it applies in the non-convex setup, even though no guarantees is provided whatsoever).
- Some undefined notation: line 8 of Algorithm 1, it seems they use the dot for the Hadamard product between two vectors. The big dot in Lemma 4.5 is most likely the dot product between matrices (but please mention explicitly for clarity).
- Typos in Section 5.1 & 5.2: several references seem wrong. E.g. Figure 5 is probably Figure 2; Table 4 is Table 1, etc... Please correct!
- Clarity: I suggest to put much more description in the captions of Figure 2, Table 2, etc. (for example it was much clearer in their arXiv version). I was very confused by the extra light blue dot in Figure 2 which is only explained in 5.2; I suggest that it is already mentioned (with forward pointer) in the caption. Use more of p.9 for the clarity sake...
- Appendix B.1: please provide hardware information when you give wall-clock comparison.


Ensuring robustness of Deep Neural Networks (DNNs) is crucial to their adoption in safety-critical applications such as self-driving cars, drones, and healthcare. Notably, DNNs are vulnerable to adversarial attacks in which small input perturbations can produce catastrophic misclassifications. In this work, we propose EMPIR, ensembles of quantized DNN models with different numerical precisions, as a new approach to increase robustness against adversarial attacks. EMPIR is based on the observation that quantized neural networks often demonstrate much higher robustness to adversarial attacks than full precision networks, but at the cost of a substantial loss in accuracy on the original (unperturbed) inputs. EM-PIR overcomes this limitation to achieve the "best of both worlds", i.e., the higher unperturbed accuracies of the full precision models combined with the higher robustness of the low precision models, by composing them in an ensemble. Further, as low precision DNN models have significantly lower computational and storage requirements than full precision models, EMPIR models only incur modest compute and memory overheads compared to a single full-precision model (<25% in our evaluations). We evaluate EMPIR across a suite of 3 different DNN tasks (MNIST, CIFAR-10 and ImageNet) and under 4 different adversarial attacks. Our results indicate that EMPIR boosts the average adversarial accuracies by 43.6%, 15.3% and 11.9% for the DNN models trained on the MNIST, CIFAR-10 and ImageNet datasets respectively, when compared to single full-precision models, without sacrificing accuracy on the unperturbed inputs.

This paper suggests using ensemble of both full-precision and low-bits precision models to defense adversarial examples.
From methodological point of view, this idea is quite straightforward and not novel, since there are already several works that applied ensemble methods to improve the robustness of NNs, including the Strauss et.al 2017 and (the following references are not included in the manuscript)
"Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song"
"Ensemble Adversarial Training: Attacks and Defenses
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel" .
"Improving Adversarial Robustness via Promoting Ensemble Diversity
Tianyu Pang, Kun Xu, Chao Du,  Ning Chen,  Jun Zhu " ICML 2019
Though these methods only considered combining full-precision models, the idea is the same in essence and let the low-bits networks involve into the ensemble is quite natural and straightforward. So I don't think the methodology contribution of this paper is enough for publication.
When checking the empirical results, the compared baselines miss a very common-used and strong baseline PGD adversarial training. And also the performance of this ensemble is not significant.
Considering the weakness of the paper both in methodology development and empirical justification, this work does not merit publication from my point of view.


Neural Architecture Search (NAS) aims at automatically finding neural network architectures within an enormous designed search space. The search space usually contains billions of network architectures which causes extremely expensive computing costs in searching for the best-performing architecture. One-shot and gradient-based NAS approaches have recently shown to achieve superior results on various computer vision tasks such as image recognition. With the weight sharing mechanism, these methods lead to efficient model search. Despite their success, however, current sampling methods are either fixed or hand-crafted and thus ineffective. In this paper, we propose a learnable sampling module based on variational auto-encoder (VAE) for neural architecture search (NAS), named as VAE-NAS, which can be easily embedded into existing weight sharing NAS framework, e.g., one-shot approach and gradient-based approach, and significantly improve the performance of searching results. VAENAS generates a series of competitive results on CIFAR-10 and ImageNet in NasNet-like search space. Moreover, combined with one-shot approach, our method achieves a new state-of-the-art result for ImageNet classification model under 400M FLOPs with 77.4% in ShuffleNetlike search space. Finally, we conduct a thorough analysis of VAENAS on NASbench-101 dataset, which demonstrates the effectiveness of our proposed methods.

============ comments after rebuttal
I would like to thank the authors for addressing some of my concerns. I believe the new results under Q2 and Q3 are useful additions to strengthen the paper.
As for the authors' comments for Q1, I'd like to point out that a "larger" search space is not necessarily more difficult (a more meaningful metric would be the average accuracy of random architectures). It is still possible that the current search space is putting the proposed method at advantage, especially if having dense connections is a useful prior.
Given the above, I would like to increase my score from 1 to 3 (weak reject).
============ previous comments
Neural architecture search can be formulated as learning a distribution of promising architectures (the sampling policy). Such a distribution is usually represented in a fully factorized fashion (e.g., as a set of multinational distributions as in DARTS). This paper proposes to model the architecture distribution using a VAE instead, where the encoder and decoder are implemented using LSTMs. The authors argue that the increased flexibility of the sampling policy leads to improved performance on CIFAR-10, NASBench and ImageNet.
The idea of representing the architecture distribution using VAEs is very natural, which in principle could offer better coverage over interesting regions in the search space as compared to traditional factorized distribution representation (which has a single mode only).
While the method itself is interesting, I do not think it has been properly backed up by controlled experiments. This is largely due to the fact that the authors are comparing their method against baselines in fundamentally different search spaces. For instance:
* For CIFAR-10 experiments, the authors mentioned in the appendix: "Different from DARTS, in our search space, one node could have more than two predecessors in one cell". This makes the search space very different from the existing ones as used by NASNet/AmoebaNet/DARTS/SNAS, and it hence remains unclear to what degree the resulting architecture has benefited from the increased in-degrees per node. Note the searched densely connected cells in Figure 4 & 5 in Appendix A.4 are clearly not part of the search space for many of the baselines.
* For ImageNet experiments, the authors are using a ShuffleNet-like search space which has fundamentally different building blocks than other architecture search baselines (commonly built on top of inverted bottleneck layers). It is unclear to what degree the 77.4 top-1 accuracy @ 365 MFlops results have benefited from this different search space.
Without fair comparisons in a controlled setup, it is impossible for readers to draw any solid conclusion about the true empirical advantages of the method. I'm therefore unable to recommend acceptance for the paper at the moment, but am willing to raise my score if the authors can properly address those issues in the rebuttal.
Additional question: How can we isolate it to tell whether the gains come from the LSTMs or the VAEs? Is there any intuition why incorporating a generative sampler based on VAEs is potentially superior to method like ENAS (which involves LSTMs decoders only)?


Network structures are important to learning good representations of many tasks in computer vision and machine learning communities. These structures are either manually designed, or searched by Neural Architecture Search (NAS) in previous works, which however requires either expert-level efforts, or prohibitive computational cost. In practice, it is desirable to efficiently and simultaneously learn both the structures and parameters of a network from arbitrary classes with budgeted computational cost. We identify it as a new learning paradigm -Boosting Network, where one starts from simple models, delving into complex trained models progressively. In this paper, by virtue of an iterative sparse regularization path --Split Linearized Bregman Iteration (SplitLBI), we propose a simple yet effective boosting network method that can simultaneously grow and train a network by progressively adding both convolutional filters and layers. Extensive experiments with VGG and ResNets validate the effectiveness of our proposed algorithms.

This paper proposes an architecture search method for deep convolutional neural network models that progressively increases the number of filters per layer as well as the number of layers, and the authors refer to this general approach as boosting networks. The algorithm for increasing the number of filters is based on split linear Bregman iteration, and the algorithm for increasing the number of layers proceeds block by block, increasing the layers per block until the accuracy does not increase. The experiments convincingly demonstrate gains in performance and smaller network sizes compared to baseline models, naive boosting methods, and a related method called Autogrow.
In my view, there are two main areas for further improvement for this work. First, the GT-Layers algorithm can be better motivated. Why is GT-Filters only run once at the beginning, rather than iteratively as the number of layers increases? Why does the procedure go from bottom blocks to up blocks (and, by the way, what are bottom blocks and up blocks)? Why measure training accuracy to determine when to add layers? I understand that these are all fairly heuristic choices, but nevertheless there needs to be proper motivation for all of the above.
Second, an additional effort should be made to compare to additional prior work in architecture search. It seems like the authors are suggesting that this method should be more computationally efficient and find smaller architectures, and demonstrating this empirically would greatly strengthen the paper. In particular, the existing results depicting the final number of parameters in the learned model are particularly striking to me. I appreciate that the authors included an experiment showing that a standard ResNet cannot be trained with the same number of FLOPs as the network found by your method. Can a similar analysis be made for wall clock time, i.e., how long the models actually take to train? A similar study (FLOPs, wall clock time, etc.) would also be very useful for the current comparison to Autogrow, as these metrics are often just as important, if not more important, than the number of parameters of the final model.
A thorough pass through the paper for spelling and grammar would be very useful.


Generating reliable illumination and viewpoint invariant keypoints is critical for tasks such as feature-based SLAM and SfM. Recently, several learned keypoint methods have demonstrated compelling performance on challenging benchmarks. However, generating consistent and accurate training data for interest-point detection in natural images still remains challenging, especially for human annotators. We introduce IO-Net (i.e. InlierOutlierNet), a novel proxy task for the self-supervision of keypoint detection, description and matching. By making the sampling of inlier-outlier sets from point-pair correspondences fully differentiable within the keypoint learning framework, we show that are able to simultaneously self-supervise keypoint description and improve keypoint matching. Second, we introduce KeyPointNet, a keypoint-network architecture that is especially amenable to robust keypoint detection and description. We design the network to allow local keypoint aggregation to avoid artifacts due to spatial discretizations commonly used for this task, and we improve fine-grained keypoint descriptor performance by taking advantage of efficient sub-pixel convolutions to upsample the descriptor feature-maps to a higher operating resolution. Through extensive experiments and ablative analysis, we show that the proposed self-supervised keypoint learning method greatly improves the quality of feature matching and homography estimation on challenging benchmarks over the state-of-the-art.

# UPDATE following rebuttal
Score increased to 6 due to architecture details in supplemental.
# Contributions
The paper contributes a self-supervised method of jointly learning 2D keypoint locations, descriptors, and scores given an input RGB image. The paper builds on previous work, adding:
* A more expressive keypoint location regression, which allows each 8x8 pixel region to vote for a keypoint location outside its boundary
* An upsampling step, similar to a U-net, to allow descriptors to be regressed with more detailed information
* An additional proxy task for the total loss, based on outlier rejection.
The authors train on COCO by manually distorting images to generate pairs with known homography, and show competitive results for keypoint detection and homography estimation tasks.
Decision: Weak reject. I would give this a 5 if the website allowed me to. A more detailed explanation of the neural network architecture, along with minor fixes described below, would make me increase my rating.
I feel the additions to existing pipelines are well motivated but insufficiently explained. In particular, the explanation of the neural network architecture along with figures 1 and 2 leaves many details unclear to me. Phrases like "a 1D CNN ... with 4 default setting residual blocks" is to me insufficient - residual networks have many details such as Resnet V1 or V2 style (ie is there a path right through the network which doesn't hit any activation functions), what kind of normalization is applied, number of channels in each block, how to do skips between different spatial resolutions, etc. The upsampling step for the descriptor head, which is claimed as a novel contribution, is not fully explained - "fast upsampling" implies (correctly) there are many variants of upsampling with different tradeoffs, but from the text I am unsure whether this is nearest neighbour upsampling, a ConvTranspose, etc. Similarly, "VGG style block" leaves some details unclear - whether the resolution downsampling is with a strided convolution / pooling / etc. Lots of the details are implied to be in other previous papers, but I feel that the paper would be hugely improved by exact architectural details.
There are various minor notational discrepancies in the paper - for example the outlier rejection is various defined as "InlierOuterNet (IONet)" and "The Inlier-Outlier model \emph{IO}", which also seems to be the same as the function $C$ defined a paragraph above. Perhaps it is common in this part of the literature, but to me an encoder decoder network is more likely to either be an autoencoder, or for the decoder to output something in the same modality (eg in machine translation). To say that some VGG blocks are an encoder, and the heads which produce keypoint locations / score / descriptor is a decoder, implies all neural networks could be described as an encoder/decoder.
The two figures showing the architecture are very different in design, which is not in itself a problem but the relationship between them could be clearer. I feel that the 'matching' box in figure 1 is misleading because it implies that matching only happens for the IONet, but the loss function for location described in Eq 1 also requires matching keypoints between the image pair. I'm also unclear on the division between direct and indirect supervisory signal - all the 4 loss components have a clear purpose, but it's not obvious what this partitioning means. "Indirect" only appears in this figure and the caption - perhaps.
The term "Anchor" appears only once with no reference, below equation 3 - I appreciate this is an existing term in this subfield, but given that the start of section 3 goes as far as explicitly defining what it means to produce 2D keypoints for an image, I feel defining this term would make the descriptor loss much clearer.
One of the main contributions, that of allowing locations to regress outside their 8x8 area, sounds like a good idea but I feel that Figure 3 does not adequately show the benefit. In both a) and b), the blue estimates appear to be roughly as good as each other - clearly from the ablation a large benefit is gained from this innovation but perhaps a better illustrative example could be made here?
On a more positive note, I feel the components of the loss function are in general very clearly motivated and defined, and the description of training & data augmentation hyperparameters appears complete. If the description of the architecture could be improved that would result in a paper very amenable to reproduction.
The experiments are well explained, and the ablation of the various proposed  components is good. I feel table 1 would be improved with error bars - given that the bold best score is not exclusively next to V4, but in many cases the difference between V4 and the best is ~1%, error bars from different training runs might make clearer that V4 is overall the best configuration.
In the conclusion - "even without an explicit loss" - what is the difference between the loss functions used in this work, and an explicit loss?
Minor corrections:
The euclidean distance between descriptors is various notated as $d$ (section 3), $x$ (above equation 5) and $d$ again (below equation 5).
Typos: "normalzation" -> "normalization", "funcion" -> "function", "tripled" -> "triplet".


We introduce CuLE (CUDA Learning Environment), a CUDA port of the Atari Learning Environment (ALE) which is used for the development of deep reinforcement algorithms. CuLE overcomes many limitations of existing CPU-based emulators and scales naturally to multiple GPUs. It leverages GPU parallelization to run thousands of games simultaneously and it renders frames directly on the GPU, to avoid the bottleneck arising from the limited CPU-GPU communication bandwidth. CuLE generates up to 155M frames per hour on a single GPU, a finding previously achieved only through a cluster of CPUs. Beyond highlighting the differences between CPU and GPU emulators in the context of reinforcement learning, we show how to leverage the high throughput of CuLE by effective batching of the training data, and show accelerated convergence for A2C+V-trace. CuLE is available at [URL revealed upon acceptance].

The work contributes a library emulating Atari games in GPU in parallel and allowing to speed-up the execution of reinforcement learning algorithms.
I see that the paper qualifies to the conference; in particular there is listed the topic:
- “implementation issues, parallelization, software platforms, hardware”
However, this is not a research paper, and I do not really see how I should asses it. What I can say about it is that it is considerable amount of work, not only implementing the simulator but also looking at what RL methods need, and how to optimize the allocation and exchange of the data so that everything would work on GPU more efficiently.
From the practical perspective, I am somewhat confused. The speed-up factors in the experiments are rather modest: about 4x for simulating and rendering frames, 2.5x for full RL, on a single GPU. Better with scaling to multi-GPU systems. In Table 1 the total training time per resources used differs dramatically. However if I look at the lines with A2C it is about the same time with 100-200 CPU cores + 1 GPU versus 12 cores + 1 GPU. So this is about factor 10 in the resources, versus CPU parallelization probably suffering overheads.
It appears that the maximum steed-ups are achieved for a particular type of the reinforcement learning algorithms, and using it in a general case would give a modest improvement.
The paper itself consists of introduction, related work, 1 page overview of what it means to simulate the Atari games, and experiments. So it is mostly about measuring the speedups, with several implementations / platforms.
I tend to think that this work will not very much boost the research for new RL methods. It is limited to Atari games, mostly helps to sample-inefficient RL methods and if it helps, the speed-up factors are not of the order that would make experiments by the researchers otherwise impossible.
I would also give priority to theoretical contributions at ICLR. In the end, we all are using CUDA and cnDNN, but presentations about how they implement things are rather given at GPU computing conferences.


Vanishing gradients pose a challenge when training deep neural networks, resulting in the top layers (closer to the output) in the network learning faster when compared with lower layers closer to the input. Interpreting the top layers as a classifier and the lower layers a feature extractor, one can hypothesize that unwanted network convergence may occur when the classifier has overfit with respect to the feature extractor. This can lead to the feature extractor being under-trained, possibly failing to learn much about the patterns in the input data. To address this we propose a good classifier hypothesis: given a fixed classifier that partitions the space well, the feature extractor can be further trained to fit that classifier and learn the data patterns well. This alleviates the problem of under-training the feature extractor and enables the network to learn patterns in the data with small partial derivatives. We verify this hypothesis empirically and propose a novel top-down training method. We train all layers jointly, obtaining a good classifier from the top layers, which are then frozen. Following re-initialization, we retrain the bottom layers with respect to the frozen classifier. Applying this approach to a set of speech recognition experiments using the Wall Street Journal and noisy CHiME-4 datasets we observe substantial accuracy gains. When combined with dropout, our method enables connectionist temporal classification (CTC) models to outperform joint CTC-attention models, which have more capacity and flexibility.

=========================
Update review
After reading the authors response I would like to keep my score as is.
I still see many unclear statements, and most importantly I feel that more analysis of the proposed method should have been done here.
=========================
This paper proposed a Top-Down method for neural networks training based on the good classifier hypothesis. In other words, after obtaining a classifier that performs well on the test set, keep fine-tuning / re-learning the data representation.
The authors provide character error rate results for the task of Automatic Speech Recognition using WSJ and CHiME-4 datasets.
Although being an interesting research idea, several issues in this paper make it not yet ready for publication at ICLR.
First, the paper is poorly written; there are many claims the authors are making without providing experiments/proofs/citations.
For example: "...since the feature extractor learns more slowly, then potentially the classifier may overfit the feature extractor before the feature extractor is able to learn much about the underlying pattern of the data...".
Or: "...We suggest that the reason for this is that when all layers are trained on the noisy dataset jointly, the middle layers overfit the bottom-most layers much faster than the bottom-most layers are able to learn input features..."
Next, since there is no theoretical/mathematical explanation of the proposed approach, I expect the authors to run an analysis on the results to better understand the effect of using such an approach. For instance, under which settings this method is most efficient? In what layer should  I start the fine-tuning? Is it better to reinitialize the bottom layers or fine-tune them? Does the proposed approach applicable to different domains? i.e. vision/nlp/other speech/signal processing tasks?  Does the proposed approach applicable to different models or only for the proposed one?
Lastly, although it is not the main point in this paper since all results are reported on ASR, did the authors tried to compute WERs too? That way, people can compare results with other ASR models. The baseline seems relatively weak, at least in Table 1.
Minor comments:
The complexity of the algorithm is written to be O(n). However, this assumes training the model takes O(1) or did I miss something?
Can the authors provide more details/insights regarding the delta differences in Table 1? Did the authors use the same initializations? Did the authors try different ones?


Imitation learning aims to inversely learn a policy from expert demonstrations, which has been extensively studied in the literature for both single-agent settings with Markov decision processes (MDPs), and multi-agent settings with Markov games (MGs). However, existing approaches for MGs only work for synchronous scenarios with all agents simultaneously making decisions at each turn, and do not work for general MGs, allowing agents to asynchronously make decisions in different turns. We propose a novel framework, asynchronous multi-agent generative adversarial imitation learning (AMAGAIL), for general Markov games. The learned expert policies are proven to guarantee subgame perfect equilibrium (SPE), a stronger equilibrium than Nash equilibrium (NE). The experiment results demonstrate that compared to state-of-the-art baselines, our AMAGAIL model can better infer the policy of each expert agent using their demonstration data collected from asynchronous decision-making scenarios.

In this work, a multi-agent imitation learning algorithm for extensive Markov Games is proposed. Compared to Markov Games (MGs), extensive Markov Games (eMGs) introduces indicator variables, which means whether agents will participate in the game at the specific time step or not, and player function, which is a probability distribution of indicator variables given histories and assumed to be governed by the environment, not by the agents. Such a model allows us to consider asynchronous participation of agents, whereas MGs only consider synchronous participation, which is assumed in the existing multi-agent imitation learning algorithms such as MA-GAIL and MA-AIRL.
The contribution of this submission can be summarized as follows. From a theoretical perspective, the submission extends the theorems in MA-GAIL to those in eMGs, where most of them deal with Lagrange multiplier, its meaning, and properties. Followed by Theorem1 and 2, authors define an extensive occupancy measure, a natural extension of occupancy measures in MGs, and cast a multi-agent imitation learning problem into extensive occupancy measure matching problem in Theorem 3. For a practical algorithm, AMA-GAIL is proposed and shown to have a performance gain relative to BC and MA-GAIL.
The submission is highly interesting, but I think section 4 (Practical Asynchronous Multi-Agent Imitation Learning) and section 5 (Experiments) should be much clearly written. The followings are comments regarding those sections:
- It seems that the key difference between MA-GAIL and AMA-GAIL is whether we consider the cost function when the indicator is 0 or not, but it’s difficult to figure out just by comparing (3) (MA-GAIL objective) and (14) (AMA-GAIL objective) at the first glance.
- Similarly in Appendix B, it’s difficult to figure out the difference between MA-GAIL algorithm and AMA-GAIL except the fact that eMGs are assumed. I think some additional explanation is needed.
- How did you generate expert trajectories in eMGs setting? Suppose there is an agent taking an action “1” at time t, but it was not applied to the dynamics because the indicator variable of the agent is equal to 0 at time t. In this case, what would be stored in the expert trajectories? “Null” or “1”? If the agent cannot take an action in advance (before looking at its indicator variable), I think adding indicator variables in a condition of policy, e.g., $\pi(a|s, i)=pi(a|s)$ if $i=1$, otherwise $\mathbb{I}\{a="Null"\}$, is mathematically rigorous.
- Assuming that experts’ trajectories include “Null” actions, how did you use MA-GAIL and BC with those trajectories?
- The performance of BC seems weird to me since adding lots of training data reduces supervised learning errors and can also reduce covariate shift problems in BC since the theorem tells us that the regret is bounded by (error) * (episode length) ^ 2 in the worst case [Ross and Bagnell, “Efficient reductions for imitation learning”]. Such a tendency is empirically shown in MA-GAIL paper as well, i.e., performance of BC increases as the amount of expert trajectories increases. Is there any reason, BC shows poor performance in eMGs?
There are some minor comments:
- In 2.1., $\eta$ (initial state distribution) is not explicitly defined.
- In 2.1., MGs assume each agent’s reward function depends on other agents’ actions as well as agents’ own actions, but in the submission, rewards only depend on agents’ own actions. This may be due to the asynchronous setting, but I think it should be mentioned in the paper.
- In Definition 1, null action is describe as $0$, whereas it was defined as $\phi$ in 2.1.
- In a sentence below Definition 1, $\eta(i)=1$ -> $\zeta(i)=1$.
- Below (14), we don’t have full knowledge of transition P, but we can sample from it (like black-box model).


The ability to transfer knowledge to novel environments and tasks is a sensible desiderata for general learning agents. Despite the apparent promises, transfer in RL is still an open and little exploited research area. In this paper, we take a brand-new perspective about transfer: we suggest that the ability to assign credit unveils structural invariants in the tasks that can be transferred to make RL more sample efficient. Our main contribution is SECRET, a novel approach to transfer learning for RL that uses a backward-view credit assignment mechanism based on a self-attentive architecture. Two aspects are key to its generality: it learns to assign credit as a separate offline supervised process and exclusively modifies the reward function. Consequently, it can be supplemented by transfer methods that do not modify the reward function and it can be plugged on top of any RL algorithm.

This paper proposes to consider the problem of transfer in the context of sequential decision-making -- in particular reinforcement learning -- from the view-point of learning transferable credit assignment capability. They hypothesize that by learning how to assign credit, structural invariants can be learned which the agent can exploit to assign credit effectively and thus learn more efficiently in new environments (be it in-domain or out-of-domain). They pose the credit assignment problem as learning to predict (sparse) rewards at the end of sub-trajectories, finding the extent to which past state-action pairs appear to be responsible for these rewards (by means of the reward-prediction training), and creating a dense reward function via reward shaping (such that the set of optimal policies does not change). This is appealing as no modifications are needed to the RL algorithm/architecture. To examine their hypothesis, they created a method, called SECRET, based on self-attention and supervised learning to train credit assignment capability offline: sample many trajectories (often a mixture of expert and non-expert ones) from the source distribution, train a self-attentive seq2seq model to predict the rewards in these trajectories offline. Once this model is trained, they apply this model to a relatively small set of sampled trajectories from the target distribution and obtain the attention weights. Then, they use these attention weights as a proxy for credit assignment and, thus, use them to form a reward redistribution function. In their experiments, they show that the average attention weights actually signal the state-actions at which the future reward is triggered. They also show in their experiments that SECRET improves learning performance on in-domain transfer learning (larger mazes), as well as an out-of-domain case (with modified dynamics).
Overall, this paper proposes an interesting general avenue for research in transfer learning in RL. Regarding the proof-of-concept method and experiments, I need some clarifications. Given these clarifications in the authors' response, I would be willing to increase my score.
1. Regarding this statement on breaking Markov property: "hide a certain amount of information from states and break the Markov assumption".
(i) It is unclear to me what this "certain amount" would need to be in general. I believe this would require domain-specific knowledge to know what can be removed to break Markov-ness while not introducing significant state-aliasing (which could hinder the agent's learning).
(ii) Does any extent of partial-observability warrant that the success in reward-prediction would mean that we have a valid credit assignment model? I feel like this is not generally true, in which case I question the statement on p.3: "Note that in POMDPs, this is unnecessary since the observations agents get from the environment are incomplete by construction."
(iii) Regarding generality, the fact that states need to be (manually) preprocessed seems to me like a downside of this approach. Can you see any way around this?
2. In p.4, this is mentioned: "In POMDPs, we recover an approximate state from the observation...".
I do not see how this is done in the DMLab experiments. If this is done manually, and not trained, then I think it should be clearly stated in the main text. I think the 2nd paragraph of Sec. A.4 is stating that extra information about the state was utilized, and not approximated via a trained model to recover the states (i.e., no s^=h^-1(o) was used)?
3. What is the observation type of Vanilla RL in the out-of-domain experiments? Is it also observing its local-view (similar partial observability as SECRET) or does it have access to the full state? I would argue that it is important that the performance of Vanilla RL with partial observation is reported. Including both cases could also be beneficial.
4. Fig.3 shows attention weights on held-out environments from an identical distribution as the source (i.e., in-domain).
I would like to see how well the attention signal works when the target distribution differs from the source. Is there a reason why this is not demonstrated?
5. Not sure about specific definitions of sub-trajectory and trajectory in the paper:
(i) What constitutes a sub-trajectory (as opposed to a trajectory) in the context of this paper?
(ii) Are the lengths of the sub-trajectories or trajectories fixed?
6. Why do the attention weights not sum to 1 in Fig.3?
7. Could you clarify the role of positional encoding and how it is done?
Minor comments:
1. M is used to denote both MDP and causal map.
2. Explicitly defining d_i in p.3 should improve clarity.
3. Using 40k and 10k trajectories of interactions to train the credit-assignment model (on Triggers and DMLab domains, respectively) seems quite demanding, which seems somewhat unrealistic to deem useful for application to robotics perhaps?


Reinforcement learning methods that continuously learn neural networks by episode generation with game tree search have been successful in two-person complete information deterministic games such as chess, shogi, and Go. However, there are only reports of practical cases and there are little evidence to guarantee the stability and the final performance of learning process. In this research, the coordination of episode generation was focused on. By means of regarding the entire system as game tree search, the new method can handle the trade-off between exploitation and exploration during episode generation. The experiments with a small problem showed that it had robust performance compared to the existing method, Alpha Zero.

Badly and unprofessionally written
I had and still have a hard time to report what this paper is about. Generally, I can tell probably this paper study the Monte Carlo Tree Search-based methods in the field of Reinforcement Learning.
The paper is badly written with grammatical issues almost in all the sentences. I definitely urge the author(s) to take the writing significantly more seriously. I almost all the time do not end up rejecting a paper due to the lack of clear writing, but for this paper, unfortunately, despite spending time, I could not understand the paper.
Comments beside the grammatical issues:
I am sorry but I could not understand the abstract.
What does the author(s) mean by "In this research, the
coordination of episode generation was focused on." Same issue with almost all the sentences in the abstract. Unfortunately, I could not follow any of them. Moreover, this paper is full of inaccurate and unjustified statements. Furthermore, again, unfortunately, this paper does not represent even the preliminary elements of scientific writing.
I decided to provide further comments, but sadly I had to change my mind because otherwise, I had to write at least one paragraph of explanation for each paragraph of this paper. With that regard, I do not think it is an appropriate way to utilize the review process. But I would be happy to provide more abstract comments upon request.


Global solutions to high-dimensional sparse estimation problems with a folded concave penalty (FCP) have been shown to be statistically desirable but are strongly NP-hard to compute which implies the non-existence of pseudopolynomial time global optimization schemes in the worst case. This paper shows that, with high probability, a global solution to generalized linear models with minimax concave penalty (MCP), a specific type of FCP, coincides with a stationary point characterized by the significant subspace second order necessary conditions (S 3 ONC). Given that the desired S 3 ONC solution admits a fully polynomialtime randomized approximation scheme (FPRAS), we are able to demonstrate the existence of an FPRAS for this strongly NP-hard problem. We further demonstrate two versions of the FPRAS for generating the desired S 3 ONC solutions. One follows the paradigm of an interior point trust region algorithm and the other is the well-studied local linear approximation (LLA). Our analysis thus provides new techniques for global optimization of certain NP-Hard problems and new insights on the effectiveness of LLA.

Summary:  The paper studies the problem of global optimization of high-dimensional sparse estimators regularized by PCP (folded concave penalty). The main result is showing that under certain conditions, with high probability, the desired global solution is an oracle stationary point satisfying the so-called S^3ONC conditions. In light of this result and an existing polynomial-time algorithm for finding an S^3ONC solution, the global solution can be recovered with polynomial computational complexity. Numerical evidence is provided to show the theoretical predictions.
Strong points:
-S1. The global optimization of non-convex (regularized) sparse learning models is a fundamental and challenging problem worth investigating.
-S2. The paper is well organized and clearly presented in general.
Weak points:
-W1. The loss function in Equation (1) is lacking in explanation. It is claimed that “traditional statistical learning schemes often resort to” such a formulation. However, it looks like only the logistic loss explicitly admits such a form while some other widely used ones such as squared/hinge/exponential loss do not. So are there any concrete statistical learning models other than logistic loss falling exactly into this framework? It could be beneficial to provide a reference, if any, to this particular problem formulation in machine learning literature.
-W2. The nominal generative model is not clearly defined. Usually, a key component of high-dimensional statistical analysis is to define a nominal statistical model for data generalization. I note the parameter vector of such a model is denoted as $\beta^{true}$ in this paper. However, the related data generation procedure seems completely missing in the statement of problem setups. Particularly, what’s the definition of the residuals W appeared in the assumption A2?
-W3. The overall novelty of theory is limited. The main result established in Theorem looks closely related to the excess risk bounds established in [Liu & Ye 2019]. Actually, provided that the risk function $L(\beta)$ has (restricted) strong convexity in $\beta$, it follows immediately based on the first excess bound in Theorem 1 of [Liu & Ye 2019] that $\|\beta^* - \beta^{true}\|$ is well bounded from above. Thus if the minimum non-zero absolute value of $\beta^{true}$ is sufficiently larger than that upper bound, then it is naturally true that $\beta^*$ shares the same supporting set as $\beta^{true}$ which in turn implies that $\beta^*$ is an oracle solution and also is globally optimal under some more stringent conditions. Unless the author(s) can justify the value-added beyond the results in [Liu & Ye 2019] as sufficient, the degree of novelty of the current theory seems fairly low given that prior work.
-W4. The proof of Theorem 1 has flaws. The proof of main result relies largely on Lemma 5 which basically bounds the cardinality $\|\beta^*-\beta^{true}\|_0$. However, when invoking Lemma 5, such a $L_0$ bound shifts to an $L_2$-norm one. This misleading point needs to be clarified.
-W5. The numerical study can be improved. The majority of the reported results is about the advantage of FCP over Lasso and ridge regression, which however is less relevant to the global recovery theory developed in the paper. I think the numerical study needs to be re-designed to put more focus on the effects of some key factors, e.g , the sample size n and the signal strength $\beta_\min$, on the global optimization performance.
-W6. Concerning the fitness to venue, although somewhat relevant,  I am not sure the main topic of this paper (i.e., statistical analysis of high-dimensional GLM) would gain significant interests in the community of ICLR. Actually in my opinion, the novelty/importance of this work is more suitable to be evaluated in a high-dimensional learning theory intensive journal or conference rather than in a DL/RL conference.
Minor issues:
- M1. Check the correctness of notation $\beta$ in Definition 1.
- M2. Equation (3): Why not directly writing out “$0 \in 1/n\sum_{i=1}^n…$”?
- M3. Statement of Theorem 1: satisfy -> satisfies. The quantities $t$ and $t’$ appeared in Theorem1 are hard to understand without further explanation.
=== update after author response ===
Thank you for the response which is helpful for clarifying some of my concerns. However, after reading the revised paper, I am still not quite convinced that the current theoretical analysis is particularly novel given the results in [Liu & Ye 2019]. Also, it seems that the noconvexity of the considered problem mainly arises from the regularization term rather than the loss term, and thus can hard to be claimed as "central to ICLR".  In view of these, I choose to maintain my assessment of this paper as borderline leaning to rejection.


We present a novel method of compression of deep Convolutional Neural Networks (CNNs) by weight sharing through a new representation of convolutional filters. The proposed method reduces the number of parameters of each convolutional layer by learning a 1D vector termed Filter Summary (FS). The convolutional filters are located in FS as overlapping 1D segments, and nearby filters in FS share weights in their overlapping regions in a natural way. The resultant neural network based on such weight sharing scheme, termed Filter Summary CNNs or FSNet, has a FS in each convolution layer instead of a set of independent filters in the conventional convolution layer. FSNet has the same architecture as that of the baseline CNN to be compressed, and each convolution layer of FSNet has the same number of filters from FS as that of the basline CNN in the forward process. With compelling computational acceleration ratio, the parameter space of FSNet is much smaller than that of the baseline CNN. In addition, FSNet is quantization friendly. FSNet with weight quantization leads to even higher compression ratio without noticeable performance loss. We further propose Differentiable FSNet where the way filters share weights is learned in a differentiable and end-to-end manner. Experiments demonstrate the effectiveness of FSNet in compression of CNNs for computer vision tasks including image classification and object detection, and the effectiveness of DFSNet is evidenced by the task of Neural Architecture Search.

In this paper authors propose a novel idea (called Filter Summary, FS)  how to compress convolutional neural networks with 2D convolutions (kernels are 3D tensors, it is also applicable to the 1D convolutions). Compression of the convolution operation is done with weight sharing: unwrapped kernel (channel-major) is packed into 1D vector by having intersected segments (shared weights).
The strong sides of the paper are the following:
- Proposed algorithm for efficient computation of convolution operation with FS (less number of multiplication operations with comparison to the standard convolution).
- Via experiments it is demonstrated that proposed approach provides compression of the model while having close to the baseline quality for image classification and object detection tasks and for small and large datasets.
- Proposed method gives better compression factor and better quality than state-ot-the-art models for classification and object detection tasks.
- It is experimentally showed that architecture search works for proposed convolution.
There are still ways to improve the paper:
- In the introduction section it is written that "FS is quantization friendly". I would say the main concern should be "FS quantization achieves the same quality as original FS while having higher compression factor". Standard convolutions are also quantization friendly, no? At the same time I don't see huge gain to do quantization of the model. Yes, to store it we need less memory, however, for the inference we still use float-point computation. How will this quantization be helpful for inference on CPU/GPU (memory/speed)?
- Did you implement CUDA kernel to have efficient FS convolution (I suspect the answer yes)? It would be interesting to see the training time per epoch/forward/backward comparison for standard CNN and FS CNN as soon as you are using the same training process (for models you trained to have an idea for practical usage). Also it is interesting to see comparison of inference time on CPU/GPU.
- For illustrations (figs 1, 2, 3) of input and filters packing it would be very helpful to mark where are channel and spatial dimensions on the figure for simpler understanding the packing. Typos for the figures: "in Figure 2, two slices, marked in green and blue" -> "in Figure 2, two slices, marked in green and red"; "Figure 3 by dashed lines in green and blue that cross A, where the dashed line in red" -> "Figure 3 by dashed lines in green and blue that cross A, where the dashed line in green" (on Figure 3 there are blue and green lines only).
- In the paragraph where the number of elements of matrix A is specified to be computed, it would be better to have detailed explanation why we skip C_in*S_1 elements. It is not so trivial when you read the paper the first time.
- In section 2.3 for complexity of the three stages: seems the third stage doesn't have float-point computations (this is addition for S_2 locations in matrix A), however, the third stage is included into analysis of multiplication operations (this doesn't affect the final result of complexity)
- For DFSNet are filters ordered for FS F (monotonically located)? Or now they can be located independently and starting point defined by its alpha?
- Typo: page 7 "in Table 9" -> "in Table 2"
- In Table 4 it would be better to have results for FSNet-1 too (without quantization).
- In section 3.3 there is no details how the alpha parameter from DFSNet is used in DARTS.
- In Tables 10 and 11 there are linear quantization, what does it mean? what is the difference in quantization procedure with FSNet-WQ (maybe I missed something in the paper).


We study theoretical properties of embedding methods for knowledge graph completion under the "missing completely at random" assumption. We prove generalization error bounds for this setting. Even though the missing completely at random setting may seem naive, it is actually how knowledge graph embedding methods are typically benchmarked in the literature. Our results provide, to a certain extent, an explanation for why knowledge graph embedding methods work (as much as classical learning theory results provide explanations for classical learning from i.i.d. data).

Summary
This paper studied the knowledge completion problem from the viewpoint of statistical learning theory. More specifically, it formalized a knowledge graph completion problem as an estimation of the optimal embedding by assuming the canonical distribution derived from the embedding and derived the upper bound of misclassification rate of entities under the missing-completely-at-random assumption.
Decision
Although the formalization of knowledge graph completion problems is novel (to the best of my knowledge), there is much room for improvement in the organization of the paper. Therefore, I would recommend to reject the paper this time and to ask authors to revise the paper so that the paper is more accessible to researchers and engineers.
There are numerous existing works about the knowledge graph embedding problem and the knowledge graph completion problem using embeddings. However, few works have been done to justify these methods theoretically. This paper attempted to answer this question via statistical learning theorem perspectives. The theorems gave sufficient conditions in terms of the size of a knowledge graph and properties of embedding functions under which the misclassification rate goes to $0$. In this aspect, this paper gave insights into how we can give justifications to existing embedding methods.
However,  I think it is hard for those who are not familiar with this field to follow the logic of the paper, as I write in detail in the following sections. I understand some people argue that the paper organization is not essential for how a paper contributes to science and technology. However, the accessibility to the paper is vital to promote technical communications in different fields. Also, I believe it is beneficial for the paper to maximize its value. Thus, I would ask the authors to polish the paper.
Suggestions
- Introduction
	- Please explain what knowledge graphs are in the introduction before explaining how knowledge graphs represent knowledge, or what problems of practically available knowledge graphs have.
	- I think it is a bit too casual and conversational to use wordings like "Please bare in mind" in papers.
	- I think it is helpful if the authors add a summary of theorems and its implication in the introduction to grasp the theoretical contributions the paper has made.
- Section 6, Theorem 1
	- Please describe what is the probability of $P$ and with respect to which probability distribution $\mathbb{E}$ takes the expectation.
- Section 7
	- The authors claim that Pinsker's inequality plays a central role in deriving the upper bound for the ratio of wrongly predicted triples. However, we cannot know how the authors used the inequality unless we see the proof of Theorem 3, which is available in the appendix.
	- In Theorem 3, the definition of "wrongly predicted triple" is missing. I only find a description related to it in the proof of Theorem 3 in the appendix. Could you add the definition in the main part?
- Section 8, Theorem 4
	- The statement of Theorem 4 says that $\mathbb{X}$ is a learned representation. But it is not available in the main article how we obtain it. We know that the authors used the maximum-likelihood estimator if we read the proof of the theorem. The author should clarify it in the statement of the theorem.
- Section 10
	- What does "this" mean in the title of the section?
	- In the initial reading, I could not understand the main point the authors want to address in the paragraph starting with Loss. I think authors can clarify the point by restructuring the paragraph.
Minor comments
- Section 2 and Section 6
	- The authors treated $\mathbb{X}$, which is the set of vector representations of objects and relations, as a lookup table in Section 2.2 and reinterpreted it as the subset of $\mathcal{X}^{|\mathcal{O}|+|\mathcal{R}|}$ in Section 6. I think we can simplify the description if we treat $\mathbb{X}$ as the subset of $\mathcal{X}^{\mathcal{O} \sqcup \mathcal{R}} := \{f: \mathcal{O} \sqcup \mathcal{R} \to \mathcal{X} \}$ and denote $\mathbb{X}(o)$ as $x_o$ in short hand for a representation $\mathbb{X}$ and an object $o\in \mathcal{O}$ (and similarly for $x_r$).
- Section 4
	- It would be helpful to make the correspondence explicit between each sentence and theorem later in the last paragraph. For example, "We prove a generalization bound for log-likelihood from which a bound on Kullback-Leibler divergence follows (Theorem 3)" or something like that.
- Section 6, Theorem 1
	- The definitions of $\mathbb{X}_o$ and $\mathbb{X}_r$ are missing, if I do not miss something.
	- It is better to add the assumption that $\sup_{h, r, t} |\psi(x_h, x_r, x_t)|$ is finite.
- Section 12
	- "simplistic" means too simple by itself. So, "too simplistic" should be "too simple" or "simplistic".
Questions
- Section 4
	- I could not fully understand the intuition of the proof in the second paragraph. The authors think that the situation is desirable if multiple knowledge graphs are available. However, in the first approach, they concatenated the graphs into one (I interpreted the union of graphs as $\widehat{\mathcal{G}}_1 \cup \widehat{\mathcal{G}}_2 \cup \cdots \widehat{\mathcal{G}}_n$. Correct me if I am wrong). Since this operation would reduce the situation to the single-graph case, we cannot take advantage of multiple graphs.
- Section 7, Theorem 4
	- The upper bound is in terms of the estimator which maximizes the expected log-likelihood. I am wondering whether this estimator can achieve the best possible misclassification rate (i.e., $|\mathcal{F}/|\mathcal{O}^2|/|\mathcal{R}||$). I understand that it minimizes the discrepancy between the inferred distribution and the data generating distribution. But I am not sure it implies the least misclassification rate.
	- The small terms in the upper bound (i.e., first and second terms) depend on the number of objects $|\mathcal{O}|$ and not on relations $|\mathcal{R}|$. Since the role of $\mathcal{O}^2$ and $\mathcal{R}$ is symmetric mathematically (if I understand correctly), I feel it is weird that the additional terms do not necessarily go to zero when $\mathcal{R}\to \infty$. I particular, I expected that the additonal small terms depend solely on $|\mathcal{G}|=|\mathcal{O}|^2|\mathcal{R}|$.


Modern deep artificial neural networks have achieved impressive results through models with orders of magnitude more parameters than training examples which control overfitting with the help of regularization. Regularization can be implicit, as is the case of stochastic gradient descent and parameter sharing in convolutional layers, or explicit. Explicit regularization techniques, most common forms are weight decay and dropout, have proven successful in terms of improved generalization, but they introduce sensitive hyper-parameters and, incongruously, often require deeper and wider architectures to compensate for the reduced capacity. In contrast, data augmentation techniques exploit domain knowledge to increase the number of training examples and improve generalization without reducing the representational capacity and without introducing model-dependent parameters, since it is applied on the training data. In this paper we systematically contrast data augmentation and explicit regularization on three popular architectures and three image object classification data sets. Our results demonstrate that data augmentation alone can achieve the same performance or higher as regularized models and exhibits much higher adaptability to changes in the architecture and the amount of training data.

This paper proposed to adopt data augmentation over explicit regularization (weight decay and dropout), supported by a comparative study of 3 ConvNets (all-conv, wide-resnet and densenet) on three image classification benchmarks (ImageNet, CIFAR-10 and CIFAR-100), as well as variants of experiments such as training with smaller training set.
Most of the observations presented here are largely known to the community, though I think at least a few of them are worth emphasizing:
1. One major downside of explicit regularization the requirement to tune hyper-parameters on each different scenarios. For example, the paper shows that the default hyper-parameter becomes suboptimal when the architecture becomes either deeper or shallower.
2. Given strong enough data augmentation, sometimes turning off weight decay / dropout could give better test performance (i.e. the optimal hyper-parameters are zeros in those cases).
That being said, I think this the paper at the current state does not contain sufficient message to stand as a full paper. I list a few areas that I think could potentially improve the paper if properly addressed:
1. Proper definition of explicit / implicit regularization: one goal this paper is trying to achieve is an unambiguous definition of explicit / implicit regularization. However, the current definition given by the paper is not any less ambiguous than prevision conventions. If I understand correctly, the paper defines explicit regularization as mechanism that is explicitly designed to reduce the model capacity, and implicit one as anything else (that happens to improve generalization). There is still a lot ambiguity here: for example, one technique, say, random left-right flip of the inputs, is considered implicit regularization because it improves CIFAR-10 classification performance. However, applying it to wrong data, e.g. speech recognition, would potentially hurt the test performance. Would this technique sometimes be an implicit regularization and sometimes not? Another question is regarding dropout, which is classified as explicit regularization by the paper. Yet, if dropout is applied not to some intermediate layer, but to the input layer, does it suddenly become more similar to data augmentation (think of, e.g. cutout augmentation), and therefore an *implicit* regularization? I think a if clear, mathematical and formally verifiable definition of different types of regularization, if possible, will definite make the paper stronger.
2. Theoretical analysis: currently the paper shows the definition of the Rademacher complexity and some handwavy discussions. It does not provide any additional insights apart from what the definition says: explicit regularization constrains the capacity and data augmentation increase the number of training examples. The discussion mentioned that the augmented data are non i.i.d. which would be an interesting topic for providing theoretical insights, but it is ruled "out of the scope of the paper". Besides, the definition of the Rademacher complexity captures the capacity of the hypothesis via the mathematical sup operator, and is completely independent of the underlying algorithm used to compute the sup. Therefore, data augmentation, which is part of the training procedure, does not change the Rademacher complexity, unless we completely re-define the hypothesis space to be a complicated notion that captures something like "all the functions achievable by SGD with this augmentation and that hyper-parameters, etc.". But this will turn the "implicit regularization" nature of "data augmentation" into the regime of "explicit regularization", because it is now confining the hypothesis space into a subset (so reducing the capacity).
In summary, data augmentation clearly improves the generalization performance, but to formally characterize it, one needs an alternative approach from the default Rademacher complexity way. It would make the paper much stronger if a viable approach is proposed, and even stronger if the non-i.i.d. nature of augmented data points could be discussed under the proposed framework.
3. Empirical studies: I think the paper could still be strong even if without any theoretical characterization, if it contains strong supporting experiments. However, currently the paper only studies image classification tasks and a few convolutional neural networks. Moreover, even within those domain and tasks, there are many well known techniques, such as the cutout augmentation, that are not studied. It would be of a much useful paper to the community if the paper could provide a comprehensive survey of existing explicit / implicit regularization techniques across multiple tasks and domains (e.g. the paper mentioned vision, speech recognition, NLP, etc.). Ideally, pros and cons of each regularization technique could be discussed and if the message that "data augmentation is better than explicit regularization" could hold across multiple domains and tasks, then this will definitely be delivering a strong message. Even if that is not the case, this paper could still serve as practical guides to practitioners for choosing between different approaches of regularization techniques.
Other potential improvements are:
a. hyper-parameter tuning for each regularization technique. The paper acknowledged that the hyper-parameters are suboptimal when they change settings for explicit regularization. This is demonstrating a good point that explicit regularization is inconvenient. However, it would be better if in parallel with the default hyper-parameter, we could learn what would be the best results if we re-tune the best hyper-parameters. Because sometimes people would like to achieve the best performance in practice with all the resources and hammers they could get their hands on. So, for example, with strong data augmentation, would the (re-tuned) optimal dropout rate actually perform even better? (if true, this is also contradicting with the main message of the paper)
b. to some extent data augmentation also have "hyper-parameters", which control how to augment the data. It would be interesting to see, for example, controlled experiments on the effects on test performance when the "wrong" type or magnitudes of augmentation is applied to the data.
minor things:
* all bars in the plots are red / purplish colors. Maybe more diverse colors could be used to make it easier to distinguish which is which?
* Why is there no DenseNet results on ImageNet? (e.g. Figure 1 missing one row?)
* As a paper that studies augmentation, it would be great to provide full details on all the details (including the hyper parameters on the magnitudes of all perturbations) of the data augmentation used, especially for the "heavier augmentation" variant.
Finally, I'm not sure if this violate the policy of anonymous submission --- the acknowledgements section mentioned explicit names when thanking for feedback.


Few-shot learning is often motivated by the ability of humans to learn new tasks from few examples. However, standard few-shot classification benchmarks assume that the representation is learned on a limited amount of base class data, ignoring the amount of prior knowledge that a human may have accumulated before learning new tasks. At the same time, even if a powerful representation is available, it may happen in some domain that base class data are limited or non-existent. This motivates us to study a problem where the representation is obtained from a classifier pre-trained on a large-scale dataset of a different domain, assuming no access to its training process, while the base class data are limited to few examples per class and their role is to adapt the representation to the domain at hand rather than learn from scratch. We adapt the representation in two stages, namely on the few base class data if available and on the even fewer data of new tasks. In doing so, we obtain from the pre-trained classifier a spatial attention map that allows focusing on objects and suppressing background clutter. This is important in the new problem, because when base class data are few, the network cannot learn where to focus implicitly. We also show that a pre-trained network may be easily adapted to novel classes, without meta-learning.

The paper introduces a problem “few-shot few-shot learning” that aims to firstly transfer prior knowledge from one domain to the domain where the base training tasks reside, and then train a few-shot learning model on training tasks and apply it to novel test tasks. The two “few-shot” in the name refers to base training tasks and novel test tasks. In their algorithm, they use a model pre-trained on another dataset as the prior knowledge and fine-tune it on training tasks. During the test, they use the weighted average of samples’ representations per class as the prototype of each class, where the weight is large for samples with more discriminative prediction over pre-trained domain’s classes. Afterward, classification is reduced to finding the nearest neighbor among the class prototypes. Some experiments show that the pre-trained model can improve few-shot classification accuracy.
My major concerns:
1) They try to propose a new problem, but their description shows that the problem is exactly the same as what most “few-shot learning” works aim to solve: use a pre-trained model, train a meta-learner on few-shot training tasks, and apply it to novel test tasks.
2) The algorithm does not have any important contributions comparing to existing ones: they define a prototype per class based on the pre-trained model and apply the nearest neighbor classification. The so-called “prototypical classifier” is actually the nearest neighbor classifier since no prototypical network structure is learned in the proposed method.
3) I would not call the weighted average as “attention” because it is not: the weight in attention is computed by a module with learnable parameters, while the weight in this paper is computed by the entropy of a pre-defined model’s output prediction.
4) The “spatial attention” only makes sense when the pre-trained domain’s classes can describe the main concepts appearing in the images of novel classes. This assumption is too strong since it requires class-level (rather than lower-level) relationships.
5) The base training is not necessary in the algorithm: it is used to only fine-tuning theta and W. As the author said in the beginning of Section 4.1, they can directly solve novel tasks based on the pre-trained model.
6) The experiments show that the pre-trained model is helpful in few-shot learning, which is a known fact.
7) The writing of this paper is very poor: a lot of typos and grammar errors, inconsistency between narratives, abuse of notations, wrong equation reference, even missing punctuations. They make the paper hard to understand.
-------------
Update:
Thanks for the authors' rebuttal! After reading their rebuttal, I still have main concerns about the novelty of the problem and the writing quality. The proposed method tends to be incremental.


In state of the art model-free off-policy deep reinforcement learning (RL), a replay memory is used to store past experience and derive all network updates. Even if both state and action spaces are continuous, the replay memory only holds a finite number of transitions. We represent these transitions in a data graph and link its structure to soft divergence. By selecting a subgraph with a favorable structure, we construct a simple Markov Decision Process (MDP) for which exact Q-values can be computed efficiently as more data comes in -resulting in a QGRAPH. We show that the Q-value for each transition in the simplified MDP is a lower bound of the Q-value for the same transition in the original continuous Q-learning problem. By using these lower bounds in TD learning, our method is less prone to soft divergence and exhibits increased sample efficiency while being more robust to hyperparameters. QGRAPHs also retain information from transitions that have already been overwritten in the replay memory, which can decrease the algorithm's sensitivity to the replay memory capacity.

The paper proposes Qgraph, an algorithm that  addresses the problem of extrapolation error that appear in RL tasks with continuous action spaces. The authors describe a method to construct a graph from transitions generated by some policy. In this graph nodes correspond to states and (s, a, r, s’, t) define transitions between these nodes. Then this representation is simplified and  used to compute Q-values using methods for tabular MDPs.
The related work section is missing several methods that attempt to address the same problem. Batch Constrained Q-learning (Fujimoto etal, 2018) introduces a formulation of Q-learning that constrains action selection with a generative model trained on a replay buffer in order to omit unseen actions. BEAR-QL (Kumar etal, 2019) describes a similar approach that uses a hard constraint based on MMD. It would be interesting to discuss connections with the recent work on off-policy batch RL.
The clarity of the paper can be improved. In particular, I have several questions regarding the method:
1) How are node of the graph are constructed? Does one state correspond to a single node or several states are merged into a different node?
2) How the actions are selected?
3) What are the assumption regarding the initial state distribution? Does the set of initial states have to be finite?
4) If two similar states appear in different branches of the graphs, are the corresponding nodes merged or not?
5) If the considered environments are deterministic, what is the motivation for stochastic approximation of dynamic programming?
The approach has several major limitations. One of the main limitations of the approach is that it can be applied only to deterministic tasks. Although it is not stated clearly in the paper, it seems also requires to have a finite set of initial states.
The experimental evaluation is performed on a limited set of tasks and it is rather unclear whether the method can be scaled to higher dimensional control problems.
Overall, I feel that the paper needs to be significantly improved.


Real-world dynamical systems often consist of multiple stochastic subsystems that interact with each other. Modeling and forecasting the behavior of such dynamics are generally not easy, due to the inherent hardness in understanding the complicated interactions and evolutions of their constituents. This paper introduces the relational state-space model (R-SSM), a sequential hierarchical latent variable model that makes use of graph neural networks (GNNs) to simulate the joint state transitions of multiple correlated objects. By letting GNNs cooperate with SSM, R-SSM provides a flexible way to incorporate relational information into the modeling of multi-object dynamics. We further suggest augmenting the model with normalizing flows instantiated for vertex-indexed random variables and propose two auxiliary contrastive objectives to facilitate the learning. The utility of R-SSM is empirically evaluated on synthetic and real time series datasets.

Summary of what the paper claims and contributes
---
This paper proposes a hierarchical latent variable model of sequential dynamic processes of multiple objects when each object exhibit significant stochasticity. The model leverages Graph Neural Networks to design the architecture of the model. Normalizing flows are used to construct "Graph Normalizing Flows" that model transition densities of the per-object latent random variables, although the full model has more latent variables than tabouthe per-object latents. The approach applies and combines a host of methods to design the objective, the model, and optimize the model under the objective -- for the objective: variational inference, Contrastive Predictive Coding; for the model: normalizing flows, graph neural networks (and their combination, termed by the authors as Graph Normalizing Flows); for the optimization: variational sequential Monte Carlo. From my understanding, the main claims are that the model is a flexible way to incorporate relational information, and that the resulting model outperforms other multi-object time-series forecasting approaches.
Experiments are conducted on a synthetic toy dataset of 1d observations per-object, a real dataset of basketball player movement, corresponding to 2d observations of each object's position (the agent's and the ball's positions), and a real dataset of traffic speed forecasting (1d observations per-object). The experiments demonstrate that the model compares favorably to a baseline and a few other time-series forecasting models from the literature. The experiments show that the method achieves higher test log-likelihood (lower-bounds), and lower MSE than the compared methods. The paper discussed many related works, but it's not clear why the specific methods were chosen for comparisons. More motivation is needed for these comparisons.
Evaluation
---
>Originality:
Are the tasks or methods new?
The method is new.
Is the work a novel combination of well-known techniques?
Yes, albeit a lot of techniques.
Is it clear how this work differs from previous contributions?
Besides the fact that the proposed method is a combination of many techniques, the main / key technical difference was actually unclear to me.
Is related work adequately cited?
From my understanding, yes, mostly; there's just a few places where the paper could be situated better:
- Sec 1: "as they have been shown to be fundamental NN building blocks ...". Citations needed (e.g. include the Graph NN citations later here)
- Sec 1: "R-SSM achieves superior test likelihood and good prediction performance". When compared to what methods (citations to VRNN and GNN-AR/NRI needed here too)?
- The paper contrasted the work to other dynamics modeling works with the statement "R-SSM differs from all these works by introducing structured latent variables to represent the uncertainty on state transition and estimation". The paper also said VRNNs were the most related, because they employ "hidden states of per-agent VRNNs interact through GNNs". The paper (PRECOG, arXiv:1905.01296) appears related as it also constructs a latent variable graphical model of stochastically-transitioning multiple interacting objects with normalizing flows. Like the authors said of Graph VRNNs, it appears that one of the biggest differences to PRECOG looks like the proposed method includes more sources of stochasticity outside of the objects (the global uncertainty) to model the observations.
>Quality:
Is the submission technically sound?
Yes, although not much motivation is given for many of the specific design decisions adopted. The model is a complicated combination of many techniques from the literature, which makes it difficult to understand the importance of each specific piece of the model.
- For instance, why was global stochasticity introduced? Because per-object stochasticity is modeled with normalizing flows, the model would have been trainable with maximum likelihood estimation of a distribution over observations that's efficiently analytically computable, but after the addition of global stochasticity, the joint distribution over the random variables becomes intractable, and training must resort to variational inference. This leads to, as the paper mentioned, RSSM suffering from posterior collapse, which, to my knowledge, does not occur when training direct maximum likelihood estimation procedures (e.g. GLOW, RealNVP).
- As mentioned above, it's not clear what the key technical innovation of this approach is (Graph Normalizing Flows?) It would be good if the claimed innovation was made clear in both the abstract and the introduction.
Are claims well supported by theoretical analysis or experimental results?
The experimental results provide evidence that 1) the model can flexibly incorporate relation information and some evidence that 2) the model outperforms some multi-object time-series forecasting approaches from the literature.
Is this a complete piece of work or work in progress?
The submission appears complete.
Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?
Yes, and they included discussion of when the model is under-performing w.r.t. other methods (at longer time horizons).
>Clarity:
Is the submission clearly written?
As mentioned above, many of the numerous design decisions are not motivated well. Essentially, the paper is not very self-contained.
Is it well organized?
Yes.
Does it adequately inform the reader?
No -- the paper assumes a lot of prior knowledge, which most potential readers would likely possess. This issue would be less significant if the main innovation was clearer.
>Significance:
Are the results important?
It's difficult to tell, because it's not very clear why, of the many cited works, the specific ones chosen for comparison were used. More motivation for the comparison is needed.
Are others (researchers or practitioners) likely to use the ideas or build on them?
Possibly, although the model is complicated, which makes it less likely.
Does the submission address a difficult task in a better way than previous work?
It's plausible that the model performs better, but it's unclear how widely-applicable the method is. The model appears to be restricted to lower dimensional settings.
Does it advance the state of the art in a demonstrable way?
It's plausible, but I am not sure, as mentioned above.
Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach?
No
Additional feedback
---
Sec 5.2 describe what "OOB rate of the rollouts" means.
Sec 3.3 Coupling layer equation is missing a final parenthesis.
3.3 "complicate the learning a lot." -> "significantly complicate the learning." (original is subjectively too informal)
Sec 5.2 "like to dig into it in future work" -> "like to explore it in future work" (original is subjectively too informal)
Recommendation
---
It is unclear what the main key contributions are, outside of usually performing better than some related methods. This lack of clarity, along with a complicated design that is not very well-motivated, make me lean towards rejection. An improved version of this paper would make clear what the precise contributions are, explain how the related work did not achieve them, and why the specific comparisons were adopted.


Though visual information has been introduced for enhancing neural machine translation (NMT), its effectiveness strongly relies on the availability of large amounts of bilingual parallel sentence pairs with manual image annotations. In this paper, we present a universal visual representation learned over the monolingual corpora with image annotations, which overcomes the lack of largescale bilingual sentence-image pairs, thereby extending image applicability in NMT. In detail, a group of images with similar topics to the source sentence will be retrieved from a light topic-image lookup table learned over the existing sentence-image pairs, and then is encoded as image representations by a pretrained ResNet. An attention layer with a gated weighting is to fuse the visual information and text information as input to the decoder for predicting target translations. In particular, the proposed method enables the visual information to be integrated into large-scale text-only NMT in addition to the multimodel NMT. Experiments on four widely used translation datasets, including the WMT'16 English-to-Romanian, WMT'14 English-to-German, WMT'14 Englishto-French, and Multi30K, show that the proposed approach achieves significant improvements over strong baselines.

The authors propose to augment NMT with a grounded inventory of images.  The intuition is clear and the premise is very tempting.  The key architectural choice is to allow the transformer to use language embeddings to attend into a topic-image lookup table.  The proportion is learned to balance how much signal comes from each source.    Figure 4, attempts to investigate the importance of this sharing and its effects on performance.
While reviewing this paper I went back and read the EN-DE evaluation data for the last few years trying to see how often I could reason that images would help and I came up severely lacking.  For example, "The old system of private arbitration courts is off the table" from DE-EN 2016 Dev doesn't seem like it should benefit from this architecture.  It's then hard for me to square that with the +VR gains seen throughout this work on non-grounded datasets.  I trust that the authors did in fact achieve these results but I cannot figure out how or why.  This is all further confused by the semantic topics used for clustering the images which ignores stop words and therefore spatial relations or any grammatical nuances.
In contrast, it does make sense that Multi30K would benefit from this architecture.  As a minor note, were different feature extractors compared? The recent flurry of papers on multimodal transformers indicate that deeper resnet stacks correspond to improved downstream performance.  Is that also true in this domain?


Model-based reinforcement learning (MBRL) has been shown to be a powerful framework for data-efficiently learning control of continuous tasks. Recent work in MBRL has mostly focused on using more advanced function approximators and planning schemes, leaving the general framework virtually unchanged since its conception. In this paper, we identify a fundamental issue of the standard MBRL framework -what we call the objective mismatch issue. Objective mismatch arises when one objective is optimized in the hope that a second, often uncorrelated, metric will also be optimized. In the context of MBRL, we characterize the objective mismatch between training the forward dynamics model w.r.t. the likelihood of the one-step ahead prediction, and the overall goal of improving performance on a downstream control task. For example, this issue can emerge with the realization that dynamics models effective for a specific task do not necessarily need to be globally accurate, and vice versa globally accurate models might not be sufficiently accurate locally to obtain good control performance on a specific task. In our experiments, we study this objective mismatch issue and demonstrate that the likelihood of the one-step ahead prediction is not always correlated with downstream control performance. This observation highlights a critical flaw in the current MBRL framework which will require further research to be fully understood and addressed. We propose an initial method to mitigate the mismatch issue by reweighting dynamics model training. Building on it, we conclude with a discussion about other potential directions of future research for addressing this issue.

This paper discusses the old problem of mismatch between the ultimate reward obtained after optimizing a  decision (planning or control) over a probabilistic model (of dynamics) and  the training  objective for the model (log-likelihood). Experiments highlight that the NLL and reward can be very poorly correlated, that improvements in NLL initially improve reward but can later degrade it, and that models with similar NLLs can lead to very different rewards. A  reweighting trick is proposed and summarily evaluated.
I like the topic of this paper but there are several aspects which I see as making it weaker than my acceptance threshold.
First, the paper overclaims in originality. This mismatch problem is not new, it is an instance of a more general issue that end-to-end training and meta-learning try to address, and has been already studied in the context of MBRL by many authors, who actually proposed more substantial solutions. When I read the abstract I had the impression that the paper actually had a theoretical analysis showing the correlation problem, but there is no such thing, only experiments. Section 3 does not actually provide a new insight. Still, the experiments are interesting in that they reveal that the magnitude of the mismatch is probably more serious than most RL researchers believed.
Second, the 'fix' proposed is not well justified nor well tested (e.g. no quantiative comparisons, no comparisons against existing alternative methods to address the same problem, etc). This seriously weakens conclusions like "shows improvements in sample efficiency".
One concern I have about the experiments of fig 3 is that NLL can be really bad, thus distorting rho, which is not a robust measure. So I would only look at NLLs of models with good NLLs, to obtain a more interesting analysis.
Another concern about experiments is that I am not convinced that they were performed with SOTA MBRL methods and hyper-parameters (as demonstrated by SOTA performance on known benchmarks). Otherwise I could easily imagine how the mismatch could be much more severe than in the actual scenarios of interest.
Minor points:
Bottom of page 6 refers to visualizations but I did not see if or where they were shown.
Why the e in the numerator of eq 2e? Seems useless to put any constant there.
The section on 'Shaping the cost or reward' was not clear enough to me (please expand).


Graph convolutional networks (GCNs) have achieved remarkable performance in a variety of network science learning tasks. However, theoretical analysis of such approaches is still at its infancy. Graph scattering transforms (GSTs) are non-trainable deep GCN models that are amenable to generalization and stability analyses. The present work addresses some limitations of GSTs by introducing a novel so-termed pruned (p)GST approach. The resultant pruning algorithm is guided by a graph-spectrum-inspired criterion, and retains informative scattering features on-the-fly while bypassing the exponential complexity associated with GSTs. It is further established that pGSTs are stable to perturbations of the input graph signals with bounded energy. Experiments showcase that i) pGST performs comparably to the baseline GST that uses all scattering features, while achieving significant computational savings; ii) pGST achieves comparable performance to state-of-the-art GCNs; and iii) Graph data from various domains lead to different scattering patterns, suggesting domain-adaptive pGST network architectures.

A scattering transform on graphs consists in the cascade of wavelets, modulus non-linearity and a low-pass filter. The wavelets and the low-pass are designed in the spectral domain, which is computationally extensive. Instead to compute any cascades of wavelets, this paper proposes to prune scattering paths which have the lowest energy. This is simple, and numerically efficient.
I find this idea elegant because it is simple, and it seems to work well(and even improve standard GST). Yet, I would have liked to see more explanations concerning the threshold selection: indeed, a naive simple grid search to find the best threshold could be less efficient than doing no pruning.(at least at training, but also for testing: maybe some guarantees that the test accuracy won't be lower under reasonable assumptions)
Pros :
- Good performances with a simple method. This is a positive aspect of the paper.
- This is an attempt to scale spectral methods on large graphs.
- A stability study of the practical impact of this threshold is done.
Cons:
- The bound in lemma 1 sounds pretty un-optimal. Indeed, the frame bound is B, thus the corresponding operator is B-lipschitz, and the modulus operator is 1-lipscthiz. Thus, shouldn't the lipschitz constant be simply $\sim B^{L+1}$? I don't understand why it does depend on the number of filters. Also, if B>1, then the lipschitz constant can be arbitrary high. This sounds to me like a big issue, because high lipschitz constant would indicate an unstability. For instance if $B\geq 1$, for order 1:
$B^4\Vert x\Vert^2\geq B^2\Vert Hx\Vert^2 + B^2\Vert Ux\Vert^2\geq \Vert |H||H|x\Vert^2+\Vert U|H|x\Vert^2+B^2\Vert Ux\Vert^2\geq \Vert \Phi x\Vert^2$
- How is the threshold selected? It is obvious that the normalisation of the filters play an extremely important role(if the frame was tight, I wouldn't complain). This should be well documented and this is currently not the case. I find this is critical. Maybe a parallel with the bias of soft-thresholding operators could be drawn to find an optimal value?
- I also believe the threshold shouldn't take in account the low-pass filter contribution. Indeed, as for equation (5), the energy of z_(p) is the sum of the energy of the high frequencies of z_p and the low frequency energy. It would not add any compute time(because this quantity is computed anyway) to measure the relative energy w.r.t. the high frequencies without the low-pass filter. It would write then:
||z_(puj)||^2/(||z_p||^2-||U(z_p)||^2)
. Indeed, in classical signal processing (e.g., images, spectrogram), the low pass filter contains much more energy than other frequencies, which are however informative. In the current setting, discriminative scattering path could be discarded simply because the wavelet combined with a modulus did a good job at demodulating the signal to the low frequency domain.
- ". Furthermore, stable features are not necessary useful for learning. For example, an all-zero scattering feature at a certain output channel of the GST is non-informative albeit stable to any deformation" > I find this statement pretty obvious, could the authors think of a better example? In my opinion, stability to deformation at not extra-cost is always desirable because it helps creating a linear invariants.
minor:
- "Under certain conditions on graph filter banks, GSTs are endowed with energy conservation properties, as well as stability meaning robustness to graph topology deformations (Gama et al., 2019a)." > I think the citation about (Zou&Lerman) should be added here. (next to "conservation property")
- The notation p^{(l)}\cup j can be a bit confusing, couldn't this author write (p^{(l)},j) or anything similar?
- Top equation of page 5: $V^Th_j(S)x=diag(h_j)$. Shouldn't there be a \hat on diag(h_j)? Indeed, the current notation is a bit confusing... I guess it would be nice to introduce the convolution operator, in Fourier.
- "if the graph spectrum {hj (λn)}n is not aligned with [xb]n" . I'm not sure to understand the term "align" in this context, in particular because it seems to be used in different contexts in the paper. Could you simplify to "if the support of h_j is included in the support of \hat x?"
- Figures and Tables are too small...
post rebuttal:
I found the arguments of the authors convincing, they addressed my concern and revised the paper accordingly.