It has been argued that current machine learning models do not have common sense, and therefore must be hard-coded with prior knowledge (Marcus, 2018). Here we show surprising evidence that language models can already learn to capture certain common sense knowledge. Our key observation is that a language model can compute the probability of any statement, and this probability can be used to evaluate the truthfulness of that statement. On the Winograd Schema Challenge (Levesque et al., 2011), language models are 11% higher in accuracy than previous state-of-the-art supervised methods. Language models can also be fine-tuned for the task of Mining Commonsense Knowledge on ConceptNet to achieve an F1 score of 0.912 and 0.824, outperforming previous best results (Jastrzebski et al., 2018). Further analysis demonstrates that language models can discover unique features of Winograd Schema contexts that decide the correct answers without explicit supervision.

This paper evaluates language models for tasks that involve "commonsense knowledge" such as the Winograd Schema Challenge (WSC), Pronoun Disambiguation Problems (PDP), and commonsense knowledge base completion (KBC).
Pros:
The approach is relatively simple in that it boils down to just applying language models.
The results outperform prior work, in some cases by pretty large margins.
The language models are quite large and it appears that this is the first time that large-scale language models have been applied seriously to the Winograd Schema Challenge (rather than, say, to the NLI version of it in GLUE, to which it is hard to compare these results).
Some of the additional and ablation experiments are interesting.
Cons:
While this paper has some nice results, there are some aspects of it that concern me, specifically related to hyperparameter tuning and experimental rigor:
There are three methods given for using an LM to make a prediction: full, full-normalized, and partial. For PDP, full (or perhaps full-normalized?) works best, while for WSC, partial works best. The differences among methods, at least for WSC, are quite large: from 2% to 10% based on Figure 3. I don't see a numerical comparison for PDP, so I'm not sure how these methods compare on it. Since the datasets are so small, there is no train/dev/test split, so how were these decisions made? They seem to be oracle decisions. This is concerning to me, as there is not much explanation given for why one method is better than another method.
My guess is that the reason why partial works better than full for WSC is because the WSC sentences were constructed such that the words up to and including the ambiguous pronoun were written such that it would be difficult to identify the antecedent of the pronoun. The rest of the sentence would be needed to identify the antecedent. I'll assume for this discussion that the sentence can be divided into three parts x, y, and z, where x is the part before the pronoun, y is the phrase that replaces the pronoun, and z is the part after the pronoun. Then p(z|xy), which is partial scoring, corresponds to p(xyz)/p(xy), which can be viewed as "discounting" or "normalizing for" the probability of putting y in place of the pronoun given the context x. For WSC, I think one of the goals in writing the instances is to make the "true" p(xy) approximately equal for both values of y. The language model will not naturally have this be the case (i.e., that p(xy) is the same for both antecedents), so dividing by p(xy) causes the resulting partial score to account for the natural differences in p(xy) for different antecedents. This could be explored empirically. For example, the authors could compute p(xy) for both alternatives for all PDP and WSC instances and see if the difference (|p(xy_1) - p(xy_2)|, where y_1 and y_2 are the two alternatives) is systematically different between WSC and PDP. Or one could see if p(xy) is greater for the antecedent that is closer to the pronoun position or if it is triggered by some other effects. It could be the case that the PDP instances are not as carefully controlled as the WSC instances and therefore some of the PDP instances may exhibit the situation where the prediction can be made partially based on p(xy). The paper does not give an explanation for why full scoring works better for PDP and chalks it up to noise from the small size of PDP, but I wonder if there could be a good reason for the difference.
The results on KBC are positive, but not super convincing. The method involves fine-tuning pretrained LMs on the KBC training data, the same training data used by prior work. The new result is better than prior work (compared to the "Factorized", the finetuned LM is 2.1% better on the full test set, and 0.3% better on the novelty-based test set), but also uses a lot more unlabeled data than the prior work (if I understand the prior work correctly). It would be more impressive if the LM could use far fewer than the 100K examples for fine-tuning. Also, when discussing that task, the paper says: "During evaluation, a threshold is used to classify low-perplexity and high-perlexity instances as fact and non-fact." How was this threshold chosen?
I also have a concern about the framing of the overall significance of the results. While the results show roughly a 9% absolute improvement on WSC, the accuracies are still far from human performance on the WSC task. The accuracy for the best pretrained ensemble of LMs in this paper is 61.5%, and when training on WSC-oriented training data, it goes up to nearly 64%. But humans get at least 92% on this task. This doesn't mean that the results shouldn't be taken seriously, but it does suggest that we still have a long way to go and that language models may only be learning a fraction of what is needed to solve this task. This, along with my concerns about the experimental rigor expressed above, limits the potential impact of the paper.
Minor issues/questions:
In Sec. 3.1: Why refer to the full scoring strategy as "naive"? Is there some non-empirical reason to choose partial over full?
The use of SQuAD for language modeling data was surprising to me. Why SQuAD? It's only 536 articles from Wikipedia. Why not use all of Wikipedia? Or, if you're concerned about some of the overly-specific language in more domain-specific Wikipedia articles, then you could restrict the dataset to be the 100K most frequently-visited Wikipedia articles or something like that.
I think it would be helpful to give an example from PDP-60.
Sec. 5.1: How is F_1(n) defined?  I also don't see how a perfect score is 1.0, but maybe it's because I don't understand how F_1(n) is defined.
Sec. 6.1: Why would t range from 1 to n for full scoring? Positions before k are unchanged, right? So q_1 through q_{k-1} would be the same for both, right?
In the final example in Figure 2, I don't understand why "yelled at" is the keyword, rather than "upset". Who determined the special keywords?
I was confused about the keyword detection/retrieval evaluation. How are multi-word keywords handled, like the final example in Figure 2? The caption of Table 5 mentions "retrieving top-2 tokens". But after getting the top 2 tokens, how is the evaluation done?
Sec. 6.3 says: "This normalization indeed fixes full scoring in 9 out of 10 tested LMs on PDP-60." Are those results reported somewhere in the paper? Was that normalization used for the results in Table 2?
Sec. 6.3 says: "On WSC-273, the observation is again confirmed as partial scoring, which ignores c [the candidate] altogether, strongly outperforms the other two scorings in all cases" -- What is meant by "which ignores c altogether"?  c is still being conditioned on and it must not be ignored or else partial scoring would be meaningless (because c is the only part that differs between the two options).
Typos and minor issues:
Be consistent about "common sense" vs. "commonsense".
Be consistent about "Deepnet" vs. "DeepNet" (Tables 2-3).
Sec. 1:
"even best" --> "even the best"
"such as Winograd" --> "such as the Winograd"
"a few hundreds" --> "a few hundred"
"this type of questions" --> "this type of question"
"does not present" --> "is not present"
"non-facts tuples" --> "non-fact tuples"
Sec. 2:
"solving Winograd" --> "solving the Winograd"
"Store Cloze" --> "Story Cloze"
"constructed by human" --> "constructed by humans"
Sec. 4:
What is "LM-1-Billion"?
Why SQuAD?
"Another test set in included" --> "Another test set is included"
Sec. 5.2:
Check margin in loss_new
"high-perlexity" --> "high-perplexity"
Sec. 6:
Figure 2 caption: "keyword appear" --> "keyword appears"
Sec. 6.2:
"for correct answer" --> "for the correct answer"
Appendix A:
"acitvation" --> "activation"
Appendix B:
Figure 4 caption: "is of" --> "is"
The right part of Figure 4 has some odd spacing and hyphenation.


Graph node representation learning is a central problem in social network analysis, aiming to learn the vector representation for each node in a graph. The key problem is how to model the dependence of each node to its neighbor nodes since the neighborhood can uniquely characterize a graph. Most existing approaches rely on defining the specific neighborhood dependence as the computation mechanism of representations, which may exclude important subtle structures within the graph and dependence among neighbors. Instead, we propose a novel graph node embedding method (namely P 2 IR) via developing a novel notion, namely partial permutation invariant set function to learn those subtle structures. Our method can 1) learn an arbitrary form of the representation function from the neighborhood, without losing any potential dependence structures, 2) automatically decide the significance of neighbors at different distances, and 3) be applicable to both homogeneous and heterogeneous graph embedding, which may contain multiple types of nodes. Theoretical guarantee for the representation capability of our method has been proved for general homogeneous and heterogeneous graphs. Evaluation results on benchmark data sets show that the proposed P 2 IR outperforms the state-of-the-art approaches on producing node vectors for classification tasks.

The authors introduce the idea of a partial permutation invariant set function and use this to learn node embeddings. The paper is well-written and the discussion is quite easy to follow along. The paper also introduces some interesting concepts like a partial permutation invariant set function. However, I find that some of the paper's main claims can be further backed up by more experiments.
The paper is well-written. The authors evaluate their approach on a large number of real-world homogeneous and heterogeneous graphs. Furthermore, they show the stability of their method by showing consistently good results even when training set size is varied. Defined the notion of a partial permutation invariant set function and provided theoretical guarantees pertaining to this.
One of the strengths of the proposed method is its ability to "automatically decide the significance of nodes at different distances." The authors devote a good portion of their paper to talk about this. However, a recent paper published in NIPS '18 [1] with a pre-print available much earlier solves this problem by applying attention over powers of a transition matrix. The authors should talk about [1] and ideally compare against them.
I feel that a lot of the authors main claims can be strengthened further if more experiments were shown to back these up. What I mean to say is, it would be nice to see some other experiments apart from just classification performance. To compare with [1], for instance, they show link prediction/classification results but on top of these they also show that their method may choose very different "neighborhoods" to attend to.
The authors gave a fairly comprehensive review of related literature (which was good!) and they mentioned that "most existing methods either explicitly or implicitly restrict the dependence form of each node to its neighbors and also the depth of neighbors." I feel another approach they should compare against which does not seem to have this problem is [2] since the method learns "role-based" embeddings which are more dependent on structure rather than proximity.
The paper in its current form is fairly good. If comparison can be made against [1] & [2] and some additional experiments can be added, the quality of the paper can be improved further.
[1] Watch Your Step: Learning Node Embeddings via Graph Attention. Abu-El-Haija et al. In Proc. of NIPS 2018.
[2] Higher-Order Network Representation Learning. Rossi et al. In Proc. of WWW 2018.
There are some minor errors in the paper:
Table 1 in page 7 seems to be an error. It's an empty table and it is not referred to anywhere in the paper.
The format of some references needs double-checking. For example,
"Jian Tang, Meng Qu, and Qiaozhu Mei. Pte: Predictive text embedding through large-scale heterogeneous text networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1165–1174. ACM, 2015a."
(1) "21th" should be "21st".
"Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C Aggarwal, and Thomas S Huang. Heterogeneous network embedding via deep architectures. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 119–128. ACM, 2015."
(2) "21th" should be "21st".
"Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. Asymmetric transitivity preserving graph embedding. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pp. 1105–1114, 2016."
(3) "22Nd" should be "22nd"
"Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network embedding. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’16, pp. 1225–1234, 2016."
(4) "22Nd" should be "22nd"
"Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 701–710. ACM, 2014"
(5) "international conference on Knowledge discovery and data mining" should be "International Conference on Knowledge Discovery and Data Mining"
"Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pp. 3111–3119, 2013."
(6) "Advances in neural information processing systems" should be "Advances in Neural Information Processing Systems"
"Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077. International World Wide Web Conferences Steering Committee, 2015b."
(7) "International World Wide Web Conferences Steering Committee" should be removed.
The 3rd sentence in the abstract is a little bit too long. It would be better if the authors could break the sentence into shorter ones. Here is the sentence: "While most existing approaches rely on defining the specific neighborhood dependence as the computation mechanism of representations, which may exclude important subtle structures within the graph and dependence among neighbors, we propose a novel graph node embedding method (namely P2IR) via developing a novel notion, namely partial permutation invariant set function."


LEARNING PROGRAMMATICALLY STRUCTURED REPRESENTATIONS WITH PERCEPTOR GRADIENTS
We present the perceptor gradients algorithm -a novel approach to learning symbolic representations based on the idea of decomposing an agent's policy into i) a perceptor network extracting symbols from raw observation data and ii) a task encoding program which maps the input symbols to output actions. We show that the proposed algorithm is able to learn representations that can be directly fed into a Linear-Quadratic Regulator (LQR) or a general purpose A* planner. Our experimental results confirm that the perceptor gradients algorithm is able to efficiently learn transferable symbolic representations as well as generate new observations according to a semantically meaningful specification.

The high-level problem this paper tackles is that of learning symbolic representations from raw noisy data, based on the hypothesis that symbolic representations that are grounded in the semantic content of the environment are less susceptible to overfitting.
The authors propose the perceptor gradients algorithm, which decouples the policy into 1) a perceptor network that maps raw observations to domain-specific representations, which are inputs to 2) a pre-specified domain-specific control or planning program. The authors claim that such a decomposition is general enough to accommodate any task encoding program.
The proposed method is evaluated on three experiments: a simple control task (cartpole-balancing), a navigation task (minecraft: go to pose), and a stochastic single-object retrieval task (minecraft: collect wood). The authors show that the perceptor gradients algorithm learns much faster than vanilla policy gradient. They also show that the program provides an inductive bias that helps ground the representations to the true state of the agent by manually inspecting the representations and by reconstructing the representation into a semantically coherent scene.
This paper is clear and well-written and I enjoyed reading it. It proposes a nice perspective of leveraging programmatic domain knowledge and integrating such knowledge with a learned policy for planning and control. If the following concerns were addressed I would consider increasing my score.
1. To what extent do the experiments support the authors' claims: Although the existing experiments are very illustrative and clear, they did not seem to me to illustrate that the learned representations are transferable as the authors claimed in the introduction. This perhaps is due to the ambiguous definition of "transferable;" it would be helpful if the authors clarified what they mean by this. Nevertheless, as the paper suggests in the introduction that symbolic representations are less likely to overfit to the training distribution, I would be interested to see an experiment that illustrates the capability of the program-augmented policy to generalize to new tasks. For example, Ellis et al. [1] suggested that the programs can be leveraged to extrapolate to problems not previously seen in the input (e.g. by running the for loop for more iterations). To show the transferability of such symbolic representations, is it possible for the authors to include an experiment to show to what extent the perceptor gradients algorithm can generalize to new problems? For example, is it possible for the proposed approach to train on "Minecraft: Go to Pose" and generalize to a larger map? Or is it possible for the proposed approach to train on one wood block and generalize to more wood blocks?
2. Experiment request: The paper seems to suggest that the "Minecraft: Go to Pose" task and the "Minecraft: Collect Wood" task were trained with an autoencoding perceptor. To more completely assess to what extent the program is responsible for biasing the representations to be semantically grounded in environment, would the authors please provide ablation experiments (learning curves, and visualization of the representations) for these two tasks where only the encoder was used?
3. Question: I am a bit confused by the beta-VAE results in Figure 4. If the beta-VAE is trained to not only reconstruct its input as well as perform a "linear regression between the learnt latent space and the ground truth values" (page 6), then I would have expected that the latent space representations to match the ground truth values much more closely. Would the authors be able to elaborate more on the training details and objective function of the beta-VAE and provide an explanation for why the learned latent space deviates so far from the ground truth?
5. Related work:
    a) The paper briefly discusses representation learning in computer vision and physical dynamics modeling. However, in these same domains it lacks a discussion of approaches that do use programs to constrain learned representations, as in [1-3]. Without this discussion, my view is that the related work would be very incomplete because program-induced constraints are core to this paper. Can the authors please provide a more thorough and complete treatment of this area?
    b) The approaches that this paper discusses for representation learning have been around for quite a long time, but it seems rather a misrepresentation of the related work to have all but two citations in the Related Work section 2017 and after. For example, statistical constraints on the latent space have been explored in [4-5]. Can the authors please provide a more thorough and complete treatment of the related work?
6. Possible limitation: A potential limitation for decoupling the policy in this particular way is that if the perceptor network produced incorrect representations that are fed into the program, the program cannot compensate for these errors. It would be helpful for the authors to include a discussion about this in paper.
7. How does this scale? As stated in the intro, the motivation for this work is for enabling autonomous agents to learn from raw visual data. Though the experiments in this paper were illustrative of the approach, these experiments assumed that the agent had access to the true state variables of its environment (like position and velocity), and the perceptor network is just inferring the particular values of these variables for a particular problem instance. However, presumably the motivation for learning from raw visual data is that the agent does not have access to the simulator of the environment. How do the authors envision their proposed approach scaling to real world settings where the true state variables are unknown? There is currently not an experiment that shows a need for learning from raw visual data. This is a major concern, because if the only domains that the perceptor gradients algorithm can be applied are those where the agent already has access to the true state variables, then there may be no need to learn from pixels in the first place. This paper would be made significantly stronger with an experiment where 1) learning from raw visual data is necessary (for example, if it is the real world, or if the true state variables were unknown) and 2) where the inductive bias provided by the program helps significantly on that task in terms of learning and transfer. Such an experiment would decisively reflect the paper's claims.
8. Clarity: The paper mentions that it is possible to generate new observations from the latent space. The paper can be made stronger by a more motivated discussion of why generating new observations is desirable, beyond just as a visualization tool. For example, the authors may consider making a connection with the analysis-by-synthesis paradigm that characterizes the Helmholtz machine.
[1] Ellis et al. (https://arxiv.org/pdf/1707.09627.pdf)
[2] Wu et al. (http://papers.nips.cc/paper/6620-learning-to-see-physics-via-visual-de-animation.pdf)
[3] Kulkarni et al. (https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Kulkarni_Picture_A_Probabilistic_2015_CVPR_paper.html)
[4] Schmidhuber (ftp://ftp.idsia.ch/pub/juergen/factorial.pdf)
[5] Bengio et al. (https://arxiv.org/abs/1206.5538)


Current deep neuroevolution models are usually trained in a large parameter search space for complex learning tasks, e.g. playing video games, which needs billions of samples and thousands of search steps to obtain significant performance. This raises a question of whether we can make use of sequential data generated during evolution, encode input samples, and evolve in low dimensional parameter space with latent state input in a fast and efficient manner. Here we give an affirmative answer: we train a VAE to encode input samples, then an RNN to model environment dynamics and handle temporal information, and last evolve our low dimensional policy network in latent space. We demonstrate that this approach is surprisingly efficient: our experiments on Atari games show that within 10M frames and 30 evolution steps of training, our algorithm could achieve competitive result compared with ES, A3C, and DQN which need billions of frames.

Summary: They propose "Sample Efficient Deep Neuroevolution" (SEDN) model and experiment on Atari games. In this model, they use a Variational Encoder (VAE) to encode state frames into a latent vector, and use an LSTM to encode the current latent vector and action to predict the next latent space. A policy network (trained using CMA-ES) takes the latent space, and hidden state of the RNN as an input, and outputs an action to execute.
The strengths of this paper is that it is clearly written. They even explain details of RL, background of evolution strategies, motivation of using CMA-ES (and also the algorithm itself, which is no small ordeal), so it might a good background review paper of the literature. The experimental setup is relatively easy to understand. I suggest putting Algorithm 3 before Table 1 since presenting the algorithm before results may be more natural.
That being said, there are issues with this work that needs to be addressed before publication. I will list the issues and some suggestions I have, in order to help make this work better, hopefully good enough for acceptance:
1) The authors cited [1] a few times in the paper, but actually their approach of using a VAE to compress frames into a latent, an LSTM to predict the next latent, and a CMA-ES trained network for the policy is precisely what is proposed in [1] (which had experiments that trained on the actual environment, like in this work, and also the generated environment). This paper reads like they have proposed the setup, and lacks clarity as to which parts are their contribution, and which parts are prior work, which I believe is important for a paper submitted to an academic venue. Not to say at all that there's no contribution or originally in this work - there are many, but I feel they should list out which bits are their contribution, and which bits are prior work more clearly. Doing so will make this paper and their contributions stronger.
In my opinion, their contributions are: Expanding on the approach of [1] to study on a larger set of environments (the Atari suite), where they also incorporated an iterative training loop (described Algorithm 3) that was not used in [1]. Also, unlike [1], they used a multi-layer policy network, and also explained and rationalized the intuition behind the choice of CMA-ES. I think by listing out the contributions, and separating them from previous work, the paper will be much stronger.
2) The results are not terribly strong. They achieved good results on 7 games out of 50 using 10M frames. To me, that's actually not a deal breaker, since research is not a SOTA game, but I would like to see a more detailed analysis of why the algorithm works, and when it fails so people know what future work needs to be done to address this. I'm also not convinced that using CMA-ES would have an advantage over A3C (with the same latent / hidden features going into A3C as inputs), so perhaps the author might achieve better results if A3C was used to train the policy network (or not, but would be nice to see this experiment). It would offer more insight if we know what kind of terminal scores can be achieved using this algorithm, if it were allowed to train for 1B frames like the other 2 setup. Finally, if the author was able to show that training inside a generated environment, even for pre-training before going back to the actual environment, helps sample efficiency, that would be a very interesting result to me.
I'm assigning a preliminary score of 5 for this work for now, but if the author address point (1) to my satisfaction I will revise the score to +1 points, and if the author is able to achieve much better results, or address items in point (2) to my satisfaction, I will revise the score by +1 or potentially +2 points, so the final score of this work may lie in the range of 5 -> 8. I feel the author should be able to improve the paper to get a score of 6-7 in the end, at least from me. Good luck!
[1] https://arxiv.org/abs/1803.10122


To gain high rewards in muti-agent scenes, it is sometimes necessary to understand other agents and make corresponding optimal decisions. We can solve these tasks by first building models for other agents and then finding the optimal policy with these models. To get an accurate model, many observations are needed and this can be sample-inefficient. What's more, the learned model and policy can overfit to current agents and cannot generalize if the other agents are replaced by new agents. In many practical situations, each agent we face can be considered as a sample from a population with a fixed but unknown distribution. Thus we can treat the task against some specific agents as a task sampled from a task distribution. We apply meta-learning method to build models and learn policies. Therefore when new agents come, we can adapt to them efficiently. Experiments on grid games show that our method can quickly get high rewards.

This paper focuses on fast adaptation to new behaviour of the other agents of the environment, be it opponents or allies. To achieve this, a method based on MAML is proposed, with two main components:
1) Learn a model of some characteristics of the opponent, such as "the final goal, next action, or any other character we wish to predict"
2) Learn a policy that takes as input the output of the model and the state, and that outputs the action of the agent.
The goal is that after a phase of meta learning, where the agents learns how to play against some new agents sampled from the distribution of opponents, it can quickly adapt to a new unseen agent. ("Experimental results show that the agent can adapt to the new opponent with a small number of interactions with the opponent")
While the motivation of this work is clear and the goal important for the RL community, the experiments fail to support the claim above.
The first task they demonstrate their approach on is a chasing game, where the opponent has a private goal cell it tries to reach, and the agent has to chase it. At the end of the game, it gets a reward of 10 if it is on the same cell, 5 if in an adjacent cell, and 0 otherwise. The exact details of the dynamic are not really clear, for example what happens in the event of a collision is not mentionned, and the termination condition is not mentionned either. (the text reads "One game lasts for at least 15 steps", maybe it was meant to be "at most 15 steps" ?).
The first incoherent aspect of this experiment is that they use 800 iterations of meta-learning, and then, when testing, they fine-tune their networks against each test opponent during 4000 games. That is, they use 5 times more game when fine-tuning as opposed to when pre-training, which contradicts the claim "the agent can adapt to the new opponent with a small number of interactions with the opponent" (this is not really few-shot learning anymore).
Further more, they compare their approach with various ablations of it: they either remove the meta-learning for the model (MA), for the policy (MO), or both (NM). The description of the NM baseline is not very precise, but it seems that it simply boils down to a regular (dueling) DQN: In this setting, since the opponent appears to have a fixed goal, finetuning against a single opponent simply boils down to learning a policy that reaches a specific cell of the grid, which we can expect DQN to solve perfectly on a 8x8 grid with 4000 training games. And yet, the curves for NM in graph 2c is not only really noisy, but also falls far from the optimum, which the authors don't discuss. There might be a problem with the hyperparameters used or the training loop.
The second task is a blocking game: the opponent has to choose amongst 5 paths to get to the top, and the agent has to choose the same path in order to block it. The action space should be precisely described, as it stands it is difficult to understand the dynamic. There are at least two possible ways to parametrize the actions:
1) Similarly to the blocking game, the agents could move in the 8 directions. In that case, based on the picture 3a, it seems that the agent can just mirrors the move of the opponent: since the moves are simultaneous, that would mean that the agent is always one step late, but each path is long enough for the agent to reach the exit before its opponent (it explicitly stated that the agent needs to block the exit, and that the opponent will not change path during one game). That would imply that perfect play is possible without any meta-learning or oponent modeling, and once again the NM baseline (or any vanilly DQN/Policy gradient method) should perform much better.
2) One other alternative is to have an action space of 5 actions, which correspond to the 5 paths. In that case the game boils down to a bandit, since both agents only take one action. Note that under this assumption, the random policy would get the right path (and reward +10) with probability 1/5 and a wrong one (reward -10) with probability 4/5, which leads to an expectated reward of -10*4/5 + 10/5 = -6. This is not consistent with the graph 3c, since at the beginning of the training, the NM agent should have a random policy, and yet the graph reports an average reward of -10 (the -6 mark seems to be reached after ~1000 episodes)
The last task boils down to one opponent that reaches one cell on the right, and the agent must reach the matching cell on the left. In this setting, the same discussion on the action space as the second task can be made. We note that the episode for 16 steps, and the distance from the center to any cell is at most 4 steps: an optimal policy would be to wait for 4 steps in the middle, and as soon as the opponent has reached its goal, use the remaining 12 steps to get to the mirror one. Once again, this policy doesn't require any prediction on the opponent's goal, and it's hard to believe that DQN (possibly with an lstm) is not able to learn that near perfectly.
In a last test the authors compare the performance of their algorithms in a one shot transfer setting: they sample 100 opponents for each task and play only one game against it (no fine-tuning). It is not clear whether special care has been taken to ensure that none of the sampled opponents has already been seen during training.
We note that the rewards reported for MO and MA (resp 0.0 and -0.08) are not consistent with the description of the reward function: on the worst case, the opponent chooses a goal on one extreme (say y1 = 1) and the agent chooses an object on the other end (say y2 = 7). In that case, the reward obtained is sampled from a gaussian with mean \mu = 10 - 3/2 * |y1 - y2| (which in this case evalutes to 1), and variance 1. This is highly unlikely to give such a low average reward over 100 episodes (note that this is worst case, if the opponent's goal is not on the extreme, the expected reward is necessarily higher). One possibility is that the agent never reaches an object, but in that case it would imply the that the meta-learning phase was problematic.
We also note that it is explicited that the MOA, MO and MA methods are tested after meta-training, but nothing is precised for NM. Has it been trained at all? Against which opponents? Is it just a random policy? There are too many missing details for the results to be interpretable.
Apart from that, the paper contains a significant amount of typos and gramatical mistakes please proof-read carefully. Some of them are:
"To demonstrate that meta-learning can do take"
"player 1 is the red grid and player 1 is the green one"
"we further assume that there exist a distribution"
" the goal’s location over the map is visualize in figure"
"Both players takes actions simultaneously"


Generative Adversarial Networks (GANs) are a very powerful framework for generative modeling. However, they are often hard to train, and learning of GANs often becomes unstable. Wasserstein GAN (WGAN) is a promising framework to deal with the instability problem as it has a good convergence property. One drawback of the WGAN is that it evaluates the Wasserstein distance in the dual domain, which requires some approximation, so that it may fail to optimize the true Wasserstein distance. In this paper, we propose evaluating the exact empirical optimal transport cost efficiently in the primal domain and performing gradient descent with respect to its derivative to train the generator network. Experiments on the MNIST dataset show that our method is significantly stable to converge, and achieves the lowest Wasserstein distance among the WGAN variants at the cost of some sharpness of generated images. Experiments on the 8-Gaussian toy dataset show that better gradients for the generator are obtained in our method. In addition, the proposed method enables more flexible generative modeling than WGAN.

The paper ‘Generative model based on minimizing exact empirical Wasserstein distance' proposes
a variant of Wasserstein GAN based on a primal version of the Wasserstein loss rather than the relying
on the classical Kantorovich-Rubinstein duality as first proposed by Arjovsky in the GAN context.
Comparisons with other variants of Wasserstein GAN is proposed on MNIST.
I see little novelty in the paper. The derivation of the primal version of the problem is already
given in
Cuturi, M., & Doucet, A. (2014, January). Fast computation of Wasserstein barycenters. In ICML (pp. 685-693).
Using optimal transport computed on batches rather the on the whole dataset is already used in (among
others)
 Genevay, A., Peyré, G., & Cuturi, M. (2017). Learning generative models with sinkhorn divergences. AISTATS
 Damodaran, B. B., Kellenberger, B., Flamary, R., Tuia, D., & Courty, N. (2018). DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation. ECCV
Also, the claim that the exact empirical Wasserstein distance is optimized is not true. The gradients, evaluated on
batches, are biased. Unfortunately, the Wasserstein distance does not enjoy similar U-statistics as MMD. It is very
well described in the paper (Section 3):
https://openreview.net/pdf?id=S1m6h21Cb
Computing the gradients of Wasserstein on batches might be seen a kind of regularization, but it remains to be
proved and discussed.
Finally, the experimental validation appears insufficient to me (as only MNIST or toy datasets are considered).
Typos:
 Eq (1) and (2): when taken over the set of all Lipschitz-1 functions, the max should be a sup


VARIATIONAL SMOOTHING IN RECURRENT NEURAL NETWORK LANGUAGE MODELS
We present a new theoretical perspective of data noising in recurrent neural network language models (Xie et al., 2017). We show that each variant of data noising is an instance of Bayesian recurrent neural networks with a particular variational distribution (i.e., a mixture of Gaussians whose weights depend on statistics derived from the corpus such as the unigram distribution). We use this insight to propose a more principled method to apply at prediction time and propose natural extensions to data noising under the variational framework. In particular, we propose variational smoothing with tied input and output embedding matrices and an element-wise variational smoothing method. We empirically verify our analysis on two benchmark language modeling datasets and demonstrate performance improvements over existing data noising methods.

This submission closely builds upon an earlier work (Xie et al., 2017, Gal and Ghahramani, 2016) and proposes a new data noising technique motivated by Bayesian RNNs. Specifically, the key contribution is to extend Gal and Ghahramani (2016) to word embedding noising, while drawing inspiration from trational data smoothing techniques. Some variants are discussed, including those motivated by linear interpolation and Kneser-Ney smoothing, just as in Xie et al., 2017. Empirical evaluation is performed with language modeling experiments, and the proposed methods outperforms comparable baselines. One can imagine such a technique could be useful in many other sequence tasks.
The paper is well-motivated and clearly written, and the experiments seem reasonable to me. Therefore I would vote for acceptance. My concern, which is not major, is that the proposed method might be a bit incremental based on Xie et al. (2017) and Gal and Ghahramani (2016).
Pros:
- Theoratical justification seems reasonably sound to me.
- Strong empirical performance.
Cons:
- It would be interesting to see how the proposed technique works when applied to state-of-the-art models.
Details:
- I'm not entirely familiar with Gal and Ghahramani (2016), but I'm assuming the discussion in Section 3 and how it extends to word embeddings are from this earlier work. Please correct me if I'm wrong, so that I can adjust my recommendation accordingly.
- I can't find anything describing how \sigma is determined.
- Page 4, the paragraph of `Training.` I can't parse `we go though sequence t multiple times`.
- \ell_2 Regularization for `VS` models. I'm confused here, isn't the coefficient for VS determined by Eq. 3? Why is it still tuned in Section 5.1?
- Elementwise smoothing: I'm confused why one needs to sample \alpha for each dimension. Can't it be done by sampling a mask, just as in dropout?


Semantic parsing, which maps a natural language sentence into a machinereadable representation of its meaning, is highly constrained by the limited annotated training data. Inspired by the idea of coarse-to-fine, we propose a generalto-detailed neural network (GDNN) by incorporating middle coarse cross-domain sketch (CDS) among utterances and their logic forms. For utterances in different domains, the General Network will extract CDS using an encoder-decoder model in a multi-task learning setup. Then for some utterances in a specific domain, the Detailed Network will generate the detailed target parts using a sequenceto-sequence architecture with advanced attention to both utterance and generated CDS. Our experiments show that compared to direct multi-task learning, CDS has improved the performance in semantic parsing task which converts users' requests into meaning representation language (MRL). We also use experiments to illustrate that CDS works by adding some constraints to the target decoding process, which further proves the effectiveness and rationality of CDS.

Overall Score 3
This paper introduces “Cross domain schemas” (CDS) for semantic parsing of utterances made to a virtual assistant. CDS captures similarities in requests according to the underlying actions or attributes being discussed, regardless of the user’s high-level intent. Also introduced is a model which leverages CDS to improve semantic parsing of utterances to a meaning representation language (MRL). This model first parses an utterance to CDS, then uses an encoding of the CDS jointly with the utterance encoding to decode a meaning representation. By treating different intents as separate domains, the authors construct a multi-task learning setup for CDS and MRL parsing. Results are provided for the Snips dataset of virtual assistant queries.
Unfortunately, this paper fails to sufficiently explain its main proposal, the CDS. The stated goal is to explicitly define the cross-domain features that would otherwise be implicitly learned by the parameters of a neural network, yet no explicit definition is given. No rough quantification of how many or what percent of features appear across domains is provided. Rather, significant time and space is given to describing a fairly unsophisticated two-decoder model for inserting the mysterious CDS representation into the final decoding task.
The paper ignores standard semantic parsing datasets (GeoQuery and ATIS) due to their size and scope. However, comparable models (Goo et al. 2018) are trained and tested on ATIS. Moreover, an evaluation on a small, unseen target domain would be the perfect justification for the kind of cross-domain learning proposed here.
Instead, this paper opts only to evaluate on the recent Snips dataset. This dataset seems to be best suited to evaluating intent classification and slot filling (intent-slot), but the current work fails to improve over what Goo et al. 2018 report on this data. In the current work, the Snips dataset is used to evaluate MRL parsing, where the CDS model shows improvements over other seq2seq models. However, since MRL can be parsed from intent-slot format by predefined rules, it is uncertain whether the CDS model outperforms the Goo et al. model at even the task of MRL parsing (no such comparison is provided).
Overall, the paper suffers from some clarity issues especially regarding the definition and value of CDS. The model provided may be slightly original but is quite similar to the model of Dong and Lapata 2018. The significance of this work is questionable due to the poor comparison with recently released baseline models for the more common intent-slot task.
Pros
Introduces “Cross Domain Schemas” (CDS) for semantic parsing, which help improve robustness of semantic parsers by allowing models to learn patterns in one domain for use in another.
Through the use of CDS, train semantic parsers in a multi-task learning setup
Cons
CDS is not described in sufficient detail. In particular, the possible actions and attributes are not defined.
The model is described as “multi-task learning”, however all tasks are parsing requests made of a virtual assistant.
Results on standard data for semantic parsing such as GEO or ATIS are not reported.
The model does not appear to improve the results on the Snips dataset compared to the paper that introduces this dataset. Thus, the value of CDS is difficult to judge.
No per-domain analysis of the impact of CDS is provided.


A CONVERGENCE ANALYSIS OF GRADIENT DESCENT FOR DEEP LINEAR NEURAL NETWORKS
We analyze speed of convergence to global optimum for gradient descent training a deep linear neural network (parameterized as x → W N W N −1 · · · W 1 x) by minimizing the 2 loss over whitened data. Convergence at a linear rate is guaranteed when the following hold: (i) dimensions of hidden layers are at least the minimum of the input and output dimensions; (ii) weight matrices at initialization are approximately balanced; and (iii) the initial loss is smaller than the loss of any rank-deficient solution. The assumptions on initialization (conditions (ii) and (iii)) are necessary, in the sense that violating any one of them may lead to convergence failure. Moreover, in the important case of output dimension 1, i.e. scalar regression, they are met, and thus convergence to global optimum holds, with constant probability under a random initialization scheme. Our results significantly extend previous analyses, e.g., of deep linear residual networks (Bartlett et al., 2018).

This paper studies the convergence of gradient descent on the squared loss of deep linear neural networks. The authors prove linear convergence rate if (1) the network dimensions are big enough so that the full product can have full rank, (2) the singular values of each weight matrices are approximately the same, (3) the initialized point is “close enough” to the target.
First of all, this paper is well-written. It reads smoothly, effectively presents the key ideas and implications of the result, and properly answers to possible concerns that arise while reading. The improvement over the previous work ([Bartlett et al 18’]) is quite substantial.
Deep linear neural networks are important, and having a good understanding of linear neural networks can provide us useful insights for understanding the more complex ones, i.e., the nonlinear neural networks. In that regard, I really liked the discussion at the end of Section 3.1. My general opinion for this paper is acceptance, but I also have a number of concerns and questions.
My main concern about the study of GD on linear neural network is whether we really get any “benefit” or “acceleration” from depth, i.e., is GD on linear neural nets any faster than GD on linear models. It’s been shown that we get acceleration in some cases (e.g., $\ell_p$ regression when $p>2$ [Arora et al. 18’]), but some other results (e.g., [Shamir 18’] mentioned in Section 5) show that GD on linear neural nets (when weight matrices are all scalar) suffer exponential (in depth) increase in convergence time at near zero region, due to the vanishing gradient phenomenon. From my understanding, this paper circumvents this problem by assuming deficiency margin, because in the setting of [Shamir 18’], deficiency margin means that the initialized product ($W_{1:N}$) has the same sign as $\Phi$ and far enough from zero, so we don’t have to pass through the near-zero region.
Even with the deficiency margin assumption, the exponential dependence in depth can also be observed in this paper, if we use independent initialization of each weight matrices. In Claim 3, in order to get the probability 0.49 result, the margin $c$ must be very small (O(1/N^N)) as N goes to infinity, resulting in very small $\delta$ and $\eta$ in Theorem 1, and convergence time $T$ exploding in depth. On the other hand, if we fix $0 < c < 1$, then the probability of satisfying deficiency margin will be smaller and smaller as $N$ increases. Is this “blow-up in N” problem due to the fact that the loss is l2? Or am I making false claims? I would like to hear the authors’ opinion about this.
The paper proposes a balanced initialization scheme that doesn’t suffer exponential blow up (Procedure 1 and Theorem 2), but even with this, the learning rate must decay to zero in polynomial rate in N, also resulting in polynomial increase in convergence time as depth increases. Moreover, this type of initialization scheme (specifically tailored for linear neural networks) is not what people would do in practice; we normally would initialize each layer at random, and may suffer the problems discussed in the above paragraph. That is why I’d love to hear about the authors’ future work on layer-wise independent initialization, as noted in the conclusion section.
Below, I’ll list specific concerns/questions/comments.
* In my opinion, the statements about “necessity” of two key assumptions are too strong, because the authors only provide counterexamples of non-convergence. As [Theorem 3, Shamir 18’] shows (although in scalar case), even when the assumptions are not satisfied, a convergence rate $O(exp(N) * log(1/\epsilon))$ is possible. It will be an interesting future work to clearly delineate the boundary between convergence and non-convergence.
* In Thm 2 and Claim 3, what happens if dimension $d_0$ is smaller? What is the reason that you had to restrict it to high dimension? Is it due to high variance with few samples?
* In Thm 2, constants $d’_0$ and $a$ hide the dependence of the result on p, but I would suggest stating the dependence of those parameters on p, and also dependence on other parameters such as N.
* In Section 5, there is a statement “This negative result, a theoretical manifestation of the “vanishing gradient problem”, is circumvented by balanced initialization.” Can you elaborate more on that? If my understanding is correct, there is still $\sigma_min$ multiplier in Eq (9), which means that at near-zero regions, the gradient will still vanish.
I appreciate the authors for their efforts, especially on the heavy math in the proof of the main theorem. I would like to hear your comments and/or corrections (especially on my “T blowing up in N” claim) and discuss further.


UNSUPERVISED DOMAIN ADAPTATION FOR DISTANCE METRIC LEARNING
Unsupervised domain adaptation is a promising avenue to enhance the performance of deep neural networks on a target domain, using labels only from a source domain. However, the two predominant methods, domain discrepancy reduction learning and semi-supervised learning, are not readily applicable when source and target domains do not share a common label space. This paper addresses the above scenario by learning a representation space that retains discriminative power on both the (labeled) source and (unlabeled) target domains while keeping representations for the two domains well-separated. Inspired by a theoretical analysis, we first reformulate the disjoint classification task, where the source and target domains correspond to non-overlapping class labels, to a verification one. To handle both within and cross domain verifications, we propose a Feature Transfer Network (FTN) to separate the target feature space from the original source space while aligned with a transformed source space. Moreover, we present a non-parametric multi-class entropy minimization loss to further boost the discriminative power of FTNs on the target domain. In experiments, we first illustrate how FTN works in a controlled setting of adapting from MNIST-M to MNIST with disjoint digit classes between the two domains and then demonstrate the effectiveness of FTNs through state-of-the-art performances on a cross-ethnicity face recognition problem.

The authors studied an interesting problem of unsupervised domain adaptation when the source and the target domains have disjoin labels spaces. The paper proposed a novel feature transfer network, that optimizes domain adversarial loss and domain separation loss.
Strengths:
1) The proposed approach on Feature Transfer Network was novel and interesting.
2) The paper was very well written with a good analysis of various choices.
3) Extensive empirical analysis on multi-class settings with a traditional MNIST dataset and a real-world face recognition dataset.
Weakness:
1) Practical considerations addressing feature reconstruction loss needs more explanation.
Comments:
The technical contribution of the paper was sound and novel. The paper considered existing work and in a good way generalizes and extends into disjoint label spaces. It was easy to read and follow, most parts of the paper including the Appendix make it a good contribution. However, the reviewer has the following suggestions"
1. Under the practical considerations for preventing the mode collapse via feature reconstruction, how is the reference network trained? In the Equation(6) for feature reconstruction, the f_ref term maps the source and target domain examples to new feature space. What do you mean by references network trained on the label data? Please clarify.
2. Under the practical considerations for replacing the verification loss, it is said that "Our theoretical analysis suggests to use a verification
the loss that compares the similarity between a pair of images" - Can you please cite the references to make it easier for the reader to follow.


SELFLESS SEQUENTIAL LEARNING
Sequential learning, also called lifelong learning, studies the problem of learning tasks in a sequence with access restricted to only the data of the current task. In this paper we look at a scenario with fixed model capacity, and postulate that the learning process should not be selfish, i.e. it should account for future tasks to be added and thus leave enough capacity for them. To achieve Selfless Sequential Learning we study different regularization strategies and activation functions. We find that imposing sparsity at the level of the representation (i.e. neuron activations) is more beneficial for sequential learning than encouraging parameter sparsity. In particular, we propose a novel regularizer, that encourages representation sparsity by means of neural inhibition. It results in few active neurons which in turn leaves more free neurons to be utilized by upcoming tasks. As neural inhibition over an entire layer can be too drastic, especially for complex tasks requiring strong representations, our regularizer only inhibits other neurons in a local neighbourhood, inspired by lateral inhibition processes in the brain. We combine our novel regularizer with state-of-the-art lifelong learning methods that penalize changes to important previously learned parts of the network. We show that our new regularizer leads to increased sparsity which translates in consistent performance improvement on diverse datasets.

REVISION AFTER REBUTTAL
While the revision does not address all of my concerns about clarity, it is much better. I still think that the introduction is overly long and the subsequent sections repeat information; if this were shortened there could be room for some of the figures that are currently in appendix. I appreciate the new figures; I think that it would be great if especially figure 10 were included in the main paper.
I agree with the other two reviewers that the work is somewhat incremental, but the differences are well explained, the experimental results are interesting (particularly the differences of parameter vs representation-based sparsity, and the plots in appendix showing neuron importance over tasks), and the progression from SNI to SLNID is well-presented.  I think overall that this paper is a good contribution and I recommend acceptance. I have updated my review to a 7.
===============
"Activations" "Representation" and "Outputs" are used somewhat interchangably throughout the work; for anyone not familiar it might be worth mentioning something about this in the intro.

Problem setting is similar to open set learning (classification); could be worth mentioning algorithms for this in the related work which attempt to set aside capacity for later tasks.
Results are presented and discussed in the introduction, and overall the intro is a bit long, resulting in parts later being repetitive.
Worth discussing sparsity vs. distributed representations in the intro, and how/where we want sparsity while still having a distributed representation.
Should be made clear that this is inspired by one kind of inhibition, and there are many others (i.e. inhibition in the brain is not always about penalizing neurons which are active at the same time, as far as I know)
Changes in verb tense throughout the paper make it hard to follow sometimes. Be consistent about explaining equations before or after presenting them, and make sure all terms in the equation are defined (e.g. SNI with a hat is used before definition). Improper or useless "However" or "On the other hand" to start a lot of sentences.
Figure captions could use a lot more experimental insight and explanation - e.g. what am I supposed to take away from Figure 10 (in appendix B4), other than that the importance seems pretty sparse? It looks to me like there is a lot of overlap in which neurons are important or which tasks, which seems like the opposite of what the regularizer was trying to achieve. This is a somewhat important point to me; I think this interesting and I'm glad you show it, but it seems to contradict the aim of the regularizer.
How does multi-task joint training differ from "normal" classification? The accuracies especially for CIFAR seem very low.
Quality: 7/10 interesting and thoughtful proposed regularizer and experiments; I would be happy to increase this rating if the insights from experiments, especially in the appendix, are a bit better explained
Clarity:  6/10 things are mostly clearly explained although frequently repetitive, making them seem more confusing than they are. If the paper is reorganized and the writing cleaned up I would be happy to increase my rating because I think the work is good.
Originality: 8/10 to my knowledge the proposed regularizer is novel, and I think think identifying the approach of "selfless" sequential learning is valuable (although I don't like the name)
Significance: 7/10 I am biased because I'm interested in LLL, but I think these problems should receive more attention.
Pros:
 - proposed regularizer is well-explained and seems to work well, ablation study is helpful
Cons:
 - the intro section is almost completely repetitive of section 3 and could be significantly shortened, and make more room for some of the experimental results to be moved from the appendix to main text
 - some wording choices and wordiness make some sentences unclear, and overall the organization and writing could use some work
Specific comments / nits: (in reading order)
1. I think the name "selfless sequential learning" is a bit misleading and sounds like something to do with multiagent cooperative RL; I think "forethinking" or something like that that is an actual word would be better, but I can't think of a good word... maybe frugal?
2.  Mention continual/lifelong learning in the abstract
3. "penalize changes" maybe "reduce changes" would be better?
4. "in analogy to parameter importance" cite and explain parameter importance
5. "advocate to focus on selfless SL" focus what? For everyone doing continual learning to focus on methods which achieve that through leaving capacity for later tasks? This seems like one potentially good approach, but I can imagine other good ones (e.g. having a task model)
6. LLL for lifelong learning is defined near the end of the intro, should be at the beginning when first mentioned
7. "lies at the heart of lifelong learning" I would say it is an "approach to lifelong learning"
8. "fixed model capacity" worth being specific that you mean (I assume) fixed architecture and number of parameters
9. "those parameters by new tasks" cite this at the end of the sentence, otherwise it is unclear what explanation goes with which citation
10.  "hard attention masks, and stored in an embedding" unclear what is stored in the embedding. It would be more helpful to explain how this method relates to yours rather than just describing what they do.
11. I find the hat notation unclear; I think it would be better just to have acronyms for each setting and write out the acronyms in the caption
12."richer representation is needed and few active neurons can be tolerated" should this be "more active neurons"?
13. Comparison with state of the art section is repetitive of the results sections


Batch Normalization (BN) and its variants have seen widespread adoption in the deep learning community because they improve the training of deep neural networks. Discussions of why this normalization works so well remain unsettled. We make explicit the relationship between ordinary least squares and partial derivatives computed when back-propagating through BN. We recast the backpropagation of BN as a least squares fit, which zero-centers and decorrelates partial derivatives from normalized activations. This view, which we term gradientleast-squares, is an extensible and arithmetically accurate description of BN. To further explore this perspective, we motivate, interpret, and evaluate two adjustments to BN.

The primary technical contribution comes from Section 2, where it is demonstrated that the normalized back-propagated gradients obtained from a BN layer can be viewed as the residuals of the gradients obtained without BN regressed via a simple two-parameter model of the activations.  In some sense though this result is to be expected, since centering data (i.e., removing the mean as in BN) can be generically viewed as computing the residuals after a least squares fit of a single constant, and similarly for de-trending with respect to a single independent variable, in this case the activations.  So I'm not sure that Theorem 1 is really that much of an insightful breakthrough, even if it may be nice to work through the precise details in the specific case of a BN layer and the relationship to gradients.
But beyond this a larger issue is as follows:  This paper is framed as taking a step in explaining why batch normalization (BN) works so well.  For example, even the abstract mentions this as an unsettled issue in motivating the proposed analysis.  However, to me the interpretation of BN as introducing a form of least squares fit does not really extend our understanding of why it actually works better in practice, and this is the biggest disconnect of the paper.  The new perspective presented might be another way to interpret BN layers, but it unfortunately remains mostly unanswered exactly why this new perspective is relevant in actually explaining BN behavior.
The presented normalization theory is also used to motivate heuristic modifications to standard BN schemes.  For example, the paper proposed concatenating BN with a layer normalization layer, demonstrating some modest improvement on CIFAR-10 data.  But again, I don't see how viewing these normalization schemes as least-squares residuals motivates such concatenation any more than the merits of the original versions themselves.  Moreover, it is not even clear that BN+LN is in fact generally better since only a single data set is considered.  There are also no comparisons against competing BN modifications such as switch normalization (Luo et al. 2018) which also involves a hybrid method combining aspects of LN and BN.  Why not compare against approaches like this?
To conclude, in Section 6 the paper asks "Why do empirical improvements in neural networks with BN keep the gradient-least-squares residuals and drop the explained portion?"  But this question is not at all answered but rather deferred to future work.  For me this was a disappointment as this would seem to be an essential ingredient for actually developing a meaningful theory for why BN is helpful in practice.
Other comments:
* The analysis from Section 2, including Theorem 1, assume that the BN parameters c and b can be ignored (presumably this means fixing c = 1 and b = 0).  I did not carefully check the details, but do all the same derivations and conclusions still seamlessly go through when these parameters have general values that deviate from this standard initialization?  If not, then I don't really see what is the practical relevance, since once learning begins, both b and c will typically shift to arbitrary values.  Below eq. (1) it states that c and b are only ignored for clarity, but then later I did not see any subsequent discussion to handle the general case, which is what would be actually needed for explaining BN behavior in practice.
* Please run a speck-checker.  Example, "On some leve, the matrix gradient ..."
* The paper cites (Lipton and Steinhardt, 2018) in arguing that reasons for the effectiveness of BN are lacking.  Indeed (Lipton and Steinhardt, 2018) criticize the original BN paper for conflating speculation with explanation, or more precisely, framing speculation about why BN should be helpful as an actual true explanation without clear evidence.  But to me this submission is hovering somewhere in the same category, speculating that regressing away certain portions of the gradient could be useful but never really providing concrete evidence for why this should offer an improvement.


A CLOSER LOOK AT FEW-SHOT CLASSIFICATION
Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples. While significant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difficult. In this paper, we present 1) a consistent comparative analysis of several representative few-shot classification algorithms, with results showing that deeper backbones significantly reduce the performance differences among methods on datasets with limited domain differences, 2) a modified baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the miniImageNet and the CUB datasets, and 3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classification algorithms. Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones. In a realistic cross-domain evaluation setting, we show that a baseline method with a standard fine-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.

This paper gives a nice overview of existing works on few-shot learning. It groups them into some intuitive categories and meanwhile distills a common framework (Figure 2) employed by the methods. Moreover, the authors selected four of them, along with two baselines, to experimentally compare their performances under a cleaned experiment protocol.
The experiments cover three few-shot learning scenarios respectively for generic object recognition, fine-grained classification, and cross-domain adaptation. While I do *not* think the third scenario is “more practical”, it is certainly nice to have it included in the experiments.
The experiment setup is unfortunately questionable. Since there is a validation set, one should use it to determine the free parameters (e.g., the number of epochs, learning rates, etc.). However, it seems like the same set of free parameters are used for different methods, making the comparison unfair because this set may favor some methods and yet hurt the others.
The results of RelationNet are missing in Table 4.
Another concern is that the same number of novel classes is used in the training and the testing stage. A more practical application of the learned meta model is to use it to handle different testing scenarios. There could be five novel classes in one scenario, 10 novel classes in another, and 100 in the third, etc. The number of labeled examples per class may also vary from one testing scenario to anther.
It is misleading by the following: “Very recently, Motiian et al. (2017) addresses the few-shot domain adaptation problem.” There are a few variations in domain adaptation (DA). The learner has access to the fully labeled source domain and a small set of labeled target examples in supervised DA, to the source domain, a couple of labeled target examples, and many unlabeled target examples in semi-supervised DA, and to the source domain and many unlabeled target data points in the unsupervised DA. These have been studied long before (Motiian et al., 2017), for instance the works of Saenko et al. (2010) and Gong et al. (2013).
[ref] Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. InEuropean conference on computer vision 2010 Sep 5 (pp. 213-226). Springer, Berlin, Heidelberg.
[ref] Gong B, Grauman K, Sha F. Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. InInternational Conference on Machine Learning 2013 Feb 13 (pp. 222-230).
Overall, the paper is well written and may serve as a nice survey of existing works on few-shot learning. The unified experiment setup can facilitate the future research for fair comparisons, along with the three testing scenarios. However, I have some concerns as above about the experiment setups and hence also the conclusions.


This paper proposes ASAL, a new pool based active learning method that generates high entropy samples. Instead of directly annotating the synthetic samples, ASAL searches similar samples from the pool and includes them for training. Hence, the quality of new samples is high and annotations are reliable. ASAL is particularly suitable for large data sets because it achieves a better run-time complexity (sub-linear) for sample selection than traditional uncertainty sampling (linear). We present a comprehensive set of experiments on two data sets and show that ASAL outperforms similar methods and clearly exceeds the established baseline (random sampling).
In the discussion section we analyze in which situations ASAL performs best and why it is sometimes hard to outperform random sample selection. To the best of our knowledge this is the first adversarial active learning technique that is applied for multiple class problems using deep convolutional classifiers and demonstrates superior performance than random sample selection 1 .

The paper presents a pool-based active learning method that achieves sub-linear
runtime complexity while generating high-entropy samples, as opposed to linear
complexity of more traditional uncertainty sampling (i.e., max-entropy) methods.
This is achieved by using a generative adversarial network (GAN) to generate
high-entropy samples that are then used by a nearest neighbor method to pick
samples from a pool, that are closest to the generated samples. The sub-linear
complexity is achieved through the use of a k-d tree, combined with the fact
that similarity is computed on the feature space and samples can thus be indexed
once (as the feature space does not change while training).
The proposed idea builds on top of previously published work on Generative
Adversarial Active Learning (GAAL). The main difference is the added nearest
neighbor component, as GAAL is directly using the generated examples, thus
achieving constant runtime complexity, rather than sub-linear.
I like the overall direction and the idea of being able to perform uncertainty
sampling in sub-linear time. The approach is interesting. However, the results
presented in the paper are not strong and I do not see whether or not I should
be using this method over uncertainty sampling. Most importantly, the results
are strongest only for the MNIST experiments, which are over a small dataset.
Given that the method is motivated by the scalability argument, I would like to
see at least one large scale experiment where it performs well, and more
specifically, outperform random sampling. Also, I would really like to see a
more principled and thorough experimental investigation with justifications for
the configurations used and with more comparisons to alternative approaches,
such as GAAL, which has constant complexity.
I believe that it would be better for your paper if you work a bit more on the
experimental evaluation and submit a revised version at a later deadline.
== Background and Method ==
The background and method sections are clear and easy to follow. One improvement
I can see is making figure 1 more clear, by maybe explicitly stating in the
figure what "G" and "F" are. One more point of interest is that the way you
perform sample matching makes some smoothness assumption about the feature
space as related to the classifier uncertainty. I perceive this as a smoothness
assumption on the decision boundary of the classifier and I do not know how true
is may be for deep neural networks, but I can see how it may be true for
logistic regression models and support vector machines (SVMs), depending on the
kernel used. I believe that this point and main assumption may be worth further
discussion, given that it is also about the main difference your method has with
respect to GAAL.
I do not have any other major comments for these sections as my main concerns
are about the experiments section.
== Experiments ==
In the experiments, it would be very useful to have plots against execution
time, given that the main motivation for this method is scalability. For
example, the method outperforms random sampling for small datasets, based on
number of samples, but what happens when you look at execution time? Given that
random sampling is very cheap, I imagine that it probably does better. Also, as
mentioned earlier, I would like to see at least one experiment using a big
dataset, where the method outperforms random sampling, as I am not currently
convinced of its usefulness.
Also, you present a lot of results and list observations but I felt there was
not much discussion as to why you observe/obtain some of the results. Given that
your method is not working very well for CIFAR, I would like to see a more
thorough investigation as to why that may be the case. This investigation could
conclude with some "tips" on when it may be a good idea to use your method over
GAAL, or uncertainty sampling, for example.
Regarding the experimental setup, I find lots of configuration choices very
arbitrary and have difficulty understanding how they were chosen. For example:
  - For the two-class MNIST you use classes "5" and "7" and for the two-class
    CIFAR you use classes "automobile" and "horse". Why is that? How did you
    pick the two classes to use in each case? Do the results match for other
    class pairs?
  - "learning rate of 0.01 that we decay by a factor of 10 at the 130th and
    140th epochs" -- end of page 6
  - "In contrast to the previous experiments we use a residual Wasserstein GAN
    with gradient penalty and soft consistency term" -- page 7 -- why do you
    make that change?
Questions:
  - Why do you think using Wasserstein GANs perform better than using DCGANs? -- section 5.3.1
  - Why not compare to GAAL in all of figures 3 and 4?
  - How/why were the number of samples you start with and sample in each round,
    chosen? Do you observe any difference if you increase/decrease the number of
    samples sampled in each round or if you start with fewer samples?
  - How/why were these model architectures chosen?


Analyzing Inverse Problems with Invertible Neural Networks
For many applications, in particular in natural science, the task is to determine hidden system parameters from a set of measurements. Often, the forward process from parameter-to measurement-space is well-defined, whereas the inverse problem is ambiguous: multiple parameter sets can result in the same measurement. To fully characterize this ambiguity, the full posterior parameter distribution, conditioned on an observed measurement, has to be determined. We argue that a particular class of neural networks is well suited for this task -so-called Invertible Neural Networks (INNs). Unlike classical neural networks, which attempt to solve the ambiguous inverse problem directly, INNs focus on learning the forward process, using additional latent output variables to capture the information otherwise lost. Due to invertibility, a model of the corresponding inverse process is learned implicitly. Given a specific measurement and the distribution of the latent variables, the inverse pass of the INN provides the full posterior over parameter space. We prove theoretically and verify experimentally, on artificial data and real-world problems from medicine and astrophysics, that INNs are a powerful analysis tool to find multi-modalities in parameter space, uncover parameter correlations, and identify unrecoverable parameters.

1) Summary
The authors propose to use invertible networks to solve ambiguous inverse problems. This is done by training one group of Real-NVP output variables supervised while training the other group via maximum likelihood under a Gaussian prior as done in the standard Real-NVP. Further, the authors suggest to not only train the forward model, but also the inverse model with an MMD critic, similar to previous works that used a more flexible GAN critic [1].
2) Clarity
The paper is easy to understand and the main idea is well-motivated.
3) Significance
The main contribution of this work is of conceptual nature and illustrates how invertible networks are a promising framework for many inverse problems. I really like the main idea and think it is inspiring. However, the experiments and technical contributions are rather limited.
Theoretical / ML contribution:
Using an MMD to factorize groups of latent variables is well-known and combining flow-based maximum likelihood training in the forward model with GAN-like objectives in the inverse model has been done before as well.
Experimental contribution:
I am not fully convinced by the experiments.
The inverse kinematics experiment shows that the posterior collapses from large uncertainty to almost a point for the right-most joint. This seems like a negative result to me.
The medical experiment also seems rather limited, because if I understand correctly the tissue data is artificial and the proposed INN only outperforms competitors (despite ABC) on two out of three measurements. Further, the authors should have explained the experimental setup of the tissue experiment better, as it is not a standard task in the field.
In the astronomy experiment figure 4 shows strong correlations between some of the z variables, the authors claim that this is a feature of their method, but I argue that they should not be present if training with the factorial prior was successful. It would be good to show the correlation between y and z variables as well if they show high dependencies, learning was not very successful. Simply eyeballing the shape of the posterior is not enough to conclude independence.
In summary, even though interesting, the significance of the experimental results is hard to judge and I am a bit worried that if the proposed model is making some strange mistakes on artificial toy-data, how well it will perform on challenging realistic problems.
4) Main Concerns
The authors claim that specifying a prior/posterior distribution in density modeling is complicated and typically the chosen distributions are too simplistic. This argument is, of course, valid, but they also have the same problem and specify z to be factorial Gaussian. So the same "hen-and-egg" problem applies here.
The authors also seem to suggest that they are the first to train flow-based models in forward and inverse direction, but this has already been done in the flow-GAN paper [1].
MMD does not easily scale to high-dimensional problems, this is not a problem here as all artificial problems considered are very low-dimensional. But when applying the proposed algorithm in realistic settings, one will likely need extensions of MMD, like used in MMD GANs, which would introduce min/max games on both sides of the network. This will likely be hard to train and constitutes a fundamental limitation of the approach that needs to be discussed.
5) Minor Concerns
- Some basic citations on normalizing flows seem to be missing, e.g. [2,3].
- How does one guarantee that padded regions are actually zero on output when padding input with zeros? Small variance in those dimensions could potentially code important information. Is this considered as part of y or z?
- The authors require the existence of inverse and set this equal to bijectivity, but injectivity would be sufficient.
- The authors mention that z is conditioned on y, but in their notation, the conditional density p(z|y) never shows up explicitly. It should be made clear, that p(z)=p(z|y) is a consequence of their additional MMD penalty and only holds at convergence.
[1] Grover et al., "Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models"
[2] Tabak and Turner, "Density estimation by dual ascent of the log-likelihood"
[3] Deco and Brauer, "Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures"


In a number of practical applications that rely on dimensionality reduction, the dataset or measurement process provides valuable side information that can be incorporated when learning low-dimensional embeddings. We propose the inclusion of pairwise representation constraints into autoencoders (AEs) with the goal of promoting application-specific structure. We use synthetic results to show that only a small amount of AE representation constraints are required to substantially improve the local and global neighborhood preserving properties of the learned embeddings. To demonstrate the efficacy of our approach and to illustrate a practical application that naturally provides such representation constraints, we focus on wireless positioning using a recently proposed channel charting framework. We show that representation-constrained AEs recover the global geometry of the learned low-dimensional representations, which enables channel charting to perform approximate positioning without access to global navigation satellite systems or supervised learning methods that rely on extensive measurement campaigns.

Description:
This paper presents a variant of deep neural network autoencoders for low-dimensional embedding, where pairwise constraints are incorporated, and applies it to wireless positioning.
The four constraint types are about enforcing pairwise distance between low-dimensional points to be close to a desired value or below a maximal desired value, either as an "absolute" constraint where one point is fixed or a "relative" constraint where both points are optimized. The constraints are encoded as  nonconvex regularization terms. In addition to the constraints the method has a standard autoencoder cost function.
Authors point out that if a suitable importance weighting is done, one constraint type yields a parametric version of Sammon’s mapping.
The method is tested on four simple artificial manifolds and on a wireless positioning task.
Evaluation:
Combining autoencoders with suitable additional regularizers can be a meaningful approach. However, I find the evaluation of the proposed method very insufficient: there are no comparisons to any other dimensionality reduction methods. For example, Sammon's mapping is mentioned several times but is not compared to, and a parametric version of t-SNE is also mentioned but not compared to even though it is parametric like the authors' proposed method. I consider that to be a severe problem in a situation where numerous such methods have been proposed previously and would be applicable to the data used here.
In terms of novelty I find the method somewhat lacking: essentially it is close to simply a weighted combination of an AE cost function and a Sammon's mapping cost function when using the FRD constraints. The other types of constraints add some more novelty, however.
Detailed comments:
"Autoencoders have been shown to consistently outperform other dimensionality-reduction algorithms on real-world datasets (van der Maaten et al., 2009)": this is too old a reference, nine years old, and it does not contain numerous dimensionality reduction algorithms proposed more recently, such as any neighbor embedding based dimensionality reduction methods. Moreover, the test in van der Maate et al. 2009 was only on five data sets and in terms of a continuity measure only, too little evidence to claim consistent outperforming of other algorithms.
"van der Maaten (2009) proposes the use of AEs to learn a parametric mapping between high-dimensional datapoints and low-dimensional representations by enforcing structure obtained via Student-t stochastic neighborhood embedding (t-SNE)": this is not a correct description, van der Maaten (2009) optimizes the AE using t-SNE cost function (instead of running some separate t-SNE step to yield structural constraints as the description seems to say).
"the FRD regularizer resembles that of Sammon's mapping": actually in the general form it resembles the multidimensional scaling stress; it only becomes close to Sammon's mapping if you additionally weight each constraint by the inverse of the original distance as you suggest.
It is unclear to me where the absolute distance constraints (FAD or MAD) arise from in the synthetic experiments. You write "for FAD one of the two representations... is a constant known prior to AE learning": how can you know the desired low-dimensional output coordinate (or distance from such a coordinate) in the synthetic data case?
This reference is incorrect: "Laurens van der Maaten, Eric Postma, and Jaap Van den Herik. Dimensionality reduction: A comparative review. In Journal of Machine Learning Research, volume 10, pp. 66–71, 2009." This article has not been published in Journal of Machine Learning Research. It is only available as a technical report of Tilburg University.


Despite recent advances in deep learning, neural networks suffer catastrophic forgetting when tasks are learned sequentially. We propose a conceptually simple and general framework for continual learning, where structure optimization is considered explicitly during learning. We implement this idea by separating the structure and parameter learning. During structure learning, the model optimizes for the best structure for the current task. The model learns when to reuse or modify structure from previous tasks, or create new ones when necessary. The model parameters are then estimated with the optimal structure. Empirically, we found that our approach leads to sensible structures when learning multiple tasks continuously. Additionally, catastrophic forgetting is also largely alleviated from explicit learning of structures. Our method also outperforms all other baselines on the permuted MNIST and split CIFAR datasets in continual learning setting.

The proposed approach aims to mitigate catastrophic forgetting in continual learning (CL) problems by structure learning: determining whether to reuse or adapt existing parameters, or initialise new ones, when faced with a new task. This is framed as an architecture search problem, applying ideas from Differentiable Architecture Search (DARTS). The approach is verified on the Permuted MNIST dataset and evaluated on the Visual Decathlon, showing an improvement.
I think this is an interesting idea with potential, and is worth exploring, and the paper is well-structured and easy to follow.
Unfortunately, I feel the paper fails to consider recent work on CL, both in terms of discussion and benchmarking. The only previous work that is compared is EWC, on permuted MNIST, and the Visual Decathlon performance is only compared to simple baselines (such as adding an adapter or fine tuning) which makes it difficult to gauge the contribution.
There are recent works, some with better results on more difficult problems, such as Variational Continual Learning [1], Progress and Compress [2], or (Variational) Generative Experience Replay [3][4].
Given the approach is based on dynamically adding parameters or modules, Progressive Networks and Dynamically Expandable Networks (both cited) are especially relevant and should be compared (I believe the former may be related to the “adapter” baseline, but this should be made explicit).
I have some questions / discussion points:
- What's the intuition behind implementing the “adapt” operator as additive bias over the previous weights, rather than just copying the previous weights and fine tuning?
- In the general case, if the architecture search is a continuous relaxation (softmax combination of operators), why is the "adapt" operator necessary? Wouldn't this already be a linear combination of new and old parameters? (In the example case of a 1×1 adaptor it makes sense, but this is a special restricted case which adapts with a smaller set of parameters)
- How is the structure regulariser backpropagated into the parameters of each layer? As I understand, it is composed of a constant discrete term z (number of parameters in each option), multiplied by architecture softmaxes alpha; the gradient with respect to each alpha is a constant, and so this has the effect of scaling the gradients of each operator.
- For the "reuse - tuned" case, isn’t the model effectively maintaining a new network for each task?
I also have a number of other comments:
- Reference to figure in page 6 should be figure 4, not 5.
- I think the readability of the paper would benefit from another few proofreads; there are a number of grammatical issues throughout, and several sentence fragments, eg. in the top para of page 2: “..., it has the potential to encourage information sharing. Since now the irrelevant part can be handled…”.
I would encourage the authors to strengthen the experimental comparison by incorporating stronger, external baselines, and improving some of the minor writing issues.
[1] Nguyen, Cuong V., et al. "Variational Continual Learning." ICLR, 2018.
[2] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." ICML, 2018.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." NIPS, 2017.
[4] Farquhar, Sebastian, and Yarin Gal. "Towards Robust Evaluations of Continual Learning." arXiv, 2018.


We introduce a new convolutional layer named the Temporal Gaussian Mixture (TGM) layer and present how it can be used to efficiently capture longer-term temporal information in continuous activity videos. The TGM layer is a temporal convolutional layer governed by a much smaller set of parameters (e.g., location/variance of Gaussians) that are fully differentiable. We present our fully convolutional video models with multiple TGM layers for activity detection. The experiments on multiple datasets including Charades and MultiTHUMOS confirm the effectiveness of TGM layers, outperforming the state-of-the-arts.

Summary:
This paper proposes a temporal convolution layer called Temporal Gaussian Mixture (TGM) for video classification tasks. Usually, video classification models can be decomposed into a feature extraction step, in which frames are processed separately or in local groups, and a classification step where labels are assigned considering the full video features/longer range dependencies. Typical choices for the classification step are Recurrent Neural Networks (RNNs) and variations, pooling layers and 1D temporal convolutions. TGMs are 1D temporal convolutions with weights defined by taking samples from a mixture of Gaussians. This allows to define large convolutional kernels with a reduced number of parameters -the mean and std of the gaussians- at the cost of having reduced expressiveness. The authors experiment replacing 1D temporal convolutions with TGMs in standard video classifications models and datasets and report state-of-the-art results.
Strengths:
[+] The idea of defining weights as samples from a Mixture of Gaussians is interesting. It allows to define convolutional kernels that scale with the number of channels (equivalent to the number of Gaussians) instead of the number of channels and receptive field.
Weaknesses:
[-] The explanation of the TGM layers is very confusing in its current state.
Certain aspects of TGMs that would help understand them are not clearly stated in the text. For example, it is not clear that 1) TGMs are actually restricted 1D convolutions as the kernel is expressed through a mixture of gaussians, 2) In a TGM the number of mixtures corresponds to the number of output channels in a standard 1D convolution and 3) despite TGMs are 1D convolutions, they are applied differently by keeping the number of frame features (output channels of a CNN) constant through them. Section 3 does not explain any of these points clearly and induces confusion.
[-] The comparisons in the experimental section are unfair for the baselines and do not justify the advantages of TGMs.
TGMs are 1D convolutions but they are applied to a different axis - 1D convolutions take as input channels the frame features. Instead, TGMs add a dummy axis, keep the number of frame features constant through them, and finally use a 1x1 convolution to map the resulting tensor to the desired number of classes. There is no reason why TGMs can’t be used as standard 1D convolutions taking as the first input channels the frame features - in other words, consider as many input mixture components as frame features for the first TGM. This is a crucial difference because the number of parameters of TGMs and the baselines is greatly affected by it.
The implicit argument in the paper is that TGMs perform better than standard 1D convolutions because they have less parameters. However, in the experiments they compare to 1D convolutions that have smaller temporal ranges – TGMs have a temporal kernel of 30 timesteps or 10 per layer (in a stack of 3 TGMs), whereas the 1D convolutions either have 5 or 10 timestep kernels according to section 4. Thus, it is impossible to know whether TGMs work better because 1) in these experiments they have longer temporal range – we would need a comparison to 1D convolutions applied in the same way and with the same temporal range to clarify it, 2) because they are applied differently – we would need a comparison when TGMs are used the same way as 1D convolutions, 3) because the reduced number of parameters leads to less overfitting – we would need to see training metrics and see the generalization gap when compared to equivalent 1D convolutions or 4) because the reduced number of parameters eases the optimization process. To sum up, it's true that TGMs would have a reduced number of parameters compared to the equivalent 1D convolutions with the same temporal range, but how does that translate to better results and, in this case, why is the comparison made with non-equivalent 1D convolutions? What would happen if the authors compared to 1D convolutions used in the same way as TGMs?
[-] To claim SOTA, there are comparisons missing to recently published methods.
In particular, [1] and [2] are well-known models that report metrics for the Charades dataset also used in this paper.
Rating:
At this moment I believe the TGM layer, while a good idea with potential, is not sufficiently well explained in the paper and is not properly compared to other baselines. Thus, I encourage the authors to perform the following changes:
-Rewrite section 3, better comparing to 1D convolutions and justifying why TGMs are not used in the same manner but instead using a dummy axis and a linear/1x1 conv layer at the end and what are the repercussions of it.
-Show a fair comparison to 1D convolutions as explained above.
-If claiming SOTA results, compare to [1] or [2] which use different methods to the one proposed in these paper but outperform the baselines used in the experiments.
[1] Wang, Xiaolong, et al. "Non-local neural networks." CVPR 2018
[2] Zhou, Bolei et al. "Temporal Relational Reasoning in Videos" ECCV 2018
----------------------
Update: In light of the authors' rebuttal I have updated my rating from 5 to 6.