Published as a conference paper at ICLR 2018 LEARNING GENERAL PURPOSE DISTRIBUTED SEN- TENCE REPRESENTATIONS VIA LARGE SCALE MULTI- TASK LEARNING
A lot of the recent success in natural language processing (NLP) has been driven by distributed vector representations of words trained on large amounts of text in an unsupervised manner. These representations are typically used as general purpose features for words across a range of NLP problems. However, extending this success to learning representations of sequences of words, such as sentences, remains an open problem. Recent work has explored unsupervised as well as supervised learning techniques with different training objectives to learn general purpose fixed-length sentence representations. In this work, we present a simple, effective multi-task learning framework for sentence representations that combines the inductive biases of diverse training objectives in a single model. We train this model on several data sources with multiple training objectives on over 100 million sentences. Extensive experiments demonstrate that sharing a single recurrent sentence encoder across weakly related tasks leads to consistent improvements over previous methods. We present substantial improvements in the context of transfer learning and low-resource settings using our learned general-purpose representations.

---- updates: ----
I had a ton of comments and concerns, and I think the authors did an admirable job in addressing them.  I think the paper represents a solid empirical contribution to this area and is worth publishing in ICLR.
---- original review follows: ----
This paper is about learning sentence embeddings by combining a bunch of training signals: predicting the next & previous sentences (skip-thought), predicting the sentence's translation, classifying entailment relationships between two sentences, and predicting the constituent parse of a sentence. This is a simple idea that combines a bunch of things from prior work into one framework and yields strong results, outperforming most prior work on most tasks.
I think this paper is impressive in how it scales up training to use so many tasks and such large training sets for each task.  That and its strong experimental results make it worthy of publication. It's not very surprising that adding more tasks and data improves performance on average across downstream tasks, but it is nice to see the experimental results in detail. While many people would think of this idea, few would have the resources and expertise necessary to do it justice.  I also like how the authors move beyond the standard sentence tasks to evaluate also on the Quora question duplicate task with different amounts of training data and also consider the sentence characteristic / syntactic property tasks.  It would be great if the authors could release their pretrained sentence representation model so that other researchers could use it.
I do have some nitpicks here and there with the presentation and exposition, and I am concerned that at times the paper appears to be minimizing its weaknesses, but I think these are things that can be addressed in the next revision. I understand that sometimes it's tempting to minimize one's weaknesses in order to get a paper accepted because the reviewers may not understand the area very well and may get hung up on the wrong things. I understand the area well and so all the feedback I offer below comes from a place of desiring this paper's publication while also desiring it to be as accurate and helpful for the community as possible.
Below I'll discuss my concerns with the experiments and description of the results.
Regarding the results in Table 2:
The results in Table 2 seem a little bit unstable, as it is unclear which setting to use for the classification tasks; maybe it depends on the kind of classification being performed. One model seems best for the sentiment tasks ("+2L +STP") while other models seem best for SUBJ and MPQA.  Adding parsing as a training task hurts performance on the sentence classification tasks while helping performance on the semantic tasks, as the authors note.  It is unclear which is the best general model.  In particular, when others write papers comparing to the results in this paper, which setting should they compare to?  It would be nice if the authors could discuss this.
The results reported for the CNN-LSTM of Gan et al. do not exactly match those of any single row from Gan et al, either v1 or v2 on arxiv or the published EMNLP version. How were those specific numbers selected?
The caption of Table 2 states "All results except ours are taken from Conneau et al. (2017)." However, Conneau et al (neither the latest arxiv version nor the published EMNLP version) does not include many of the results in the table, such as CNN-LSTM and DiscSent mentioned in the following sentence in the caption. Did the authors replicate the results of those methods themselves, or report them from other papers?
What does bold and underlining indicate in Table 2?  I couldn't find this explained anywhere.
At the bottom of Table 2, in the section with approaches trained from scratch on these tasks, I'd suggest including the 89.7 SST result of Munkhdalai and Yu (2017) and the 96.1 TREC result of Zhou et al. (2016) (as well as potentially other results from Zhou et al, since they report results on others of these datasets). The reason this is important is because readers may observe that the paper's new method achieves higher accuracies on SST and TREC than all other reported results and mistakenly think that the new method is SOTA on those tasks.  I'd also suggest adding the results from Radford et al. (2017) who report 86.9 on MR and 91.4 on CR.  For other results on these datasets, including stronger results in non-fixed-dimensional-sentence-embedding transfer settings, see results and references in McCann et al. (2017).  While the methods presented in this paper are better than prior work in learning general purpose, fixed-dimensional sentence embeddings, they still do not produce state-of-the-art results on that many of these tasks, if any.  I think this is important to note.
For all tasks for which there is additional training, there's a confound due to the dimensionality of the sentence embeddings across papers. Using higher-dimensional sentence embeddings leads to more parameters in the linear model being trained on the task data. So it is unclear if the increase in hidden units in rows with "+L" is improving the results because of providing more weights for the linear model or whether it is learning a better sentence representation.
The main sentence embedding results are in Table 2, and use the SentEval framework. However, not all tasks are included. The STS Benchmark results are included, which use an additional layer trained on the STS Benchmark training data just like the SICK tasks. But the other STS results, which use cosine similarity on the embedding space directly without any retraining, are only included in the appendix (in Table 7). The new approach does not do very well on those unsupervised tasks. On two years of data it is better than InferSent and on two years it is worse. Both are always worse than the charagram-phrase results of Wieting et al (2016a), which has 66.1 on 2012, 57.2 on 2013, 74.7 on 2014, and 76.1 on 2015.  Charagram-phrase trains on automatically-generated paraphrase phrase pairs, but these are generated automatically from parallel text, the same type of resource used in the "+Fr" and "+De" models proposed in this submission, so I think it should be considered as a comparable model.
The results in the bottom section of Table 7, reported from Arora et al (2016), were in turn copied from Wieting et al (2016b), so I think it would make sense to also cite Wieting et al (2016b) if those results are to be included. Also, it doesn't seem appropriate to designate those as "Supervised Approaches" as they only require parallel text, which is a subset of the resources required by the new model.
There are some other details in the appendix that I find concerning:
Section 8 describes how there is some task-specific tuning of which function to compute on the encoder to produce the sentence representation for the task.  This means that part of the improvement over prior work (especially skip-thought and InferSent) is likely due to this additional tuning. So I suppose to use these sentence representations in other tasks, this same kind of tuning would have to be done on a validation set for each task?  Doesn't that slightly weaken the point about having "general purpose" sentence representations?
Section 9 provides details about how the representations are created for different training settings. I am confused by the language here. For example, the first setting ("+STN +Fr +De") is described as "A concatenation of the representations trained on these tasks with a unidirectional and bidirectional GRU with 1500 hidden units each." I'm not able to parse this. I think the authors mean "The sentence representation h_x is the concatenation of the final hidden vectors from a forward GRU (with 1500-dimensional hidden vectors) and a bidirectional GRU (also with 1500-dimensional hidden vectors)". Is this correct?
Also in Sec 9: I found it surprising how each setting that adds a training task uses the concatenation of a representation with that task and one without that task. What is the motivation for doing this? This seems to me to be an important point that should be discussed in Section 3 or 4. And when doing this, are the concatenated representations always trained jointly from scratch with the special task only updating a subset of the parameters, or do you use the fixed pretrained sentence representation from the previous row and just concatenate it with the new one?  To be more concrete, if I want to get the encoder for the second setting ("+STN +Fr +De +NLI"), do I have to train two times or can I just train once?  That is, the train-once setting would correspond to only updating the NLI-specific representation parameters when training on NLI data; on other data, all parameters would be updated. The train-twice setting would first train a representation on "+STN +Fr +De", then set it aside, then train a separate representation on "+STN +Fr +De +NLI", then finally concatenate the two representations as my sentence representation.  Do you use train-once or train-twice?
Regarding the results in Table 3:
What do bold and underline indicate?
What are the embeddings corresponding to the row labeled "Multilingual"?
In the caption, I can't find footnote 4.
The caption includes the sentence "our embeddings have 1040 pairs out of 2034 for which atleast one of the words is OOV, so a comparison with other embeddings isn't fair on RW."  How were those pairs handled?  If they were excluded, then I think the authors should not report results on RW.  I suspect that most of the embeddings included in the table also have many OOVs in the RW dataset but still compute results on it using either an unknown word embedding or some baseline similarity of zero for pairs with an OOV. I think the authors should find some way (like one of those mentioned, or some other way) of computing similarity of those pairs with OOVs. It doesn't make much sense to me to omit pairs with OOVs.
There are much better embeddings on SimLex than the embeddings whose results are reported in the table. Wieting et al. (2016a) report SimLex correlation of 0.706 and Mrkšić et al. (2017) report 0.751.  I'd suggest adding the results of some stronger embeddings to better contextualize the embeddings obtained by the new method.  Some readers may mistakenly think that the embeddings are SOTA on SimLex since no stronger results are provided in the table.
The points below are more minor/specific:
Sec. 2:
In Sec. 2, the paper discusses its focus on fixed-length sentence representations to distinguish itself from other work that produces sentence representations that are not fixed-length. I feel the motivation for this is lacking. Why should we prefer a fixed-length representation of a sentence? For certain downstream applications, it might actually be easier for practitioners to use a representation that provides a representation for each position in a sentence (Melamud et al., 2016; Peters et al., 2017; McCann et al., 2017) rather than an opaque sentence representation. Some might argue that since sentences have different lengths, it would be appropriate for a sentence representation to have a length proportional to the length of the sentence.  I would suggest adding some motivation for the focus on fixed-length representations.
Sec. 4.1:
"We take a simpler approach and pick a new task to train on after every parameter update sampled uniformly. An NLI minibatch is interspersed after every ten parameter updates on sequence-to-sequence tasks"
These two sentences seem contradictory. Maybe in the first sentence "pick a new task" should be changed to "pick a new sequence-to-sequence task"?
Sec. 5.1:
typo: "updating the parameters our sentence" --> "updating the parameters of our sentence"
Sec. 5.2:
typo in Table 4 caption: "and The" --> ". The"
typo: "parsing improvements performance" --> "parsing improves performance"
In general, there are many missing citations for the tasks, datasets, and prior work on them. I understand that the authors are pasting in numbers from many places and just providing pointers to papers that provide more citation info, but I think this can lead to mis-attribution of methods. I would suggest including citations for all datasets/tasks and methods whose results are being reported.
References:
McCann, Bryan, James Bradbury, Caiming Xiong, and Richard Socher. "Learned in translation: Contextualized word vectors." CoRR 2017.
Melamud, Oren, Jacob Goldberger, and Ido Dagan. "context2vec: Learning Generic Context Embedding with Bidirectional LSTM." CoNLL 2016.
Mrkšić, Nikola, Ivan Vulić, Diarmuid Ó. Séaghdha, Ira Leviant, Roi Reichart, Milica Gašić, Anna Korhonen, and Steve Young. "Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints." TACL 2017.
Munkhdalai, Tsendsuren, and Hong Yu. "Neural semantic encoders." EACL 2017.
Pagliardini, Matteo, Prakhar Gupta, and Martin Jaggi. "Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features." arXiv preprint arXiv:1703.02507 (2017).
Peters, Matthew E., Waleed Ammar, Chandra Bhagavatula, and Russell Power. "Semi-supervised sequence tagging with bidirectional language models." ACL 2017.
Radford, Alec, Rafal Jozefowicz, and Ilya Sutskever. "Learning to generate reviews and discovering sentiment." arXiv preprint arXiv:1704.01444 2017.
Wieting, John, Mohit Bansal, Kevin Gimpel, and Karen Livescu. "Charagram: Embedding words and sentences via character n-grams." EMNLP 2016a.
Wieting, John, Mohit Bansal, Kevin Gimpel, and Karen Livescu. "Towards universal paraphrastic sentence embeddings." ICLR 2016b.
Zhou, Peng, Zhenyu Qi, Suncong Zheng, Jiaming Xu, Hongyun Bao, and Bo Xu. "Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling." COLING 2016.


Active learning (AL) aims to enable training high performance classifiers with low annotation cost by predicting which subset of unlabelled instances would be most beneficial to label. The importance of AL has motivated extensive research, proposing a wide variety of manually designed AL algorithms with diverse theoretical and intuitive motivations. In contrast to this body of research, we propose to treat active learning algorithm design as a meta-learning problem and learn the best criterion from data. We model an active learning algorithm as a deep neural network that inputs the base learner state and the unlabelled point set and predicts the best point to annotate next. Training this active query policy network with reinforcement learning, produces the best non-myopic policy for a given dataset. The key challenge in achieving a general solution to AL then becomes that of learner generalisation, particularly across heterogeneous datasets. We propose a multi-task dataset-embedding approach that allows dataset-agnostic active learners to be trained. Our evaluation shows that AL algorithms trained in this way can directly generalise across diverse problems.

The approach solves an important problem as getting labelled data is hard. The focus is on the key aspect, which is generalisation across heteregeneous data. The novel idea is the dataset embedding so that their RL policy can be trained to work across diverse datasets.
Pros:
1. The approach performs well against all the baselines, and also achieves good cross-task generalisation in the tasks they evaluated on.
2. In particular, they alsoevaluated on test datasets with fairly different statistics from the training datasets, which isnt very common in most meta-learning papers today, so it’s encouraging that the method works in that regime.
Cons:
1. The embedding strategy, especially the representative and discriminative histograms, is complicated. It is unclear if the strategy is general enough to work on harder problems / larger datasets, or with higher dimensional data like images. More evidence in the paper for why it would work on harder problems would be great.
2. The policy network would have to output a probability for each datapoint in the dataset U, which could be fairly large, thus the method is computationally much more expensive than random sampling. A section devoted to showing what practical problems could be potentially solved by this method would be useful.
3. It is unclear to me if the results in table 3 and 4 are achieved by retraining from scratch with an RBF SVM, or by freezing the policy network trained on a linear SVM and directly evaluating it with a RBF SVM base learner.
Significance/Conclusion: The idea of meta-learning or learning to learn is fairly common now. While they do show good performance, it’s unclear if the specific embedding strategy suggested in this paper will generalise to harder tasks.
Comments: There’s lots of typos, please proof read to improve the paper.
Revision: I thank the authors for the updates and addressing some of my concerns. I agree the computational budget makes sense for cross data transfer, however the embedding strategy and lack of larger experiments makes it unclear if it'll generalise to harder tasks. I update my review to 6.


THE KANERVA MACHINE: A GENERATIVE DISTRIBUTED MEMORY
We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them. Inspired by Kanerva's sparse distributed memory, it has a robust distributed reading and writing mechanism. The memory is analytically tractable, which enables optimal on-line compression via a Bayesian update-rule. We formulate it as a hierarchical conditional generative model, where memory provides a rich data-dependent prior distribution. Consequently, the top-down memory and bottom-up perception are combined to produce the code representing an observation. Empirically, we demonstrate that the adaptive memory significantly improves generative models trained on both the Omniglot and CIFAR datasets. Compared with the Differentiable Neural Computer (DNC) and its variants, our memory model has greater capacity and is significantly easier to train.

The generative model comprises a real-valued matrix M (with a multivariate normal prior) that serves
as the memory for an episode (an unordered set of datapoints). For each datapoint a marginally independent
latent variable y_t is used to index into M and realize a conditional density
of another latent variable z. z_t is used to generate the data.
The proposal of learning with a probabilistic memory is interesting and the framework proposed is elegant and cleanly explained. The model is evaluated on the following tasks:
* Qualitative results on denoising and one-shot generation using the Omniglot dataset.
* Qualitative results on sampling from the model using the CIFAR dataset.
* Likelihood estimation on the Omniglot dataset
Questions and concerns:
The model appears novel and is interesting, the experiments, however, are lacking in that they
do not compare against other any recently proposed memory augmented deep generative models [Bornschein et al] and [Li et. al] (https://arxiv.org/pdf/1602.07416.pdf). At the very minimum, the paper should include a discussion and a comparison with the latter. Doing so will help better understand what is gained from using retaining a probabilistic form of memory versus a determinstic memory indexed with attention as in [Li et. al].
How does the model perform as a function of varying T (size of episodes) during training? It would be interesting to see how well the model performs in the limiting case of T=1.
What is the task being solved in Section 4.4 by the DNC and the Kanerva machine? Please state this in the main paper.
Training and Evaluation: There is a mismatch in the training and evaluation procedure the implications of which I don't
fully understand yet. The text states that the model was trained where each observation in an episode comprised randomly sampled datapoints. This corresponds to a generative process where (1) a memory is randomly drawn, (2) each observation in the episode is an independent draws from the memory conditioned decoder. During training,
points in an episode are randomly selected. At test time, (if I understand correctly, please correct me if I haven't), the model is evaluated by having multiple copies of the same test point within an episode. Is that correct? If so, doesn't that correspond to evaluating the model under a different generative assumption? Why is this OK?
Likelihood evaluation: Could you expand on how the ELBO of 68.3 is computed under the model for a single test image in the Omniglot dataset? The text says that the likelihood of each data-point was divided by T (the length of the episode considered). This seems at odds with models, such as DRAW, evaluate the likelihood -- once at the end of the generative drawing process. What is the per-pixel likelihood obtained on the CIFAR dataset and what is the likelihood on a model where T=1 (for omniglot/cifar)?
Using Labels: Following up on the previous point, what happens if labelled information from Omniglot or CIFAR is used to define points within an episode during the training procedure? Does this help or hurt performance?
For the denoising comparison, how do the results compare to those obtained if you simulate a Markov Chain (sample latent state conditioned on noisy image, sample latent state, sample denoised observation, repeat using denoised observation) using a VAE?


Published as a conference paper at ICLR 2018 HIERARCHICAL AND INTERPRETABLE SKILL ACQUI- SITION IN MULTI-TASK REINFORCEMENT LEARNING
Learning policies for complex tasks that require multiple different skills is a major challenge in reinforcement learning (RL). It is also a requirement for its deployment in real-world scenarios. This paper proposes a novel framework for efficient multi-task reinforcement learning. Our framework trains agents to employ hierarchical policies that decide when to use a previously learned policy and when to learn a new skill. This enables agents to continually acquire new skills during different stages of training. Each learned task corresponds to a human language description. Because agents can only access previously learned skills through these descriptions, the agent can always provide a human-interpretable description of its choices. In order to help the agent learn the complex temporal dependencies necessary for the hierarchical policy, we provide it with a stochastic temporal grammar that modulates when to rely on previously learned skills and when to execute new skills. We validate our approach on Minecraft games designed to explicitly test the ability to reuse previously learned skills while simultaneously learning new skills.

Summary:
This paper proposes an approach to learning hierarchical policies in a lifelong learning context. This is achieved by stacking policies - an explicit "switch" policy is then used to decide whether to execute a primitive action or call the policy of the layer below it. Additionally, each task is encoded in a human-readable template, which provides interpretability.
Review:
Overall, I found the paper to be generally well-written and the core idea to be interesting. My main concern is about the performance against existing methods (no empirical results are provided), and while it does provide interpretability, I am not sure that other approaches (e.g. Tessler et al. 2017) could not be slightly modified to do the same. I think the paper could also benefit from at least one more experiment in a different, harder domain.
I have a few questions and comments about the paper:
The first paragraph claims "This precludes transfer of previously learned simple skills to a new policy defined over a space with differing states or actions". I do not see how this approach avoids suffering from the same problem? Additionally, approaches such as agent-space options [Konidaris and Barto. Building Portable Options: Skill Transfer in Reinforcement Learning, IJCAI 2007] get around at least the state part.
I do not quite follow what is meant by "a global policy is assumed to be executable by only using local policies over specific options". It sounds like this is saying that the inter-option policy can pick only options, and not primitive actions, which is obviously untrue. Can you clarify this sentence?
In section 3.1, it may be best to mention that the policy accepts both a state and task and outputs an action. This is stated shortly afterwards, but it was confusing because section 3.1 says that there is a single policy for a set of tasks, and so obviously a normal state-action policy would not work here.
At the bottom of page 6, are there any drawbacks to the instruction policy being defined as two independent distributions? What if not all skills are applicable to all items?
In section 5, what does the "without grammar" agent entail? How is the sampling from the switch and instruction policies done in this case?
While the results in Figures 4 and 5 show improvement over a flat policy, as well as the value of using the grammar, I am *very* surprised there is no comparison to existing methods. For example, Tessler's H-DRLN seems like one obvious comparison here, since it learns when to execute a primitive action and when to reuse a skill.
There were also some typos/small issues (I may have missed some):
pg 3: "In addition, previous work usually useS..."
pg 3. "we encode a human instruction to LEARN A..." (?)
pg 4. "...with A stochastic temporal grammar..."
pg 4. "... described above through A/THE modified..."
pg 6. "...TOTALLING six colors..."
There are some issues with the references (capital letters missing e.g. Minecraft)
It also would be preferable if the figures could appear after they are referenced in the text, since it is quite confusing otherwise. For example, Figure 2 contains V(s,g), but that is only defined much later on. Also, I struggled to make out the yellow box in Figure 2, and the positioning of Figure 3 on the side is not ideal either.


Direct policy gradient methods for reinforcement learning and continuous control problems are a popular approach for a variety of reasons: 1) they are easy to implement without explicit knowledge of the underlying model 2) they are an "end-to-end" approach, directly optimizing the performance metric of interest 3) they inherently allow for richly parameterized policies. A notable drawback is that even in the most basic continuous control problem (that of linear quadratic regulators), these methods must solve a non-convex optimization problem, where little is understood about their efficiency from both computational and statistical perspectives. In contrast, system identification and model based planning in optimal control theory have a much more solid theoretical footing, where much is known with regards to their computational and statistical properties. This work bridges this gap showing that (model free) policy gradient methods globally converge to the optimal solution and are efficient (polynomially so in relevant problem dependent quantities) with regards to their sample and computational complexities.

The paper studies the global convergence for policy gradient methods for linear control problems.
(1) The topic of this paper seems to have minimal connection with ICRL. It might be more appropriate for this paper to be reviewed at a control/optimization conference, so that all the technical analysis can be evaluated carefully.
(2) I am not convinced if the main results are novel. The convergence of policy gradient does not rely on the convexity of the loss function, which is known in the community of control and dynamic programming. The convergence of policy gradient is related to the convergence of actor-critic, which is essentially a form of policy iteration. I am not sure if it is a good idea to examine the convergence purely from an optimization perspective.
(3) The main results of this paper seem technical sound. However, the results seem a bit limited because it does not apply to neural-network function approximator. It does not apply to the more general control problem rather than quadratic cost function, which is quite restricted. I might have missed something here. I strongly suggest that these results be submitted to a more suitable venue.


TOWARDS REVERSE-ENGINEERING BLACK-BOX NEURAL NETWORKS
Many deployed learned models are black boxes: given input, returns output. Internal information about the model, such as the architecture, optimisation procedure, or training data, is not disclosed explicitly as it might contain proprietary information or make the system more vulnerable. This work shows that such attributes of neural networks can be exposed from a sequence of queries. This has multiple implications. On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -we show that the revealed internal information helps generate more effective adversarial examples against the black box model. On the other hand, this technique can be used for better protection of private content from automatic recognition models using adversarial examples. Our paper suggests that it is actually hard to draw a line between white box and black box models. The code is available at goo.gl/MbYfsv.

-----UPDATE------
Having read the responses from the authors, and the other reviews, I am happy with my rating and maintain that this paper should be accepted.
----------------------
In this paper, the authors trains a large number of MNIST classifier networks with differing attributes (batch-size, activation function, no. layers etc.) and then utilises the inputs and outputs of these networks to predict said attributes successfully. They then show that they are able to use the methods developed to predict the family of Imagenet-trained networks and use this information to improve adversarial attack.
I enjoyed reading this paper. It is a very interesting set up, and a novel idea.
A few comments:
The paper is easy to read, and largely written well. The article is missing from the nouns quite often though so this is something that should be amended. There are a few spelling slip ups ("to a certain extend" --> "to a certain extent", "as will see" --> "as we will see")
It appears that the output for kennen-o is a discrete probability vector for each attribute, where each entry corresponds to a possibility (for example, for "batch-size" it is a length 3 vector where the first entry corresponds to 64, the second 128, and the third 256). What happens if you instead treat it as a regression task, would it then be able to hint at intermediates (a batch size of 96) or extremes (say, 512).
A flaw of this paper is that kennen-i and io appear to require gradients from the network being probed (you do mention this in passing), which realistically you would never have access to. (Please do correct me if I have misunderstood this)
It would be helpful if Section 4 had a paragraph as to your thoughts regarding why certain attributes are easier/harder to predict. Also, the caption for Table 2 could contain more information regarding the network outputs.
You have jumped from predicting 12 attributes on MNIST to 1 attribute on Imagenet. It could be beneficial to do an intermediate experiment (a handful of attributes on a middling task).
I think this paper should be accepted as it is interesting and novel.
Pros
------
- Interesting idea
- Reads well
- Fairly good experimental results
Cons
------
- kennen-i seems like it couldn't be realistically deployed
- lack of an intermediate difficulty task


The concepts of unitary evolution matrices and associative memory have boosted the field of Recurrent Neural Networks (RNN) to state-of-the-art performance in a variety of sequential tasks. However, RNN still have a limited capacity to manipulate long-term memory. To bypass this weakness the most successful applications of RNN use external techniques such as attention mechanisms. In this paper we propose a novel RNN model that unifies the state-of-the-art approaches: Rotational Unit of Memory (RUM). The core of RUM is its rotational operation, which is, naturally, a unitary matrix, providing architectures with the power to learn long-term dependencies by overcoming the vanishing and exploding gradients problem. Moreover, the rotational unit also serves as associative memory. We evaluate our model on synthetic memorization, question answering and language modeling tasks. RUM learns the Copying Memory task completely and improves the state-of-the-art result in the Recall task. RUM's performance in the bAbI Question Answering task is comparable to that of models with attention mechanism. We also improve the state-of-the-art result by 0.001 to 1.189 bits-percharacter (BPC) test loss in the Character Level Penn Treebank (PTB) task. Moreover, our models achieve 0.002 BPC improvement to the validation too, which is to signify the applications of RUM to real-world sequential data. The universality of our construction, at the core of RNN, establishes RUM as a promising approach to language modeling, speech recognition and machine translation.

Summary:
This paper proposes a way to incorporate rotation memories into gated RNNs. They use a specific parametrization of the rotation matrices. They run experiments on several toy tasks and on language modelling with PTB character-level language modeling (which I would still consider to be toyish.)
Question:
Can the rotation proposed here cause unintentional forgetting by interleaving the memories? Because in some sense rotations are glorified summation in high-dimensions, if you do a full-rotation of a vector (360 degrees) you can end up in the same location. Thus the model might overwrite into its past memories.
Pros:
Proposes an interesting way to incorporate the rotation operations into the gated architectures.
Cons:
The specific choice of rotation operation is not very well justified.
This paper more or less uses the same architecture from Jing et al 2017 from EU-RNNs with a different parametrization for the rotation matrices.
The experiments are still limited to simple small-scale tasks.
General Comments:
The idea and the premise of this paper is interesting. In general the paper seems to be well-written. However the most important part of the paper section 3.1 is not very well justified. Why this particular parameterization of the rotation matrices is used and where does actually that come from? Can you point out to some citation? I think the RUM architecture section also requires better explanation on for instance why why R_t is parameterized that way (as a multiplicative function of R_{t-1}). A detailed ablation study would help too.
The model seems to perform really close to the GORU on Copying Task. I would be interested in seeing comparisons to GORU on “Associative Recall” as well. On QA task, which subset of bAbI dataset have you used? 1k or 10k training sets?
On language modelling there is only insignificant difference between the FS-LSTM-2 with FS-RUM model. This does not tell us much.


A NEURAL REPRESENTATION OF SKETCH DRAWINGS
We present sketch-rnn, a recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on a dataset of human-drawn images representing many different classes. We outline a framework for conditional and unconditional sketch generation, and describe new robust training methods for generating coherent sketch drawings in a vector format.

This paper introduces a neural network architecture for generating sketch drawings. The authors propose that this is particularly interesting over generating pixel data as it emphasises more human concepts. I agree. The contribution of this paper of this paper is two-fold. Firstly, the paper introduces a large sketch dataset that future papers can rely on. Secondly, the paper introduces the model for generating sketch drawings.
The model is inspired by the variational autoencoder. However, the proposed method departs from the theory that justifies the variational autoencoder. I believe the following things would be interesting points to discuss / follow up:
- The paper preliminarily investigates the influence of the KL regularisation term on a validation data likelihood. It seems to have a negative impact for the range of values that are discussed. However, I would expect there to be an optimum. Does the KL term help prevent overfitting at some stage? Answering this question may help understand what influence variational inference has on this model.
- The decoder model has randomness injected in it at every stage of the RNN. Because of this, the latent state actually encodes a distribution over drawings, rather than a single drawing. It seems plausible that this is one of the reasons that the model cannot obtain a high likelihood with a high KL regularisation term. Would it help to rephrase the model to make the mapping from latent representation to drawing more deterministic? This definitely would bring it closer to the way the VAE was originally introduced.
- The unconditional generative model *only* relies on the "injected randomness" for generating drawings, as the initial state is initialised to 0. This also is not in the spirit of the original VAE, where unconditional generation involves sampling from the prior over the latent space.
I believe the design choices made by the authors to be valid in order to get things to work. But it would be interesting to see why a more straightforward application of theory perhaps *doesn't* work as well (or whether it works better). This would help interesting applications inform what is wrong with current theoretical views.
Overall, I would argue that this paper is a clear accept.


Deep neural networks provide state-of-the-art performance for many applications of interest. Unfortunately they are known to be vulnerable to adversarial examples, formed by applying small but malicious perturbations to the original inputs. Moreover, the perturbations can transfer across models: adversarial examples generated for a specific model will often mislead other unseen models. Consequently the adversary can leverage it to attack against the deployed black-box systems. In this work, we demonstrate that the adversarial perturbation can be decomposed into two components: model-specific and data-dependent one, and it is the latter that mainly contributes to the transferability. Motivated by this understanding, we propose to craft adversarial examples by utilizing the noise reduced gradient (NRG) which approximates the data-dependent component. Experiments on various classification models trained on ImageNet demonstrates that the new approach enhances the transferability dramatically. We also find that low-capacity models have more powerful attack capability than high-capacity counterparts, under the condition that they have comparable test performance. These insights give rise to a principled manner to construct adversarial examples with high success rates and could potentially provide us guidance for designing effective defense approaches against black-box attacks.

This paper postulates that an adversarial perturbation consists of a model-specific and data-specific component, and that amplification of the latter is best suited for adversarial attacks.
This paper has many grammatical errors. The article is almost always missing from nouns. Some of the sentences need changing. For example:
"training model paramater"  --> "training model parameters" (assuming the neural networks have more than 1 parameter)
"same or similar dataset with" --> "same or a similar dataset to"
"human eyes" --> "the human eye"!
"in analogous to" --> "analogous to"
"start-of-the-art" --> "state-of-the-art"
Some roughly chronological comments follow:
In equation (1) although it is obvious that y is the output of f, you should define it. As you are considering the single highest-scoring class, there should probably be an argmax somewhere.
"The best metric should be human eyes, which is unfortunately difficult to quantify". I don't recommend that you quantify things in terms of eyes.
In section 3.1 I am not convinced there is yet sufficient justification to claim that grad(f||)^A is aligned with the inter-class deviation. It would be helpful to put equation (8) here. The "human" line on figure 1a doesn't make much sense. By u & v in the figure 1 caption you presumably the x and y axes on the plot. These should be labelled.
In section 4 you write "it is meaningless to construct adversarial perturbations for the images that target models cannot classify correctly". I'm not sure this is true. Imagenet has a *lot* of dog breeds. For an adversarial attack, it may be advantageous to change the classification from "wrong breed of dog" to "not a dog at all".
Something that concerns me is that, although your methods produce good results, it looks like the hyperparameters are chosen so as to overfit to the data (please do correct me if this is not the case). A better procedure would be to split the imagenet validation set in two and optimise the hyperparameters on one split, and test on the second. You also "try lots of \alphas", which again seems like overfitting.
Target attack experiments are missing from 5.1, in 5.2 you write that it is a harder problem so it is omitted. I would argue it is still worth presenting these results even if they are less flattering.
Section 6.2 feels out of place and disjointed from the narrative of the paper.
A lot of choices in Section 6 feel arbitrary. In 6.3, why is resnet34 the chosen source model? In 6.4 why do you select those two target models?
I think this paper contains an interesting idea, but suffers from poor writing and unprincipled experimentation. I therefore recommend it be rejected.
Pros:
- Promising results
- Good summary of adversarial methods
Cons:
-  Poorly written
-  Appears to overfit to the test data


We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions. We analyze the convergence of GAN training from this new point of view to understand why mode collapse happens. We hypothesize the existence of undesirable local equilibria in this non-convex game to be responsible for mode collapse. We observe that these local equilibria often exhibit sharp gradients of the discriminator function around some real data points. We demonstrate that these degenerate local equilibria can be avoided with a gradient penalty scheme called DRAGAN. We show that DRAGAN enables faster training, achieves improved stability with fewer mode collapses, and leads to generator networks with better modeling performance across a variety of architectures and objective functions.

This paper contains a collection of ideas about Generative Adversarial Networks (GAN) but it is very hard for me to get the main point of this paper. I am not saying ideas are not interesting, but I think the author needs to choose the main point of the paper, and should focus on delivering in-depth studies on the main point.
1. On the game theoretic interpretations
The paper, Generative Adversarial Nets, NIPS 2014, already presented the game theoretic interpretations to GANs, so it's hard for me to think what's new in the section. Best response dynamics is not used in the conventional GAN training, because it's very hard to find the global optimal of inner minimization and outer maximization.
The convergence of online primal-dual gradient descent method in the minimax game is already well-known, but this analysis cannot be applied to the usual GAN setting because the objective is not convex-concave. I found this analysis would be very interesting if the authors can find the toy example when GAN becomes convex-concave by using different model parameterizations and/or different f-divergence, and conduct various studies on the convergence and stability on this problem.
I also found that the hypothesis on the model collapsing has very limited connection to the convex-concave case. It is OK to form the hypothesis and present an interesting research direction, but in order to make this as a main point of the paper, the author should provide more rigorous arguments or experimental studies instead of jumping to the hypothesis in two sentences. For example, if the authors can provide the toy example where GAN becomes convex-concave vs. non-convex-concave case, and how the loss function shape or gradient dynamics are changing, that will provide very valuable insights on the problem.
2. DRAGAN
As open commenters pointed out, I found it's difficult to find why we want to make the norm of the gradient to 1.
Why not 2? why not 1/2? Why 1 is very special?
In the WGAN paper, the gradient is clipped to a number less than 1, because it is a sufficient condition to being 1-Lipshitz, but this paper provides no justification on this number.
It's OK not to have the theoretical answers to the questions but in that case the authors should provide ablation experiments. For example, sweeping gradient norm target from 10^-3, 10^-2, 10^-1, 1.0, 10.0, etc and their impact on the performance.
Also scheduling regularization parameter like reducing the size of lambda exponentially would be interesting as well.
Most of those studies won't be necessary if the theory is sound. However, since this paper does not provide a justification on the magic number "1", I think it's better to include some form of ablation studies.
Note that the item 1 and item 2 are not strongly related to each other, and can be two separate papers. I recommend to choose one direction and provide in-depth study on one topic. Currently, this paper tries to present interesting ideas without very deep investigations, and I cannot recommend this paper to be published.


Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes. RAML incorporates task-specific reward by performing maximum-likelihood updates on candidate outputs sampled according to an exponentiated payoff distribution, which gives higher probabilities to candidates that are close to the reference output. While RAML is notable for its simplicity, efficiency, and its impressive empirical successes, the theoretical properties of RAML, especially the behavior of the exponentiated payoff distribution, has not been examined thoroughly. In this work, we introduce softmax Q-distribution estimation, a novel theoretical interpretation of RAML, which reveals the relation between RAML and Bayesian decision theory. The softmax Q-distribution can be regarded as a smooth approximation of the Bayes decision boundary, and the Bayes decision rule is achieved by decoding with this Qdistribution. We further show that RAML is equivalent to approximately estimating the softmax Q-distribution, with the temperature τ controlling approximation error. We perform two experiments, one on synthetic data of multi-class classification and one on real data of image captioning, to demonstrate the relationship between RAML and the proposed softmax Q-distribution estimation method, verifying our theoretical analysis. Additional experiments on three structured prediction tasks with rewards defined on sequential (named entity recognition), tree-based (dependency parsing) and irregular (machine translation) structures show notable improvements over maximum likelihood baselines.

The authors claim three contributions in this paper. (1) They introduce the framework of softmax Q-distribution estimation, through which they are able to interpret the role the payoff distribution plays in RAML. Specifically, the softmax Q-distribution serves as a smooth approximation to the Bayes decision boundary. The RAML approximately estimates the softmax Q-distribution, and thus approximates the Bayes decision rule. (2) Algorithmically, they further propose softmax Q-distribution maximum likelihood (SQDML) which improves RAML by achieving the exact Bayes decision boundary asymptotically. (3) Through one experiment using synthetic data on multi-class classiﬁcation and one using real data on image captioning, they show that SQDML is consistently as good or better than RAML on the task-speciﬁc metrics that is desired to optimize.
I found the first contribution is sound, and it reasonably explains why RAML achieves better performance when measured by a specific metric. Given a reward function, one can define the Bayes decision rule. The softmax Q-distribution (Eqn. 12) is defined to be the softmax approximation of the deterministic Bayes rule. The authors show that the RAML can be explained by moving the expectation out of the nonlinear function and replacing it with empirical expectation (Eqn. 17). Of course, the moving-out is biased but the replacing is unbiased.
The second contribution is partially valid, although I doubt how much improvement one can get from SQDML. The authors define the empirical Q-distribution by replacing the expectation in Eqn. 12 with empirical expectation (Eqn. 15). In fact, this step can result in biased estimation because the replacement is inside the nonlinear function. When x is repeated sufficiently in the data, this bias is small and improvement can be observed, like in the synthetic data example. However, when x is not repeated frequently, both RAML and SQDML are biased. Experiment in section 4.1.2 do not validate significant improvement, either.
The numerical results are relatively weak. The synthetic experiment verifies the reward-maximizing property of RAML and SQDML. However, from Figure 2, we can see that the result is quite sensitive to the temperature \tau. Is there any guidelines to choose \tau? For experiments in Section 4.2, all of them are to show the effectiveness of RAML, which are not very relevant to this paper. These experiment results show very small improvement compared to the ML baselines (see Table 2,3 and 5).  These results are also lower than the state of the art performance.
A few questions:
(1). The author may want to check whether (8) can be called a Bayes decision rule. This is a direct result from definition of conditional probability. No Bayesian elements, like prior or likelihood appears here.
(2). In the implementation of SQDML, one can sample from (15) without exactly computing the summation in the denominator. Compared with the n-gram replacement used in the paper, which one is better?
(3). The authors may want to write Eqn. 17 in the same conditional form of Eqn. 12 and Eqn. 14. This will make the comparison much more clear.
(4). What is Theorem 2 trying to convey? Although \tau goes to 0, there is still a gap between Q and Q'. This seems to suggest that for small \tau, Q' is not a good approximation of Q. Are the assumptions in Theorem 2 reasonable? There are several typos in the proof of Theorem 2.
(5). In section 4.2.2, the authors write "the rewards we directly optimized in training (token-level accuracy for NER and UAS for dependency parsing) are more stable w.r.t. τ than the evaluation metrics (F1 in NER), illustrating that in practice, choosing a training reward that correlates well with the evaluation metric is important". Could you explain it in more details?


Published as a conference paper at ICLR 2018 DEEP LEARNING AND QUANTUM ENTANGLEMENT: FUNDAMENTAL CONNECTIONS WITH IMPLICATIONS TO NETWORK DESIGN
Formal understanding of the inductive bias behind deep convolutional networks, i.e. the relation between the network's architectural features and the functions it is able to model, is limited. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning, and use it for obtaining novel theoretical observations regarding the inductive bias of convolutional networks. Specifically, we show a structural equivalence between the function realized by a convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which facilitates the use of quantum entanglement measures as quantifiers of a deep network's expressive ability to model correlations. Furthermore, the construction of a deep ConvAC in terms of a quantum Tensor Network is enabled. This allows us to perform a graph-theoretic analysis of a convolutional network, tying its expressiveness to a min-cut in its underlying graph. We demonstrate a practical outcome in the form of a direct control over the inductive bias via the number of channels (width) of each layer. We empirically validate our findings on standard convolutional networks which involve ReLU activations and max pooling. The description of a deep convolutional network in well-defined graph-theoretic tools and the structural connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.

The paper proposes a structural equivalence between the function realised by a convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which facilitates the use of quantum entanglement measures as quantifiers of a deep network’s expressive ability to model correlations.
The work is definitely worthwhile digging deeper, bridging some gap and discussions between physics and deep learning. The ultimate goal for this work, if I understands correctly, is a provide a theoretical explanation to the design of deep neural architectures. The paper is well-written (above most submissions, top 10%) with clear clarity. However, removing all the fancy stuff and looking into the picture further, I have several major concerns.
+ Potential good research direction to connect physical sciences (via TN) to deep learning theories (via ConvAC).
- [Novelty is limited and proof is vague] The paper uses physical concepts to establish a "LAYER WIDTHS EFFECT ON THE EXPRESSIVENESS OF A DEEP NETWORK", the core theory (proposed method) part is Section 5 alone and the rest (Section 2,3,4) is for introductory purposes. Putting Theorem 1 in simple, deep learning based English, it says for a dataset with features of a size D, there exists a partition of length scale \epsilon < D, which is guaranteed to separate between different parts of a feature. Based on this, they give a rule of thumb to design the width (i.e., channel numbers) of layers in a deep neural network: (a) layer l = logD is more important than those of deeper layers; (b) among these deeper layers, deeper ones need to be wider, which is derived from the min-cut in the ConvAC TN case. How (a) is derived or implied from theorem 1?
It seems to me that the paper goes with a rigorous manner till the proof of theorem 1, with all the concepts and denotations well demonstrated. Suddenly when it comes to connecting the practical design of deep networks, the conclusion becomes qualitative without much explanation via figures or visualisation of the learned features to prove the effectiveness of the proposed scheme.
- [Experiments are super weak] The paper has a good motivation and a beautiful story and yet, the experiments are poor to verify them. The reason as to why authors use ConvAC is that it more resembles the tensor operations introduced in the paper. There is a sentence, "Importantly, through the concept of generalized tensor decompositions, a ConvAC can be transformed to a standard convolutional network with ReLU activation and average/max pooling", to tell the relation between ConvAC and traditional convolutions. The theory is based on the analysis of ConvAC, and all of a sudden the experiments are conducted on the traditional convolution. This is not rigorous and not professional for a "technically-sound" paper. How the generalized concepts of tensor decompositions can be applied from ConvAC to vanilla convolutions?
The experiments seem to extend the channel width of *all* layers in a hand-crafted manner (10, 4r, 4r, xxx). Based on the derived rule of thumb, the most important layer in MNIST should be layer 3 or 4 (log 10). Some simple ablative analysis should be:
(i) baseline: fix layer 3, add more layers thereafter in the network;
(ii) fix layer 3, reduce the channel numbers after layer 3.
The (ii) case should be at least comparable to (i) if theorem 1 is correct.
Moreover, to verify conclusion (b) which I mentioned earlier, does the current setting (10, 4r, 4r, xx) consider "deeper ones need to be wider"? What is the value of r? MNIST is a over-used dataset and quite small. I see the performance in Figure 4 (the only experiment result in the paper) just exceeds 90%. A simple trained NN (not CNN) could reach well 96% or so.
More ablative study (convAC or vanilla conv, other datasets, comparison components in width design, etc.) are seriously needed. Otherwise, it is just not convincing to me.
If the authors target on the network design in a more general manner (not just in layer width, but the design of number of filters, layers, etc.), there are already some neat work in this community and you should definitely compare them: e.g., Neural Architecture Search with Reinforcement Learning, ICLR 2017. I know the paper starts with building the connection from physics to deep learning and it is natural to solve the width design issue alone. This is not a major concern.
-------------
We see lots of fancy conceptions, trying to bind interdisciplinary subjects to help analyse deep learning theories over the last few years, especially in ICLR 2017. This year same thing happens. I am not degrading the motivation/intuition of the work; instead, I think it is pretty novel to explain the design of neural nets, by way of quantum physics. But the experiments and the conclusion derived from the analysis make the paper not solid to me and I am quite skeptical about its actual effectiveness.


Published as a conference paper at ICLR 2018 DEEP ACTIVE LEARNING FOR NAMED ENTITY RECOGNITION
Deep learning has yielded state-of-the-art performance on many natural language processing tasks including named entity recognition (NER). However, this typically requires large amounts of labeled data. In this work, we demonstrate that the amount of labeled training data can be drastically reduced when deep learning is combined with active learning. While active learning is sample-efficient, it can be computationally expensive since it requires iterative retraining. To speed this up, we introduce a lightweight architecture for NER, viz., the CNN-CNN-LSTM model consisting of convolutional character and word encoders and a long short term memory (LSTM) tag decoder. The model achieves nearly state-of-the-art performance on standard datasets for the task while being computationally much more efficient than best performing models. We carry out incremental active learning, during the training process, and are able to nearly match state-of-the-art performance with just 25% of the original training data.

This paper studies the application of different existing active learning strategies for the deep models for NER.
Pros:
* Active learning may be used for improving the performance of deep models for NER in practice
* All the proposed approaches are sound and the experimental results showed that active learning is beneficial for the deep models for NER
Cons:
* The novelty of this paper is marginal. The proposed approaches turn out to be a combination of existing active learning strategies for selecting data to query with the existing deep model for NER.
* No conclusion can be drawn by comparing with the 4 different strategies.
======= After rebuttal  ================
Thank you for the clarification and revision on this paper. It looks better now.
I understand that the purpose of this paper is to give actionable insights to the practice of deep learning. However, since AL itself is a meta learning framework and neural net as the base learner has been shown to be effective for AL, the novelty and contribution of a general discussion of applying AL for deep neural nets is marginal.  What I really expected is a tightly-coupled active learning strategy that is specially designed for the particular deep neural network structure used for NER. Apparently, however, none of the strategies used in this work is designed for this purpose (e.g., the query strategy or model update strategy should at least reflex some properties of deep learning or NER). Thus, it is still below my expectation.
Anyway, since the authors had attempted to improved this paper, and the results may provide some information to practice, I would like to slightly raise my rating to give this attempt a chance.


Published as a conference paper at ICLR 2018 META LEARNING SHARED HIERARCHIES Work done as an intern at OpenAI
We develop a metalearning approach for learning hierarchically structured policies, improving sample efficiency on unseen tasks through the use of shared primitives-policies that are executed for large numbers of timesteps. Specifically, a set of primitives are shared within a distribution of tasks, and are switched between by task-specific policies. We provide a concrete metric for measuring the strength of such hierarchies, leading to an optimization problem for quickly reaching high reward on unseen tasks. We then present an algorithm to solve this problem end-to-end through the use of any off-the-shelf reinforcement learning method, by repeatedly sampling new tasks and resetting task-specific policies. We successfully discover 1 meaningful motor primitives for the directional movement of four-legged robots, solely by interacting with distributions of mazes. We also demonstrate the transferability of primitives to solve long-timescale sparse-reward obstacle courses, and we enable 3D humanoid robots to robustly walk and crawl with the same policy.

This paper proposes a novel hierarchical reinforcement learning method for a fairly particular setting.  The setting is one where the agent must solve some task for many episodes in a sequence, after which the task will change and the process repeats.  The proposed solution method splits the agent into two components, a master policy which is reset to random initial weights for each new task, and several sub-policies (motor primitives) that are selected between by the master policy every N steps and whose weights are not reset on task switches.  The core idea is that the master policy is given a relatively easy learning task of selecting between useful motor primitives and this can be efficiently learned from scratch on each new task, whereas learning the motor primitives occurs slowly over many different tasks.  To push this motivation into the learning process, the master policy is updated always but the sub-policies are only updated after an extended warmup period (called the joint-update or training period).  This experiments include both small domains (moving to 2D goals and four-rooms) and more complex physics simulations (4-legged ants and humanoids).  In both the simple and complex domains, the proposed method (MLSH) is able to robustly achieve good performance.
This approach to obtaining complex structured behavior appears impressive despite the amount of temporal structure that must be provided to the method (the choice of N, the warmup period, and the joint-update period).  Relying on the temporal structure for the hierarchy, and forcing the master policy to be relearned from scratch for each new task may be problematic in general, but this work shows that in some complex settings, a simple temporal decomposition may be sufficient to encourage the development of reusable motor primitives and to also enable quick learning of meta-policies over these motor-primitives.  Moreover, the results show that these temporal hierarchies are helpful in these domains, as the corresponding non-hierarchical methods failed on the more challenging tasks.
The paper could be improved in some places (e.g. unclear aliases of joint-update or training periods, describing how the parameters were chosen, and describing what kinds of sub-policies are learned in these domains when different parameter choices are made).


Learning rules for neural networks necessarily include some form of regularization. Most regularization techniques are conceptualized and implemented in the space of parameters. However, it is also possible to regularize in the space of functions. Here, we propose to measure networks in an L 2 Hilbert space, and test a learning rule that regularizes the distance a network can travel through L 2 -space each update. This approach is inspired by the slow movement of gradient descent through parameter space as well as by the natural gradient, which can be derived from a regularization term upon functional change. The resulting learning rule, which we call Hilbert-constrained gradient descent (HCGD), is thus closely related to the natural gradient but regularizes a different and more calculable metric over the space of functions. Experiments show that the HCGD is efficient and leads to considerably better generalization.

I have read comments and rebuttal - i do not have the luxury of time to read in depth the revision.
It seems that the authors have made an effort to accommodate reviewers' comments. I upgraded the rating.
-----------------------------------------------------------------------------------------------------------------------
Summary: The paper considers the use of natural gradients for learning. The added twist is the substitution of the KL divergence with the Wasserstein distance, as proposed in GAN training. The authors suggest that Wasserstein regularization improves generalization over SGD with a little extra cost.
The paper is structured as follows:
1. KL divergence is used as a similarity measure between two distributions.
2. Regularizing the objective with KL div. seems promising, but expensive.
3. We usually approximate the KL div. with its 2nd order approximation - this introduces the Hessian of the KL divergence, known as Fisher information matrix.
4. However, computing and inverting the Fisher information matrix is computationally expensive.
5. One solution is to approximate the solution F^{-1} J using gradient descent. However, still we need to calculate F. There are options where F could be formed as the outer product of a collection gradients of individual examples ('empirical Fisher').
6. This paper does not move towards Fisher information, but towards Wasserstein distance: after a "good" initialization via SGD is obtained, the inner loop continues updating that point using the Wasserstein regularized objective.
7. No large matrices need to be formed or inverted, however more passes needed per outer step.
Importance:
Somewhat lack of originality and poor experiments lead to low importance.
Clarity:
The paper needs major revision w.r.t. presenting and highlighting the new main points. E.g., one needs to get to page 5 to understand that the paper is just based on the WGAN ideas in Arjovsky et al., but with a different application (not GANS).
Originality/Novelty:
The paper, based on WGAN motivation, proposes Wasserstein distance regularization over KL div. regularization for training of simple models, such as neural networks. Beyond this, the paper does not provide any futher original idea. So, slight to no novelty.
Main comments:
1. Would the approximation of C_0 by its second-order Taylor expansion (that also introduces a Hessian) help? This would require the combination of two Hessian matrices.
2. Experiments are really demotivating: it is not clear whether using plain SGD or the proposed method leads to better results.
Overall:
Rejection.


We introduce a novel approach to tree-to-tree learning, the neural tree transducer (NTT), a top-down depth first context-sensitive tree decoder, which is paired with recursive neural encoders. Our method works purely on tree-to-tree manipulations rather than sequence-to-tree or tree-to-sequence and is able to encode and decode multiple depth trees. We compare our method to sequence-to-sequence models applied to serializations of the trees and show that our method outperforms previous methods for tree-to-tree transduction.

The paper introduces a neural tree decoder architecture for binary trees that conditions the next node prediction on
representations of its ascendants (encoded with an LSTM recurrent net) and left sibling subtree (encoded with a binary LSTM recursive net) for right sibling nodes.
To perform tree to tree transduction the input tree is encoded as a vector with a Tree LSTM; correspondences between input and output subtrees are not modelled directly (using e.g. attention) as is done in traditional tree transducers.
While the term context-sensitive should be used with caution, I do accept the claim here, although the notation used does not make the exposition clear.
Experimental results show that the architecture performs better at synthetic tree transduction tasks (relabeling, reordering, deletion) than sequence-to-sequence baselines.
While neural approches to tree-to-tree transduction is an understudied problem, the contributions of this paper are very narrow and it is not shown that the proposed approach will generalize to more expressive models or real-world applications of tree-to-tree transduction.
Existing neural tree decoders, such as Dong and Lapata or Alvarex-Melis and Jaakkola, could be combined with tree LSTM encoders without any technical innovations and could possibly do as well as the proposed model for the transduction tasks tested - no experiments are performed with existing tree-based decoder architectures.
Specific comments per section:
1. Unclear what is meant be "equivalent" in first paragraph.
2. The model does not assign an explicit probability to the tree structure - rather it seems to rely on the distinction between terminal and non-terimal symbols and the restriction to binary trees to know when closing brackets are implied - this is not made clear, and a general model should not have this restriction, as there are many cases where we want to generate non-binary trees.
The production rule notation used is incorrect and confusing, mixing sets with non-terminals and terminal symbols:
A better notation for the rules in 2.1.1 would be something like S -> P | v | \epsilon; P -> Q R | Q u | u Q | u w, where P, Q, R \in O and u, w \in v.
2.1.2. Splitting production rules as ->_left, ->_right is not standard notation. Rather introduce intermediate non-terminals in the grammar:
O -> O_L O_R; O_L -> a | Q, O_R -> b | Q.
2.1.3 The context-sensitively here arise when conditioning on the entire left sibling subtree (not just the top non-terimal).
The rules should have a format such as O -> O_L O_R; O_L -> a | Q; \alpha O_R -> \alpha a | \alpha Q, where \alpha is an entire subtree rooted at O_L.
2.1.4 Should be g(x|.) = exp( ), the softmax function includes the normalization which is done in the equation below.
3. Note that is is possible to restrict the decoder to produce tree structures while keeping a sequential neural architecture. For some tasks sequential decoders do actually produce mostly well-formed trees, given enough training data.
RNNG encodes completed subtrees recursively, and the stack LSTM encodes the entire partially-produced tree, so it does produce and condition on trees not just sequences. The model in this paper is not more expressive than RNNG, it just encodes somewhat different structural biases, which might or might not be suited for real tasks.
4. In the examples given, the same set of symbols are used as both terminals and non-terminals. How is the tree structure then predicted by the decoder?
Details about the training setup are missing: How is the training data generated, what is the size of the trees during training (compared to testing)?
4.2 The steep drop in performance between depth 5 and 6 indicates that model is very sensitive to its memorization capacity and might not be generalizing over the given training data.
For real tree-to-tree applications involving these operations, there is good reason to believe that some kind of attention mechanism will be needed over the input tree during decoding.
Reference should generally be to published proceedings rather than to arxiv where available - e.g. Aharoni and Goldberg, Dong and Lapata, Erguchi et al, Rush et al. For Graehl and Knight there is a published journal paper in Computational Linguistics.


Variational RNNs are proposed to output "creative" sequences. Ideally, a collection of sequences produced by a variational RNN should be of both high quality and high variety. However, existing decoders for variational RNNs suffer from a trade-off between quality and variety. In this paper, we seek to learn a variational RNN that decodes high-quality and high-variety sequences. We propose the Self-Improving Collaborative GAN (SIC-GAN), where there are two generators (variational RNNs) collaborating with each other to output a sequence and aiming to trick the discriminator into believing the sequence is of good quality. By deliberately weakening one generator, we can make another stronger in balancing quality and variety. We conduct experiments using the QuickDraw dataset and the results demonstrate the effectiveness of SIC-GAN empirically.

This paper baffles me. It appears to be a stochastic RNN with skip connections (so it's conditioned on the last two states rather than last one) trained by an adversarial objective (which is no small feat to make work for sequential tasks) with results shown on the firetruck category of the QuickDraw dataset. Yet the authors claim significantly more importance for the work than I think it merits.
First, there is nothing variational about their variational RNN. They seem to use the term to be equivalent to "stochastic", "probabilistic" or "noisy" rather than having anything to do with optimizing a variational bound. To strike the right balance between pretension and accuracy, I would suggest substituting the word "stochastic"  everywhere "variational" is used.
Second, there is nothing self-improving or collaborative about their self-improving collaborative GAN. Once the architecture is chosen to share the weights between the weak and strong generator, the only difference between the two is that the weak generator has greater noise at the output. In this sense the architecture should really be seen as a single model with different noise levels at alternating steps. In this sense, I am not entirely clear on what the difference is between the SIC-GAN and their noisy GAN baseline - presumably the only difference is that the noisy GAN is conditioned on a single timestep instead of two at a time? The claim that these models are somehow "self-improving" baffles me as well - all machine learning models are self-improving, that is the point of learning. The authors make a comparison to AlphaGo Zero's use of self-play, but here the weak and strong generators are on the same side of the game, and because there are no game rules provided beyond "reproduce the training set", there is no possibility of discovery beyond what is human-provided, contrary to the authors' claim.
Third, the total absence of mathematical notation made it hard in places to follow exactly what the models were doing. While there are plenty of papers explaining the GAN framework to a novice, at least some clear description of the baseline architectures would be appreciated (for instance, a clearer explanation of how the SIC-GAN differs from the noisy GAN). Also the description of the soft $\ell_1$ loss (which the authors call the "1-loss" for some reason) would benefit from a clearer mathematical exposition.
Fourth, the experiments seem too focused on the firetruck category of the QuickDraw dataset. As it was the only example shown, it's difficult to evaluate their claim that this is a general method for improving variety without sacrificing quality. Their chosen metrics for variety and detail are somewhat subjective, as they depend on the fact that some categories in the QuickDraw dataset resemble firetrucks in the fine detail while others resemble firetrucks in outline. This is not a generalizable metric. Human evaluation of the relative quality and variety would likely suffice.
Lastly, the entire section on the strong-weak collaborative GAN seems to add nothing. They describe an entire training regiment for the model, yet never provide any actual experimental results using that model, so the entire section seems only to motivate the SIC-GAN which, again, seems like a fairly ordinary architectural extension to GANs with RNN generators.
The results presented on QuickDraw do seem nice, and to the best of my knowledge it is the first (or at least best) applications of GANs to QuickDraw - if they refocused the paper on GAN architectures for sketching and provided more generalizable metrics of quality and variety it could be made into a good paper.


Synthesizing SQL queries from natural language is a long-standing open problem and has been attracting considerable interest recently. Toward solving the problem, the de facto approach is to employ a sequence-to-sequence-style model. Such an approach will necessarily require the SQL queries to be serialized. Since the same SQL query may have multiple equivalent serializations, training a sequenceto-sequence-style model is sensitive to the choice from one of them. This phenomenon is documented as the "order-matters" problem. Existing state-of-the-art approaches rely on reinforcement learning to reward the decoder when it generates any of the equivalent serializations. However, we observe that the improvement from reinforcement learning is limited. In this paper, we propose a novel approach, i.e., SQLNet, to fundamentally solve this problem by avoiding the sequence-to-sequence structure when the order does not matter. In particular, we employ a sketch-based approach where the sketch contains a dependency graph so that one prediction can be done by taking into consideration only the previous predictions that it depends on. In addition, we propose a sequence-to-set model as well as the column attention mechanism to synthesize the query based on the sketch. By combining all these novel techniques, we show that SQLNet can outperform the prior art by 9% to 13% on the WikiSQL task.

This paper proposes a neural network-based approach to converting natural language questions to SQL queries. The idea is to use a small grammar to facilitate the process, together making some independence assumptions. It is evaluated on a recently introduced dataset for natural language to SQL.
Pros:
- good problem, NL2SQL is an important task given how dominant SQL is
- incorporating a grammar ("sketch") is a sensible improvement.
Cons:
- The dataset used makes very strong simplification assumptions. Not  problem per se, but it is not the most challenging SQL dataset. The ATIS corpus is NL2SQL and much more challenging and realistic:
Deborah A. Dahl, Madeleine Bates, Michael Brown, William Fisher, Kate Hunicke-Smith, David Pallett, Christine Pao, Alexander Rudnicky, and Elizabeth Shriberg. 1994. Expanding the scope of the ATIS task: the ATIS-3 corpus. In Proceedings of the workshop on Human Language Technology (HLT '94). Association for Computational Linguistics, Stroudsburg, PA, USA, 43-48. DOI: https://doi.org/10.3115/1075812.1075823
- In particular, the assumption that every token in the SQL statement is either an SQL keyword or appears in the natural language statement is rather atypical and unrealistic.
- The use of a grammar in the context of semantic parsing is not novel; see this tutorial for many pointers:
http://yoavartzi.com/tutorial/
- As far as I can tell, the set prediction is essentially predicted each element independently, without taking into account any dependencies. Nothing wrong, but also nothing novel, that is what most semantic parsing/semantic role labeling baseline approaches do. The lack of ordering among the edges, doesn't mean they are independent.
- Given the rather constrained type of questions and SQL statements, it would make sense to compare it against approaches for question answering over knowledge-bases:
https://github.com/scottyih/Slides/blob/master/QA%20Tutorial.pdf
While SQL can express much more complex queries, the ones supported by the grammar here are not very different.
- Pasupat and Liang (2015) also split the data to make sure different tables appear only in training, dev, test and they developed their dataset using crowd sourcing.
- The comparison against Dong and Lapata (2016) is not fair because their model is agnostic and thus applicable to 4 datasets while the one presented here is tailored to the dataset due the grammar/sketch used. Also, suggesting that previous methods might not generalize well sounds odd given that the method proposed seems to use much larger datasets.
- Not sure I agree that mixing the same tables across training/dev/test is more realistic. If anything, it assumes more training data and manual annotation every time a new table is added.


We propose Bayesian hypernetworks: a framework for approximate Bayesian inference in neural networks. A Bayesian hypernetwork h is a neural network which learns to transform a simple noise distribution, p( ) = N (0, I), to a distribution q(θ) := q(h( )) over the parameters θ of another neural network (the "primary network"). We train q with variational inference, using an invertible h to enable efficient estimation of the variational lower bound on the posterior p(θ|D) via sampling. In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap iid sampling of q(θ). In practice, Bayesian hypernets provide a better defense against adversarial examples than dropout, and also exhibit competitive performance on a suite of tasks which evaluate model uncertainty, including regularization, active learning, and anomaly detection.

This paper presents Bayesian Hypernetworks; variational Bayesian neural networks where the variational posterior over the weights is governed by a hyper network that implements a normalizing flow (NF) such as RealNVP and IAF. As directly outputting the weight matrix with a hyper network is computationally expensive the authors instead propose to utilize weight normalisation on the weights and then use the hyper network to output scalar scaling variables for each hidden unit, similarly to what was done at [1]. The main difference with this prior work is that [1] consider these NF scaling variables as auxiliary random variables to a mean field Gaussian distribution over the weights whereas this paper attempts to posit a distribution directly on the weights via the NF. This avoids the nested variational approximation and auxiliary models of [1], which can potentially yield a tighter bound. The proposed method is evaluated on extensive experiments.
This paper seems like a plausible idea with extensive experiments but the similarity with [1] make it an incremental contribution and, furthermore, it seems that it has a technical issue with what is explained at Section 3.3. More specifically, if you generate the parameters \theta according to Eq. 7 and posit a prior over \theta then you will have a problematic variational bound as there will be a KL divergence, KL(q(\theta) || p(\theta)), with distributions of different support (since q(\theta) is defined only along the directions spanned by u), which is infinite. For the KL to be valid you will need to posit a prior distribution over `g`, p(g), and then consider KL(q(g) || p(g)), with q(g) being given by the NF. From the experiment paragraph at page 5 though I deduct that you instead employ “an isotropic standard normal prior over the weights”, i.e. \theta, thus I believe that you indeed have a problematic bound. How do you actually compute logq(\theta) when you employ the parametrisation discussed at 3.3? Did you use that parametrisation in every experiment?
Other than that, I believe that it would be interesting to experiment with a `full` hyper network, i.e. generating directly the entire parameter vector \theta, e.g. at the toy regression experiment where the dimensionality is small. This would then better illustrate the tradeoffs you make when you reduce the flexibility of the hyper-network to just outputting the row scaling variables and the effect this has at the posterior approximation.

Typos:
(1) Page 3, 3.1.1 log(\theta) -> logp(\theta).
(2) Eq. 6, it needs to be |det \frac{\partial h(\epsilon)}{\partial \epsilon}|^{-1} or |det \frac{\partial h^{-1}(\theta)}{\partial \theta}| for a valid change of variables formula.
[1] Louizos & Welling, Multiplicative Normalizing Flows for Variational Bayesian Neural Networks.


Published as a conference paper at ICLR 2018 RECASTING GRADIENT-BASED META-LEARNING AS HIERARCHICAL BAYES
Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. (2017) as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm's operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.

Summary
The paper presents an interesting view on the recently proposed MAML formulation of meta-learning (Finn et al). The main contribution is a) insight into the connection between the MAML procedure and MAP estimation in an equivalent linear hierarchical Bayes model with explicit priors, b) insight into the connection between MAML and MAP estimation in non-linear HB models with implicit priors, c) based on these insights, the paper proposes a variant of MALM using a Laplace approximation (with additional approximations for the covariance matrix. The paper finally provides an evaluation on the mini ImageNet problem without significantly improving on the MAML results on the same task.
Pro:
-            The topic is timely and of relevance to the ICLR community continuing a current trend in building meta-learning system for few-shot learning.
-            Provides valuable insight into the MAML objective and its relation to probabilistic models
Con:
-            The paper is generally well-written but I find (as a non-meta-learner expert) that certain fundamental aspects could have been explained better or in more detail (see below for details).
-            The toy example is quite difficult to interpret the first time around and does not provide any empirical insight into the converge of the proposed method (compared to e.g. MAML)
-            I do not think the empirical results provide enough evidence that it is a useful/robust method. Especially it does not provide insight into which types of problems (small/large, linear/ non-linear) the method is applicable to.
Detailed comments/questions:
-            The use of Laplace approximation is (in the paper) motivated from a probabilistic/Bayes and uncertainty point-of-view. It would, however, seem that the truncated iterations do not result in the approximation being very accurate during optimization as the truncation does not result in the approximation being created at a mode. Could the authors perhaps comment on:
a) whether it is even meaningful to talk about the approximations as probabilistic distribution during the optimization (given the psd approximation to the Hessian), or does it only make sense after convergence?
b) the consequence of the approximation errors on the general convergence of the proposed method (consistency and rate)
-            Sec 4.1, p5: Last equation: Perhaps useful to explain the term $log(\phi_j^* | \theta)$ and why it is not in subroutine 4 . Should $\phi^*$  be $\hat \phi$ ?
-            Sec 4.2: “A straightforward…”: I think it would improve readability to refer back to the to the previous equation (i.e. H) such that it is clear what is meant by “straightforward”.
-            Sec 4.2: Several ideas are being discussed in Sec 4.2 and it is not entirely clear to me what has actually been adopted here; perhaps consider formalizing the actual computations in Subroutine 4 – and provide a clearer argument (preferably proof) that this leads to consistent and robust estimator of \theta.
-            It is not clear from the text or experiment how the learning parameters are set.
-            Sec 5.1: It took some effort to understand exactly what was going on in the example and particular figure 5.1; e.g., in the model definition in the body text there is no mention of the NN mentioned/used in figure 5, the blue points are not defined in the caption, the terminology e.g.  “pre-update density” is new at this point. I think it would benefit the readability to provide the reader with a bit more guidance.
-            Sec 5.1: While the qualitative example is useful (with a bit more text), I believe it would have been more convincing with a quantitative example to demonstrate e.g. the convergence of the proposal compared to std MAML and possibly compare to a std Bayesian inference method from the HB formulation of the problem (in the linear case)
-            Sec 5.2: The abstract clams increased performance over MAML but the empirical results do not seem to be significantly better than MAML ? I find it quite difficult to support the specific claim in the abstract from the results without adding a comment about the significance.
-            Sec 5.2: The authors have left out “Mishral et al” from the comparison due to the model being significantly larger than others. Could the authors provide insight into why they did not use the ResNet structure from the  tcml paper in their L-MLMA scheme ?
-            Sec 6+7: The paper clearly states that it is not the aim to (generally) formulate the MAML as a HB. Given the advancement in gradient based inference for HB the last couple of years (e.g. variational, nested laplace , expectation propagation etc) for explicit models, could the authors perhaps indicate why they believe their approach of looking directly to the MAML objective is more scalable/useful than trying to formulate the same or similar objective in an explicit HB model and using established inference methods from that area ?
Minor:
-            Sec 4.1 “…each integral in the sum in (2)…” eq 2 is a product


Despite much success in many large-scale language tasks, sequence-to-sequence (seq2seq) models have not been an ideal choice for conversational modeling as they tend to generate generic and repetitive responses. In this paper, we propose a Latent Topic Conversational Model (LTCM) that augments the seq2seq model with a neural topic component to better model human-human conversations. The neural topic component encodes information from the source sentence to build a global "topic" distribution over words, which is then consulted by the seq2seq model to improve generation at each time step. The experimental results show that the proposed LTCM can generate more diverse and interesting responses by sampling from its learnt latent representations. In a subjective human evaluation, the judges also confirm that LTCM is the preferred option comparing to competitive baseline models.

The paper proposes a conversational model with topical information, by combining seq2seq model with neural topic models. The experiments and human evaluation show the model outperform some the baseline model seq2seq and the other latent variable model variant of seq2seq.
The paper is interesting, but it also has certain limitations:
1) To my understanding, it is a straightforward combination of seq2seq and one of the neural topic models without any justification.
2) The evaluation doesn't show how the topic information could influence word generation. No of the metrics in table 2 could be used to justify the effect of topical information.
3) There is no analysis about the model behavior, therefore there is no way we could get a sense about how the model actually works. One possible analysis is to investigate the values $l_t$ and the corresponding words, which to some extent will tell us how the topical information be used in generation. In addition, it could be even better if there are some analysis about topics extracted by this model.
This paper also doesn't pay much attention to the existing work on topic-driven conversational modeling. For example "Topic Aware Neural Response Generation" from Xing et al., 2017.
Some additional issues:
1) In the second line under equation 4, y_{t-1} -> y_{t}
2) In the first paragraph of section 3, two "MLP"'s are confusing
3) In the first paragraph of page 6, words with "highest inverse document frequency" are used as stop words?


This paper introduces a probabilistic framework for k-shot image classification. The goal is to generalise from an initial large-scale classification task to a separate task comprising new classes and small numbers of examples. The new approach not only leverages the feature-based representation learned by a neural network from the initial task (representational transfer), but also information about the classes (concept transfer). The concept information is encapsulated in a probabilistic model for the final layer weights of the neural network which acts as a prior for probabilistic k-shot learning. We show that even a simple probabilistic model achieves state-of-the-art on a standard k-shot learning dataset by a large margin. Moreover, it is able to accurately model uncertainty, leading to well calibrated classifiers, and is easily extensible and flexible, unlike many recent approaches to k-shot learning.

Authors present a k-shot learning method that is based on generating representations with a pre-trained network and learning a regularized logistic regression using the available data.  The regularised regression is formulated as a MAP estimation problem with the prior estimated from the weights of the original network connected final hidden layer to the logits — before the soft-max layer.
The motivation of the article regarding “concepts” is interesting.  It seems especially justified when the training set that is used to train the original network has similar objects as the smaller set that is used for k-shot learning.  Maps shown in Figures 6 and 7 provide good motivation for this approach.
Despite the strong motivation, the article raises some concerns regarding the method.
1. The assumption about independence of w vectors across classes is a very strong one and as far as I can see, it does not have a sound justification.  The original networks are trained to distinguish between classes.  The weight vectors are estimated with this goal. Therefore, it is very likely that vectors of different classes are highly correlated. Going beyond this assumption also seems difficult.  The proposed model estimates $\theta^{MAP}$ using only one W matrix, the one that is estimated by training the original network in the most usual way.  In this case, the prior over $\theta$ would have a large influence on the MAP estimate and setting it properly becomes important.  As far as I can see, there is no good recipe presented in the article for setting this prior.
2. How is the prior model defined?  It is the most important component of the method while precise details are not provided.  How are the hyperparameters set?  Furthermore, this detail needs to be in the main text.
3. With the isotropic assumption on the covariance matrix, the main difference between logistic regression, which is regularized by L2 norm and coefficient set proportional to the empirical variance, and the proposed method seems to be the mean vector $\mu^{MAP}$.  From the details provided in the appendix — which should be in the main text in my opinion — I believe this vector is a combination of the prior and mean of w_c across classes.  If the prior is set to 0, how different is this vector from 0?  Authors should focus on this in my opinion to explain why methods work differently in 1-shot learning.  In the other problems, the results suggest they are pretty much the same.
4. Authors’ motivation about concepts is interesting however, if the model bases its prediction on mean of w_c vectors over classes, then I am not sure if authors really achieve what they motivate for.
5. Results are not very convincing.  If the method was substantially different than baseline, I believe this would have been no problem.  Given the proximity of the proposed method to the baseline with regularised logistic regression, lack of empirical advantage is an issue.  If the proposed model works better in the 1-shot scenario, then authors should delve into it to explain the advantage.
Minor comments:
Evaluation in an online setting section is unclear.  It needs to be rewritten in my opinion.