EMERGENCE OF GRID-LIKE REPRESENTATIONS BY TRAINING RECURRENT NEURAL NETWORKS TO PERFORM SPATIAL LOCALIZATION
Decades of research on the neural code underlying spatial navigation have revealed a diverse set of neural response properties. The Entorhinal Cortex (EC) of the mammalian brain contains a rich set of spatial correlates, including grid cells which encode space using tessellating patterns. However, the mechanisms and functional significance of these spatial representations remain largely mysterious. As a new way to understand these neural representations, we trained recurrent neural networks (RNNs) to perform navigation tasks in 2D arenas based on velocity inputs. Surprisingly, we find that grid-like spatial response patterns emerge in trained networks, along with units that exhibit other spatial correlates, including border cells and band-like cells. All these different functional types of neurons have been observed experimentally. The order of the emergence of grid-like and border cells is also consistent with observations from developmental studies. Together, our results suggest that grid cells, border cells and others as observed in EC may be a natural solution for representing space efficiently given the predominant recurrent connections in the neural circuits.

This paper aims at better understanding the functional role of grid cells found in the entorhinal cortex by training an RNN to perform a navigation task.
On the positive side:
This is the first paper to my knowledge that has shown that grid cells arise as a product of a navigation task demand. I enjoyed reading the paper which is in general clearly written. I have a few, mostly cosmetic, complaints but this can easily be addressed in a revision.
On the negative side:
The manuscript is not written in a way that is suitable for the target ICLR audience which will include, for the most part, readers that are not expert on the entorhinal cortex and/or spatial navigation.
First, the contributions need to be more clearly spelled out. In particular, the authors tend to take shortcuts for some of their statements. For instance, in the introduction, it is stated that previous attractor network type of models (which are also recurrent networks) “[...] require hand-crafted and fined tuned connectivity patterns, and the evidence of such specific 2D connectivity patterns has been largely absent.” This statement is problematic for two reasons:
(i) It is rather standard in the field of computational neuroscience to start from reasonable assumptions regarding patterns of neural connectivity then proceed to show that the resulting network behaves in a sensible way and reproduces neuroscience data. This is not to say that demonstrating that these patterns can arise as a byproduct is not important, on the contrary. These are just two complementary lines of work. In the same vein, it would be silly to dismiss the present work simply because it lacks spikes.
(ii) the authors do not seem to address one of the main criticisms they make about previous work and in particular "[a lack of evidence] of such specific 2D connectivity patterns". My understanding is that one of the main assumptions made in previous work is that of a center-surround pattern of lateral connectivity. I would argue that there is a lot of evidence for local inhibitory connection in the cortex. Somewhat related to this point, it would be insightful to show the pattern of local connections learned in the RNN to see how it differs from the aforementioned pattern of connectivity.
Second, the navigation task used needs to be better justified. Why training a network to predict 2D spatial location from velocity inputs? Why is this a reasonable starting point to study the emergence of grid cells? It might be obvious to the authors but it will not be to the ICLR audience. Dead-reckoning (i.e., spatial localization from velocity inputs) is of critical ecological relevance for many animals. This needs to be spelled out and a reference needs to be added.  As a side note, I would have expected the authors to use actual behavioral data but instead, the network is trained using artificial trajectories based on "modified Brownian motion”. This seems like an important assumption of the manuscript but the issue is brushed off and not discussed. Why is this a reasonable assumption to make? Is there any reference demonstrating that rodent locomotory behavior in a 2D arena is random?
Figure 4 seems kind of strange. I do not understand how the “representative units” are selected and where the “late” selectivity on the far right side in panel a arises if not from “early” units that would have to travel “far” from the left side… Apologies if I am missing something obvious.
I found the study of the effect of regularization to be potentially the most informative for neuroscience but it is only superficially treated. It would have been nice to see a more systematic treatment of the specifics of the regularization needed to get grid cells.


In the pursuit of increasingly intelligent learning systems, abstraction plays a vital role in enabling sophisticated decisions to be made in complex environments. The options framework provides formalism for such abstraction over sequences of decisions. However most models require that options be given a priori, presumably specified by hand, which is neither efficient, nor scalable. Indeed, it is preferable to learn options directly from interaction with the environment. Despite several efforts, this remains a difficult problem: many approaches require access to a model of the environmental dynamics, and inferred options are often not interpretable, which limits our ability to explain the system behavior for verification or debugging purposes. In this work we develop a novel policy gradient method for the automatic learning of policies with options. This algorithm uses inference methods to simultaneously improve all of the options available to an agent, and thus can be employed in an off-policy manner, without observing option labels. Experimental results show that the options learned are interpretable. Further, we find that the method presented here is more sample efficient than existing methods, leading to faster and more stable learning of policies with options.

The paper presents a new policy gradient technique for learning options. The option index is treated as latent variable and, in order to compute the policy gradient, the option distribution for the current sample is computed by using a forward pass. Hence, a single sample can be used to update all options and not just the option that has been used for this sample.

The idea of the paper is good but the novelty is limited. As noted by the authors, the idea of using inference for option discovery has already been presented in Daniel2016. Note that the option discovery process is Daniel2016 is not limited to linear sub-policies, only the policy update strategy is. So the main contribution is to use a new policy update strategy, i.e., policy gradients, for inference based option discovery. Thats fine but should be stated more clearly in the paper. The paper is also written very well and the topic is relevant for the ICLR conference.
However, the paper has two main problems:
- The results are not convincing. In most domains, the performance is similar to the A3C algorithm (which does not use inference based option discovery), so the impact of this paper seems limited.
- One of the main assumptions of the algorithm is wrong. The assumption is that rewards from the past are not correlated with actions in the future conditioned on the state s_t (otherwise we would always have a correlation) ,which  is needed to use the policy gradient theorem. The assumption is only true for MDPs. However, using the option index as latent variable yields a PoMDP. There, this assumption does not hold any more. Example: Reward at time step t-1 depends on the action, which again depends on the option o_t-1. Action at time step t depends on o_t. Hence, there is a strong correlation between reward r_t-1 and action a_t+1 as o_t and o_t+1 are strongly correlated. o_t is not a conditional variable of the policy as it is not part of the state, thats why this assumption does not work any more.
Summary: The paper is well written and presents a good extension of inference based option discovery. However, the results are not convincing and there is a crucial issue in the assumptions of the algorithm.


Published as a conference paper at ICLR 2018 MASTERING THE DUNGEON: GROUNDED LANGUAGE LEARNING BY MECHANICAL TURKER DESCENT
Contrary to most natural language processing research, which makes use of static datasets, humans learn language interactively, grounded in an environment. In this work we propose an interactive learning procedure called Mechanical Turker Descent (MTD) and use it to train agents to execute natural language commands grounded in a fantasy text adventure game. In MTD, Turkers compete to train better agents in the short term, and collaborate by sharing their agents' skills in the long term. This results in a gamified, engaging experience for the Turkers and a better quality teaching signal for the agents compared to static datasets, as the Turkers naturally adapt the training data to the agent's abilities.

TL;DR of paper: Improved human-in-the-loop data collection using crowdsourcing. The basic gist is that on every round, N mechanical turkers will create their own dataset. Each turker gets a copy of a base model which is trained on their own dataset, and each trained model is evaluated on all the other turker datasets. The top-performing models get a cash bonus, incentivizing turkers to provide high quality training data. A new base model is trained on the pooled-together data of all the turkers, and a new round begins. The results indicate an improvement over static data collection.
This idea of HITL dataset creation is interesting, because the competitive aspect incentivizes turkers to produce high quality data. Judging by the feedback given by turkers in the appendix, the workers seem to enjoy the competitive aspect, which would hopefully lead to better data. The results seem to suggest that MTD provides an improvement over non-HITL methods.
The authors repeatedly emphasize the "collaborative" aspect of MTD, saying that the turkers have to collaborate to produce similar dataset distributions, but this is misleading because the turkers don't get to see other datasets. MTD is mostly competitive, and the authors should reduce the emphasis on a stretched definition of collaboration.
One questionable aspect of MTD is that the turkers somehow have to anticipate what are the best examples for the model to train with. That is, the turkers have to essentially perform the example selection process in active learning with relatively little interaction with the training model. While the turkers are provided immediate feedback when the model already correctly classifies the proposed training example, it seems difficult for turkers to anticipate when an example is too hard, because they have no idea about the learning process.
My biggest criticism is that MTD seems more like an NLP paper rather than an ICLR paper. I gave a 7 because I like the idea, but I wouldn't be upset if the AC recommends submitting to an NLP conference instead.


Recent advances in computing technology and sensor design have made it easier to collect longitudinal or time series data from patients, resulting in a gigantic amount of available medical data. Most of the medical time series lack annotations or even when the annotations are available they could be subjective and prone to human errors. Earlier works have developed natural language processing techniques to extract concept annotations and/or clinical narratives from doctor notes. However, these approaches are slow and do not use the accompanying medical time series data. To address this issue, we introduce the problem of concept annotation for the medical time series data, i.e., the task of predicting and localizing medical concepts by using the time series data as input. We propose Relational Multi-Instance Learning (RMIL) -a deep Multi Instance Learning framework based on recurrent neural networks, which uses pooling functions and attention mechanisms for the concept annotation tasks. Empirical results on medical datasets show that our proposed models outperform various multi-instance learning models.

The paper addresses the classification of medical time-series data by formulating the problem as a multi-instance learning (MIL) task, where there is an instance for each timestep of each time series, labels are observed at the time-series level (i.e. for each bag), and the goal is to perform instance-level and series-level (i.e. bag-level) prediction.  The main difference from the typical MIL setup is that there is a temporal relationship between the instances in each bag.  The authors propose to model this using a recurrent neural network architecture.  The aggregation function which maps instance-level labels to bag-level labels is modeled using a pooling layer (this is actually a nice way to describe multi-instance classification assumptions using neural network terminology).  An attention mechanism is also used.
The proposed time-series MIL problem formulation makes sense.  The RNN approach is novel to this setting, if somewhat incremental.  One very positive aspect is that results are reported exploring the impact of the choice of recurrent neural network architecture, pooling function, and attention mechanism.  Results on a second dataset are reported in the appendix, which greatly increases confidence in the generalizability of the experiments.  One or more additional datasets would have helped further solidify the results, although I appreciate that medical datasets are not always easy to obtain.  Overall, this is a reasonable paper with no obvious major flaws.  The novelty and impact may be greater on the application side than on the methodology side.
Minor suggestions:
-The term "relational multi-instance learning" seems to suggest a greater level of generality than the work actually accomplishes.  The proposed methods can only handle time-series / longitudinal dependencies, not arbitrary relational structure.  Moreover, multi-instance learning is typically viewed as an intermediary level of structure "in between" propositional learning (i.e. the standard supervised learning setting) and fully relational learning, so the "relational multi-instance learning" terminology sounds a little strange. Cf.:
De Raedt, L. (2008). Logical and relational learning. Springer Science & Business Media.
-Pg 3, a capitalization typo: "the Multi-instance learning framework"
-The equation for the bag classifier on page 4 refers to the threshold-based MI assumption, which should be attributed to the following paper:
Weidmann, N., Frank, E. & Pfahringer, B. 2003. A two-level learning method for generalized multi-instance problems. In Proceedings of the 14th European Conference on Machine Learning,
Springer, 468-479.
(See also: J. R. Foulds and E. Frank. A review of multi-instance learning assumptions. Knowledge Engineering Review, 25(1):1-25, 2010. )
- Pg 5, "Table 1" vs "table 1" - be consistent.
-A comparison to other deep learning MIL methods, i.e. those that do not exploit the time-series nature of the problem, would be valuable.  I wouldn't be surprised if other reviewers insist on this.


We establish a theoretical link between evolutionary algorithms and variational parameter optimization of probabilistic generative models with binary hidden variables. While the novel approach is independent of the actual generative model, here we use two such models to investigate its applicability and scalability: a noisy-OR Bayes Net (as a standard example of binary data) and Binary Sparse Coding (as a model for continuous data). Learning of probabilistic generative models is first formulated as approximate maximum likelihood optimization using variational expectation maximization (EM). We choose truncated posteriors as variational distributions in which discrete latent states serve as variational parameters. In the variational E-step, the latent states are then optimized according to a tractable free-energy objective. Given a data point, we can show that evolutionary algorithms can be used for the variational optimization loop by (A) considering the bit-vectors of the latent states as genomes of individuals, and by (B) defining the fitness of the individuals as the (log) joint probabilities given by the used generative model. As a proof of concept, we apply the novel evolutionary EM approach to the optimization of the parameters of noisy-OR Bayes nets and binary sparse coding on artificial and real data (natural image patches). Using point mutations and single-point cross-over for the evolutionary algorithm, we find that scalable variational EM algorithms are obtained which efficiently improve the data likelihood. In general we believe that, with the link established here, standard as well as recent results in the field of evolutionary optimization can be leveraged to address the difficult problem of parameter optimization in generative models.

## Review summary
Overall, the paper makes an interesting effort to tightly integrate
expectation-maximization (EM) training algorithms with evolutionary algorithms
(EA). However, I found the technical description lacking key details and the
experimental comparisons inadequate. There were no comparisons to non-
evolutionary EM algorithms, even though they exist for the models in question.
Furthermore, the suggested approach lacks a principled way to select
and tune key hyperparameters. I think the broad idea of using EA as a substep
within a monotonically improving free energy algorithm could be interesting,
but needs far more experimental justification.
## Pros / Stengths
+ effort to study more than one model family
+ maintaining monotonic improvement in free energy
## Cons / Limitations
- poor technical description and justification of the fitness function
- lack of comparisons to other, non-EA algorithms
- lack of study of hyperparameter sensitivity
## Paper summary
The paper suggests a variant of the EM algorithm for binary hidden variable
models, where the M-step proceeds as usual but the E-step is different in two
ways. First, following work by J. Lucke et al on Truncated Posteriors, the
true posterior over the much larger space of all possible bit vectors is
approximated by a more tractable small population of well-chosen bit vectors,
each with some posterior weight. Second, this set of bit vectors is updated
using an evolutionary/genetic algorithm. This EA is the core contribution,
since the work on Trucated Posteriors has appeared before in the literature.
The overall EM algorithm still maintains monotonic improvement of a free
energy objective.
Two well-known generative models are considered: Noisy-Or models for discrete
datasets and Binary Sparse Coding for continuous datasets. Each has a
previously known, closed-form M-step (given in supplement). The focus is on
the E-step: how to select the H-dimensional bit vector for each data point.
Experiments on artificial bars data and natural image patch datasets compare
several variants of the proposed method, while varying a few EA method
substeps such as selecting parents by fitness or randomly, including crossover
or not, or using generic or specialized mutation rates.
## Significance
Combining evolutionary algorithms (EA) within EM has been done previously, as
in Martinez and Vitria (Pattern Recog. Letters, 2000) or Pernkopf and
Bouchaffra (IEEE TPAMI, 2005) for mixture models. However, these efforts seem
to use EA in an "outer loop" to refine different runs of EM, while the present
approach uses EA in a substep of a single run of EM. I guess this is
technically different, but it is already well known that any E-step method
which monotonically improves the free energy is a valid algorithm. Thus, the
paper's significance hinges on demonstrating that the particular E step chosen
is better than alternatives. I don't think the paper succeeded very well at
this: there were no comparisons to non-EA algorithms, or to approaches that
use EA in the "outer loop" as above.
## Clarity of Technical Approach
What is \tilde{log P} in Eq. 7? This seems a fundamental expression. Its
plain-text definition is: "the logarithm of the joint probability where
summands that do not depend on the state s have been elided". To me, this
definition is not precise enough for me to reproduce confidently... is it just
log p(s_n, y_n | theta)? I suggest revisions include a clear mathematical
definition. This omission inhibits understanding of this paper's core
contributions.
Why does the fitness expression F defined in Eq. 7 satisfy the necessary
condition for fitness functions in Eq. 6? This choice of fitness function does
not seem intuitive to me. I think revisions are needed to *prove* this fitness
function obeys the comparison property in Eq. 6.
How can we compute the minimization substep in Eq. 7 (min_s \tilde{logP})? Is
this just done by exhaustive search over bit vectors? I think this needs
clarification.
## Quality of Experiments
The experiments are missing a crucial baseline: non-EA algorithms. Currently
only several varieties of EA are compared, so it is impossible to tell if the
suggested EA strategies even improve over non-EA baselines. As a specific
example, previous work already cited in this paper -- Henniges et al (2000) --
has developed a non-EA EM algorithm for Binary Sparse Coding, which already
uses the truncated posterior formulation. Why not compare to this?
The proposed algorithm has many hyperparameters, including number of
generations, number of parents, size of the latent space H, size of the
truncation, etc. The current paper offers little advice about selecting these
values intelligently, but presumably performance is quite sensitive to these
values. I'd like to see some more discussion of this and (ideally) more
experiments to help practitioners know which parameters matter most,
especially in the EA substep.
Runtime analysis is missing as well: Is runtime dominated by the EA step? How
does it compare to non-EA approaches? How big of datasets can the proposed
method scale to?
The reader walks away from the current toy bars experiment somewhat confused.
The Noisy-Or experiment did not favor crossover and and favored specialized
mutations, while the BSC experiment reached the opposite conclusions. How does
one design an EA for a new dataset, given this knowledge? Do we need to
exhaustively try all different EA substeps, or are there smarter lessons to
learn?
## Detailed comments
Bottom of page 1: I wouldn't say that "variational EM" is an approximation to
EM. Sometimes moving from EM to variational EM can mean we estimate posteriors
(not point estimates) for both local (example-specific) and global parameters.
Instead, the *approximation* comes simply from restricting the solution space
to gain tractability.
Sec. 2: Make clear earlier that hidden var "s" is assumed to be discrete, not
continuous.
After Mutation section: Remind readers that "N_g" is number of generations


Neural conversational models are widely used in applications like personal assistants and chat bots. These models seem to give better performance when operating on word level. However, for fusion languages like French, Russian and Polish, vocabulary size sometimes become infeasible since most of the words have lots of word forms. To reduce vocabulary size we propose a new pipeline for building conversational models: first generate words in standard form and then transform them into a grammatically correct sentence. For this task we propose a neural network architecture that efficiently employs correspondence between standardised and target words and significantly outperforms character-level models while being 2x faster in training and 20% faster at evaluation. The proposed pipeline gives better performance than character-level conversational models according to assessor testing.

In this work, the authors propose a sequence-to-sequence architecture that learns a mapping from a normalized sentence to a grammatically correct sentence. The proposed technique is a simple modification to the standard encoder-decoder paradigm which makes it more efficient and better suited to this task. The authors evaluate their technique using three morphologically rich languages French, Polish and Russian and obtain promising results.
The morphological agreement task would be an interesting contribution of the paper, with wider potential. But one concern that I have is regarding the evaluation metrics used for it. Firstly, word accuracy rate doesn't seem appropriate, as it does not measure morphological agreement. Secondly, sentence accuracy (w.r.t. the sentences from which the normalized sentences are derived) is not indicative of morphological agreement: even "wrong" sentences in the output could be perfectly valid in terms of agreement. A grammatical error rate (fraction of grammatically wrong sentences produced) would probably be a better measure.
Another concern I have is regarding the quality of the baseline: Additional variants of the baseline models should be considered and the best one reported. Specifically, in the conversation task, have the authors considered switching the order of normalized answer and context in the input? Also, the word order of the normalized answer and/or context could be reversed (as is done in sequence-to-sequence translation models).
Also, many experimental details are missing from the draft:
-- What are the sizes of the train/test sets derived from the OpenSubtitles database?
-- Details of the validation sets used to tune the models.
-- In Section 5.4, no details of the question-answer corpus are provided. How many pairs were extracted? How many were used for training and testing?
-- In Section 5.4.1, how many assessors participated in the evaluation and how many questions were evaluated?
-- In some of the tables (e.g. 6, 7, 8) which show example sentences from Polish, Russian and French, please provide some more information in the accompanying text on how to interpret these examples (since most readers may not be familiar with these languages).
Pros:
-- Efficient model
-- Proposed architecture is general enough to be useful for other sequence-to-sequence problems
Cons:
-- Evaluation metrics for the morphological agreement task are unsatisfactory
-- It would appear that the baselines could be improved further using standard techniques


Published as a conference paper at ICLR 2018 KERNEL IMPLICIT VARIATIONAL INFERENCE
Recent progress in variational inference has paid much attention to the flexibility of variational posteriors. One promising direction is to use implicit distributions, i.e., distributions without tractable densities as the variational posterior. However, existing methods on implicit posteriors still face challenges of noisy estimation and computational infeasibility when applied to models with high-dimensional latent variables. In this paper, we present a new approach named Kernel Implicit Variational Inference that addresses these challenges. As far as we know, for the first time implicit variational inference is successfully applied to Bayesian neural networks, which shows promising results on both regression and classification tasks.

This paper presents Kernel Implicit Variational Inference (KIVI), a novel class of implicit variational distributions. KIVI relies on a kernel approximation to directly estimate the density ratio. Importantly, the optimal kernel approximation in KIVI has closed-form solution, which allows for faster training since it avoids gradient ascent steps that may soon get "outdated" as the optimization over the variational distribution runs. The paper presents experiments on a variety of scenarios to show the performance of KIVI.
Up to my knowledge, the idea of estimating the density ratio using kernels is novel. I found it interesting, specially since there is a closed-form solution for this estimate. The closed form solution involves a matrix inversion, but this shouldn't be an issue, as the matrix size is controlled by the number of samples, which is a parameter that the practitioner can choose. I also found interesting the implicit MMNN architecture proposed in Section 4.
The experiments seem convincing too, although I believe the paper could probably be improved by comparing with other implicit VI methods, such as [Liu & Feng], [Tran et al.], or others.
My major criticism with the paper is the quality of the writing. I found quite a few errors in every page, which significantly affects readability. I strongly encourage the authors to carefully review the entire paper and search for typos, grammatical errors, unclear sentences, etc.
Please find below some further comments broken down by section.
Section 1: In the introduction, it is unclear to me what "protect these models" means. Also, in the second paragraph, the authors talk about "often leads to biased inference". The concept to "biased inference" is unclear. Finally, the sentence "the variational posterior we get in this way does not admit a tractable likelihood" makes no sense to me; how can a posterior admit (or not admit) a likelihood?
Section 3: The first paragraph of the KIVI section is also unclear to me. In Section 3.1, it looks like the cost function L(\hat(r)) is different from the loss in Eq. 1, so it should have a different notation. In Eq. 4, I found it confusing whether L(r)=J(r). Also, it would be nice to include a brief description of why the expectation in Eq. 4 is taken w.r.t. p(z) instead of q(z), for those readers who are less familiar with [Kanamori et al.]. Finally, the motivation behind the "reverse ratio trick" was unclear to me (the trick is clear, but I didn't fully understand why it's needed).
Section 4: The first paragraph of the example can be improved with a brief discussion of why the methods of [Mescheder et al.] and [Song et al.] "are nor applicable". Also, the paragraph above Eq. 11 ("When modeling a matrix...") was unclear to me.
Section 6: In Figure 1(a), I think there must be something wrong, because it is well-known that VI tends to cover one of the modes of the posterior only due to the form of the KL divergence (in contrast to EP, which should look like the curve in the figure). Additionally, Figure 3(a) (and the explanation in the text) was unclear to me. Finally, I disagree with the discussion regarding overfitting in Figure 3(b): that plot doesn't show overfitting because it is a plot of the training loss (and overfitting occurs on test); instead it looks like an optimization issue that makes the bound decrease.
**** EDITS AFTER AUTHORS' REBUTTAL ****
I increased the rating to 7 after reading the revised version.


Many regularization methods have been proposed to prevent overfitting in neural networks. Recently, a regularization method has been proposed to optimize the variational lower bound of the Information Bottleneck Lagrangian. However, this method cannot be generalized to regular neural network architectures. We present the activation norm penalty that is derived from the information bottleneck principle and is theoretically grounded in a variation dropout framework. Unlike in previous literature, it can be applied to any general neural network. We demonstrate that this penalty can give consistent improvements to different state of the art architectures both in language modeling and image classification. We present analyses on the properties of this penalty and compare it to other methods that also reduce mutual information.

This paper tries to create a mapping between activation norm penalties and information bottleneck framework using variational dropout framework. While I find the path taken interesting, the paper itself is hard to follow, mostly due to constantly flipping notation (cons section below lists some of the issues) and other linguistic errors. In the current form, this work is somewhere between a theoretical paper and an empirical one, however for a theoretical one it lacks strictness, while for empirical one - novelty.
From theoretical perspective:
The main claim in this paper seems to be (10), however it is not formalised in any form of theorem, and so -- lacks a lock of strictness. Even under the assumption that it is updated, and made more strict - what is a crucial problem is a claim, that after arriving at:
tr[ K(X, X) ] - ln( det[ K(X, X) ] )
dropping the log determinant is anyhow justified, to keep the reasoning/derivation of the whole method sound. Authors claim that quote "As for the determinant of the covariance matrix of Gaussian Process, we cannot easily evaluate or
derive its gradient, so we do not include it in our computation." Which is not a justification for treating the derivation as a proper connection between penalising activities norm and information bottleneck idea. Terms like this will emerge in many other models, where one assumes diagonal covariance Gaussians; in fact the easiest model to justify this penalty is just to say one introduces diagonal Gaussian prior over activations, and that's it. Well justified penalty, easy to connect to many generalisation bound claims. However in the current approach the connection is simply not proven in the paper.
From practical perspective:
Activation norm penalties are well known objects, used for many years (L1 activation penalty for at least 6 years now, see "Deep Sparse Rectiﬁer Neural Networks"; various activation penalties, including L2, changes in L2, etc. in Krueger PhD dissertation). Consequently for a strong empirical paper I would expect much more baselines, including these proposed by Krueger et al.
Pros:
- empirical improvements shown on two different classes of problems.
- interesting path through variational dropout is taken to show some equivalences
Cons:
- there is no proper proof of claimed connection between IB and ANP, as it would require a determinant of K(X, X) to be 1.
- work is not strict enough for a theoretical paper, and does not include enough comparison for empirical one
- paper is full of typing/formatting/math errors/not well explained objects, which make it hard to read, to name a few:
 * fonts of objects used in equations change through the text - there is a \textbf{W} and normal W, \textbf{I} and normal I, similarly with X, Ts etc. without any explanation. I am assuming font are assigned randomly and they represent the same object.
 * dydp in (5) is in the wrong integral
* \sigma switches meaning between non-linearity and variance
* what does it mean to define a normal distribution with 0 variance (like in (1)). Did authors mean  an actual "degenerate Gaussian", which does not have a PDF? But then p(y|t) is used in (5), and under such definition it does not exist, only CDF does.
* \Sigma_1 in 3.2 is undefined, was r(t) supposed to be following N(0, \Sigma_1)  instead of written N(0, I)?


Published as a conference paper at ICLR 2018 SPECTRAL NORMALIZATION FOR GENERATIVE ADVERSARIAL NETWORKS
One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to incorporate into existing implementations. We tested the efficacy of spectral normalization on CIFAR10, STL-10, and ILSVRC2012 dataset, and we experimentally confirmed that spectrally normalized GANs (SN-GANs) is capable of generating images of better or equal quality relative to the previous training stabilization techniques. The code with Chainer (Tokui et al., 2015), generated images and pretrained models are available at https://github.com/pfnet-research/sngan_ projection.

This paper borrows the classic idea of spectral regularization, recently applied to deep learning by Yoshida and Miyato (2017) and use it to normalize GAN objectives. The ensuing GAN, coined SN-GAN, essentially ensures the Lipschitz property of the discriminator. This Lipschitz property has already been proposed by recent methods and has showed some success. However,  the authors here argue that spectral normalization is more powerful; it allows for models of higher rank (more non-zero singular values) which implies a more powerful discriminator and eventually more accurate generator. This is demonstrated in comparison to weight normalization in Figure 4. The experimental results are very good and give strong support for the proposed normalization.
While the main idea is not new to machine learning (or deep learning), to the best of my knowledge it has not been applied on GANs. The paper is overall well written (though check Comment 3 below), it covers the related work well and it includes an insightful discussion about the importance of high rank models. I am recommending acceptance, though I anticipate to see a more rounded evaluation of the exact mechanism under which SN improves over the state of the art. More details in the comments below.
Comments:
1. One concern about this paper is that it doesn’t fully answer the reasons why this normalization works better. I found the discussion about rank to be very intuitive, however this intuition is not fully tested.  Figure 4 reports layer spectra for SN and WN. The authors claim that other methods, like (Arjovsky et al. 2017) also suffer from the same rank deficiency. I would like to see the same spectra included.
2. Continuing on the previous point: maybe there is another mechanism at play beyond just rank that give SN its apparent edge? One way to test the rank hypothesis and better explain this method is to run a couple of truncated-SN experiments. What happens if you run your SN but truncate its spectrum after every iteration in order to make it comparable to the rank of WN? Do you get comparable inception scores? Or does SN still win?
3. Section 4 needs some careful editing for language and grammar.


Recurrent Neural Networks (RNNs) are powerful autoregressive sequence models for learning prevalent patterns in natural language. Yet language generated by RNNs often shows several degenerate characteristics that are uncommon in human language; while fluent, RNN language production can be overly generic, repetitive, and even self-contradictory. We postulate that the objective function optimized by RNN language models, which amounts to the overall perplexity of a text, is not expressive enough to capture the abstract qualities of good generation such as Grice's Maxims. In this paper, we introduce a general learning framework that can construct a decoding objective better suited for generation. Starting with a generatively trained RNN language model, our framework learns to construct a substantially stronger generator by combining several discriminatively trained models that can collectively address the limitations of RNN generation. Human evaluation demonstrates that text generated by the resulting system is preferred over that of baselines by a large margin and significantly enhances the overall coherence, style, and information content of the generated text.

This paper proposes to bring together multiple inductive biases that hope to correct for inconsistencies in sequence decoding. Building on previous works that utilize modified objectives to generate sequences, this work proposes to optimize for the parameters of a pre-defined combination of various sub-objectives. The human evaluation is straight-forward and meaningful to compensate for the well-known inaccuracies of automatic evaluation.
While the paper points out that they introduce multiple inductive biases that are useful to produce human-like sentences, it is not entirely correct that the objective is being learnt as claimed in portions of the paper. I would like this point to be clarified better in the paper.
I think showing results on grounded generation tasks like machine translation or image-captioning would make a stronger case for evaluating relevance. I would like to see comparisons on these tasks.
----
After reading the paper in detail again and the replies, I am downgrading my rating for this paper. While I really like the motivation and the evaluation proposed by this work, I believe that fixing the mismatch between the goals and the actual approach will make for a stronger work.
As pointed out by other reviewers, while the goals and evaluation seem to be more aligned with Gricean maxims, some components of the objective are confusing. For instance, the length penalty encourages longer sentences violating quantity, manner (be brief) and potentially relevance. Further, the repetition model address the issue of RNNs failing to capture long-term contextual dependencies -- how much does such a modified objective affect models with attention / hierarchical models is not clear from the formulation.
As pointed out in my initial review evaluation of relevance on the current task is not entirely convincing. A very wide variety of topics are feasible for a given context sentence. Grounded generation like MT / captioning would have been a more convincing evaluation. For example, Wu et al. (and other MT works) use a coverage term and this might be one of the indicators of relevance.
Finally, I am not entirely convinced by the update regarding "learning the objective". While I agree with the authors that the objective function is being dynamically updated, the qualities of good language is encoded manually using a wide variety of additional objectives and only the relative importance of each of them is learnt.


Published as a conference paper at ICLR 2018 GENERALIZING ACROSS DOMAINS VIA CROSS-GRADIENT TRAINING
We present CROSSGRAD, a method to use multi-domain training data to learn a classifier that generalizes to new domains. CROSSGRAD does not need an adaptation phase via labeled or unlabeled data, or domain features in the new domain. Most existing domain adaptation methods attempt to erase domain signals using techniques like domain adversarial training. In contrast, CROSSGRAD is free to use domain signals for predicting labels, if it can prevent overfitting on training domains. We conceptualize the task in a Bayesian setting, in which a sampling step is implemented as data augmentation, based on domain-guided perturbations of input instances. CROSSGRAD parallelly trains a label and a domain classifier on examples perturbed by loss gradients of each other's objectives. This enables us to directly perturb inputs, without separating and re-mixing domain signals while making various distributional assumptions. Empirical evaluation on three different applications where this setting is natural establishes that (1) domain-guided perturbation provides consistently better generalization to unseen domains, compared to generic instance perturbation methods, and that (2) data augmentation is a more stable and accurate method than domain adversarial training.

This paper proposed a domain generalization approach by domain-dependent data augmentation. The augmentation is guided by a network that is trained to classify a data point to different domains. Experiments on four datasets verify the effectiveness of the proposed approach.
Strengths:
+ The proposed classification model is domain-dependent, as opposed to being domain-invariant. This is new and differs from most existing works on domain adaptation/generalization, to the best of my knowledge.
+ The experiments show that the proposed method outperforms two baselines. However, more related approaches could be included to strengthen the experiments (see below for details).
Weaknesses:
- The paper studies domain generalization and yet fails to position it in the right literature. By a simple search of "domain generalization" using Google Scholar, I found several existing works on this problem and have listed some below. The authors may consider to include them in both the related works and the experiments.
Questions:
1. It is intuitive to directly define the data augmentation by x_i+Grad_x J_d. Why is it necessary to instead define it as the inverse transformation G^{-1}(g') and then go through the approximations to derive the final augmentation?
2. Is the CrossGrad training necessary? What if one trains the network in two steps? Step 1: learn G using J_d and a regularization to avoid misclassification over the labels using the original data. Step 2: Learn the classification network (possibly different from G) by the domain-dependent augmentation.
Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, and Gianfranco Doretto. Unified deep supervised
domain adaptation and generalization. In IEEE International Conference on Computer
Vision (ICCV), 2017.
Muandet, K., Balduzzi, D. and Schölkopf, B., 2013. Domain generalization via invariant feature representation. In Proceedings of the 30th International Conference on Machine Learning (ICML-13) (pp. 10-18).
Xu, Z., Li, W., Niu, L. and Xu, D., 2014, September. Exploiting low-rank structure from latent domains for domain generalization. In European Conference on Computer Vision (pp. 628-643). Springer, Cham.
Ghifary, M., Bastiaan Kleijn, W., Zhang, M. and Balduzzi, D., 2015. Domain generalization for object recognition with multi-task autoencoders. In Proceedings of the IEEE international conference on computer vision (pp. 2551-2559).
Gan, C., Yang, T. and Gong, B., 2016. Learning attributes equals multi-source domain generalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 87-97).


The learnability of different neural architectures can be characterized directly by computable measures of data complexity. In this paper, we reframe the problem of architecture selection as understanding how data determines the most expressive and generalizable architectures suited to that data, beyond inductive bias. After suggesting algebraic topology as a measure for data complexity, we show that the power of a network to express the topological complexity of a dataset in its decision boundary is a strictly limiting factor in its ability to generalize. We then provide the first empirical characterization of the topological capacity of neural networks. Our empirical analysis shows that at every level of dataset complexity, neural networks exhibit topological phase transitions and stratification. This observation allowed us to connect existing theory to empirically driven conjectures on the choice of architectures for a single hidden layer neural networks.

The authors propose to use the homology of the data as a measurement of the expressibility of a deep neural network. The paper is mostly experimental. The theoretical section (3.1) is only reciting existing theory (Bianchini et al.). Theorem 3.1 is not surprising either: it basically says spaces with different topologies differ at some parts.
As for the experiments, the idea is tested on synthetic and real data. On synthetic data, it is shown that the number of neurons of the network is correlated with the homology it can express. On real data, the tool of persistent homology is applied. It is observed that the data in the final layer do have non-trivial signal in terms of persistent homology.
I do like the general idea of the paper. It has great potentials. However, it is much undercooked. In particular, it could be improved as follows:
* 1) the main message of the paper is unclear to me.  Results observed in the synthetic experiments seem to be a confirmation of the known results by Bianchini et al.: the Betti number a network can express is linear to the number of hidden units, h, when the input dimension n is a constant.
To be convinced, I would like to see much stronger experimental evidence: Reporting results on a single layer network is unsettling. It is known that the network expressibility is highly related to the depth (Eldan & Shamir 2016). So what about networks with more layers? Is the stratification observation statistically significant? These experiments are possible for synthetic data.
* 2) The usage of persistent homology is not well justified. A major part of the paper is devoted to persistent homology. It is referred to as a robust computation of the homology and is used in the real data experiments. However, persistent homology itself was not originally invented to recover the homology of a fixed space. It was intended to discover homology groups at all different scales (in terms of the function value). Even with the celebrated stability theorem (Cohen-Steiner et al. 2007) and statistical guarantees (Chazal et al. 2015), the relationship between the Vietoris-Rips filtration persistent homology and the homology of the classifier region/boundary is not well established. To make a solid statement, I suggest authors look into the following papers
Homology and robustness of level and interlevel sets
P Bendich, H Edelsbrunner, D Morozov, A Patel, Homology, Homotopy and Applications 15 (1), 51-72, 2013
Herbert Edelsbrunner, Michael Kerber: Alexander Duality for Functions: the Persistent Behavior of Land and Water and Shore. Proceedings of the 28th Annual Symposium on Computational Geometry, pp. 249-258 (SoCG 2012)
There are also existing work on how the homology of a manifold or stratified space can be recovered using its samples. They could be useful. But the settings are different: in this problem, we have samples from the positive/negative regions, rather than the classification boundary.
Finally, the gap in concepts carries to experiments. When persistent homology of different real data are reported. It is unclear how they reflect the actually topology of the classification region/boundary. There are also significant amount of approximation due to the natural computational limitation of persistent homology. In particular, LLE and subsampling are used for the computation. These methods can significantly hurt persistent homology computation. A much more proper way is via the sparsification approach.
SimBa: An Efficient Tool for Approximating Rips-Filtration Persistence via Simplicial Batch-Collapse
T. K. Dey, D. Shi and Y. Wang. Euro. Symp. Algorithms (ESA) 2016, 35:1--35:16
* 3) Finally, to support the main thesis, it is crucial to show that the topological measure is revealing information existing ones do not. Some baseline methods such as other geometric information (e.g., volume and curvature) are quite necessary.
* 4) Important papers about persistent homology in learning could be cited:
Using persistent homology in deep convolutional neural network:
Deep Learning with Topological Signatures
C. Hofer, R. Kwitt, M. Niethammer and A. Uhl, NIPS 2017
Using persistent homology as kernels:
Sliced Wasserstein Kernel for Persistence Diagrams
Mathieu Carrière, Marco Cuturi, Steve Oudot, ICML 2017.
* 5) Minor comments:
Small typos here and there: y axis label of Fig 5, conclusion section.


Published as a conference paper at ICLR 2018 INITIALIZATION MATTERS: ORTHOGONAL PREDICTIVE STATE RECURRENT NEURAL NETWORKS
Learning to predict complex time-series data is a fundamental challenge in a range of disciplines including Machine Learning, Robotics, and Natural Language Processing. Predictive State Recurrent Neural Networks (PSRNNs) (Downey et al.,  2017)  are a state-of-the-art approach for modeling time-series data which combine the benefits of probabilistic filters and Recurrent Neural Networks into a single model. PSRNNs leverage the concept of Hilbert Space Embeddings of distributions (Smola et al., 2007) to embed predictive states into a Reproducing Kernel Hilbert Space, then estimate, predict, and update these embedded states using Kernel Bayes Rule. Practical implementations of PSRNNs are made possible by the machinery of Random Features, where input features are mapped into a new space where dot products approximate the kernel well. Unfortunately PSRNNs often require a large number of RFs to obtain good results, resulting in large models which are slow to execute and slow to train. Orthogonal Random Features (ORFs) (Yu et al., 2016) is an improvement on RFs which has been shown to decrease the number of RFs required for pointwise kernel approximation. Unfortunately, it is not clear that ORFs can be applied to PSRNNs, as PSRNNs rely on Kernel Ridge Regression as a core component of their learning algorithm, and the theoretical guarantees of ORF do not apply in this setting. In this paper, we extend the theory of ORFs to Kernel Ridge Regression and show that ORFs can be used to obtain Orthogonal PSRNNs (OPSRNNs), which are smaller and faster than PSRNNs. In particular, we show that OPSRNN models clearly outperform LSTMs and furthermore, can achieve accuracy similar to PSRNNs with an order of magnitude smaller number of features needed.

I was very confused by some parts of the paper that are simple copy-past from the paper of Downey et al.  which has been accepted for publication in NIPS. In particular, in section 3, several sentences are taken as they are from the Downey et al.’s paper. Some examples :
« provide a compact representation of a dynamical system
by representing state as a set of predictions of features of future observations. »
« a predictive state is defined as… , where…  is a vector of features of future observations and ...  is a vector of
features of historical observations. The features are selected such that ...  determines the distribution
of future observations … Filtering is the process of mapping a predictive state… »
Even the footnote has been copied & pasted: « For convenience we assume that the system is k-observable: that is, the distribution of all future observations
is determined by the distribution of the next k observations. (Note: not by the next k observations
themselves.) At the cost of additional notation, this restriction could easily be lifted. »
«  This approach is fast, statistically consistent, and reduces to simple
linear algebra operations. »
Normally, I should have stopped reviewing, but I decided to continue  since those parts only concerned the preliminaries part.
A key element in PSRNN is to used as an initialization a kernel ridge regression. The main result here, is to show that using orthogonal random features approximates well the original kernel comparing to random fourrier features as considered in PSRNN. This result is formally stated and proved in the paper.
The paper comes with some experiments in order to empirically demonstrate the superiority  orthogonal random features over RFF. Three data sets are considered (Swimmer,  Mocap and  Handwriting).
I found it that the contribution of the paper is very limited. The connexion to PSRNN is very tenuous since the main results are about the regression part. in Theorems 2 and 3 there are no mention to PSRNN.
Also the experiment is not very convincing. The datasets are too small with observations in low dimensions, and I found it not very fair to consider LSTM in such settings.
Some minor remarks:
- p3: We use RFs-> RFFs
- p5: ||X||, you mean |X| the size of the dataset
- p12: Eq (9). You need to add « with probability $1-\rho$ as in Avron’s paper.
- p12: the derivation of Eq (10) from Eq (9) needs to be detailed.
I thank the author for their detailed answers. Some points have been clarified but other still raise issues. In particular, I continue thinking that the contribution is limited. Accordingly, I did not change my scores.


Training methods for deep networks are primarily variants on stochastic gradient descent. Techniques that use (approximate) second-order information are rarely used because of the computational cost and noise associated with those approaches in deep learning contexts. However, in this paper, we show how feedforward deep networks exhibit a low-rank derivative structure. This low-rank structure makes it possible to use second-order information without needing approximations and without incurring a significantly greater computational cost than gradient descent. To demonstrate this capability, we implement Cubic Regularization (CR) on a feedforward deep network with stochastic gradient descent and two of its variants. There, we use CR to calculate learning rates on a per-iteration basis while training on the MNIST and CIFAR-10 datasets. CR proved particularly successful in escaping plateau regions of the objective function. We also found that this approach requires less problem-specific information (e.g. an optimal initial learning rate) than other first-order methods in order to perform well.

This paper proposes to set a global step size gradient-based optimization algorithms such as SGD and Adam using second order information. Instead of using second-order information to compute the update directly (as is done in e.g. Newton method), it is used to estimate the change of the objective function in a pre-computed direction. This is computationally much cheaper than full Newton because (a) the Hessian does not need to be inverted (b) vector-Hessian multiplication is only O(#parameters) for a single sample.
There are many issues.
### runtime and computational issues ###
Firstly, the paper does not clearly specify the algorithm it espouses. It states: "once the step direction had been determined, we considered that fixed, took the average of gT Hg and gT ∇f over all of the sample points to produce m (α) and then solved for a single αj value" You should present pseudo-code for this computation and not leave the reader to determine the detailed order of computation for himself. As it stands, it is not only difficult for the reader to infer these details, but also laborious to determine the computational cost per iteration on some network the reader might wish to apply your algorithm to. Since the paper discusses the computational cost of CR only in vague terms, you should at least provide pseudo-code.
Specifically, consider equation (80) at the very end of the appendix and consider the very last term in that equation. It contains d^2v/dwdw. This is a "heavy" term containing the second derivative of the last hidden layer with respect to weights. You do not specify how you compute this term or quantities involving this term. In a ReLU network, this term is zero due to local linearity, but since you claim that your algorithm is applicable to general networks, this term needs to be analyzed further.
While the precise algorithm you suggest is unclear, it's purpose is also unclear. You only use the Hessian to compute the g^THg terms, i.e. for Hessian-vector multiplication. But it is well-known that Hessian-vector multiplication is "relatively cheap" in deep networks and this fact has been used for several algorithms, e.g. http://www.iro.umontreal.ca/~lisa/pointeurs/ECML2011_CAE.pdf and https://arxiv.org/pdf/1706.04859.pdf. How is your method for computing g^THg different and why is it superior?
Also note that the low-rank structure of deep gradients is well-known and not a contribution of this paper. See e.g. https://www.usenix.org/system/files/conference/atc17/atc17-zhang.pdf
### Experiments ###
The experiments are very weak. In a network where weights are initialized to sensible values, your algorithm is shown not to improve upon straight SGD. You only demonstrate superior results when the weights are badly initialized. However, there are a very large number of techniques already that avoid the "SGD on ReLU network with bad initial weights" problem. The most well-known are batch normalization, He initialization and Adam but there are many others. I don't think it's a stretch to consider that problem "solved". Your algorithm is not shown to address any other problems, but what's worse is that it doesn't even seem to address that problem well. While your learning curves are better than straight SGD, I suspect they are well below the respective curves for He init or batchnorm. In any case, you would need to compare your algorithm against these state-of-the-art methods if your goal is to overcome bad initializations. Also, in appendix A, you state that CR can't even address weights that were initialized to values that are too large.
You claim that your algorithm helps with "overcoming plateaus". While I have heard the claim that deep network optimization suffers from intermediate plateaus before, I have not seen a paper studying / demonstrating this behavior. I suggest you cite several papers that do this and then replicate the plateau situations that arose in those papers and show that CR overcomes them, instead of resorting to a platenau situation that is essentially artificially induced by intentionally bad hyperparameter choices.
I do not understand why your initial learning rate for SGD in figures 2 and 3 (0.02 and 0.01 respectively) differ so much from the initial learning rate under CR. Aren't you trying to show that CR can find the "correct" learning rate? Wouldn't that suggest that initial learning rate for SGD should be comparable to the early learning rates chosen by CR? Wouldn't that suggest you should start SGD with a learning rate of around 2 and 0.35 respectively? Since you are annealing the learning rate for SGD, it's going to decline and get close to 0.02 / 0.01 anyway at some point. While this may not be as good as CR or indeed batchnorm or Adam, the blue constant curve you are showing does not seem to be a fair representation of what SGD can do.
You say the minibatch size is 32. For MNIST, this means that 1 epoch is around 1500 iterations. That means your plots only show the first epoch of training. But MNIST does not converge in 1 epoch. You should show the error curve until convergence is reached. Same for CIFAR.
"we are not interested in network performance measures such as accuracy and validation error" I strongly suspect your readers may be interested in those things. You should show validation classification error or at least training classification error in addition to cross-entropy error.
"we will also focus on optimization iteration rather than wall clock time" Again, your readers care more about the latter. You need to show either error curves by clock time or the total time to convergence or supplement your iteration-based graphs with a detailed discussion of how long an iteration takes.
The scope of the experiments is limited because only a single network architecture is considered, and it is not a state-of-the art architecture (no convolution, no normalization mechanism, no skip connections).
You state that you ran experiments on Adam, Adadelta and Adagrad, but you do not show the Adam results. You say in the text that they were the least favorable for CR. This suggests that you omitted the detailed results because they were unfavorable to you. This is, of course, unacceptable!
### (Un)suitability of ReLU for second-order analysis ###
You claim to use second-order information over the network to set the step size. Unfortuantely, ReLU networks do not have second-order information! They are locally linear. All their nonlinearity is contained in non-differentiable region boundaries. While this may lead to the Hessian being cheaper to compute, it means it is not representative of the actual behavior of the network. In fact, the only second-order information that is brought to bear in your experiments is the second-order information of the error function. I am not saying that this particular second-order information could not be useful, but you need to make a distinction in your paper between network second-order info and error function second-order info and make explicit that you only use the former in your experiments. As far as I know, most second-order papers use either tanh or a smoothed ReLU (such as the smoothed hinge used recently by Koh & Liang (https://arxiv.org/pdf/1703.04730.pdf)) for experiments to overcome the local linearity.
### The \sigma hyperparameter ###
You claim that \sigma is not as important / hard to set as \alpha in SGD or Adam. You state: "We also found that this ap- proach requires less problem-specific information (e.g. an optimal initial learning rate) than other first-order methods in order to perform well." You have not provided sufficient evidence for this claim. You say that \sigma can be chosen by considering powers of 10. In many networks, choosing \alpha by considering powers of 10 is sufficient! Even if powers of 2 are considered for \alpha, this would reduce the search effort only by factor log_2(10). Also, what if the range of \sigma values that need to be considered is larger than the range of \alpha values? Then setting \sigma would take more effort.
You do not give precise protocols how you set \sigma and how you set \alpha for non-CR algorithms. This should be clearly specified in Appendix A as it is central to your argument of easing hyperparameter search.
### Minor points ###
- Your introduction could benefit from a few more citations
- "The rank of the weighted sum of low rank components (as occurs with mini-batch sampling) is generally larger than the rank of the summed components, however." I don't understand this. Every sum can be viewed as a weighted sum and vice versa.
- Equation (8) could be motivated a bit better. I know it derives from Taylor's theorem, but it might be good to discuss how Taylor's theorem (and its assumptions) relate to deep networks.
- why the name "cubic regularization"? shouldn't it be something like "quadratic step size tuning"?
.
.
.
The reason I am giving a 2 instead of a 1 is because the core idea behind the algorithm given seems to me to have potential, but the execution is sorely lacking.
A final suggestion: You advertise as one of your algorithms upsides that it uses exact Hessian information. Howwever, since you only care about the scale of the second-order term and not its direction, I suspect exact calculation is far from necessary and you could get away with very cheap approximations, using for example techniques such as mean field analysis (e.g. http://papers.nips.cc/paper/6322-exponential-expressivity-in-deep-neural-networks-through-transient-chaos.pdf).


A COMPRESSED SENSING VIEW OF UNSUPERVISED TEXT EMBEDDINGS, BAG-OF-n-GRAMS, AND LSTMS
Low-dimensional vector embeddings, computed using LSTMs or simpler techniques, are a popular approach for capturing the "meaning" of text and a form of unsupervised learning useful for downstream tasks. However, their power is not theoretically understood. The current paper derives formal understanding by looking at the subcase of linear embedding schemes. Using the theory of compressed sensing we show that representations combining the constituent word vectors are essentially information-preserving linear measurements of Bag-of-n-Grams (BonG) representations of text. This leads to a new theoretical result about LSTMs: low-dimensional embeddings derived from a low-memory LSTM are provably at least as powerful on classification tasks, up to small error, as a linear classifier over BonG vectors, a result that extensive empirical work has thus far been unable to show. Our experiments support these theoretical findings and establish strong, simple, and unsupervised baselines on standard benchmarks that in some cases are state of the art among word-level methods. We also show a surprising new property of embeddings such as GloVe and word2vec: they form a good sensing matrix for text that is more efficient than random matrices, the standard sparse recovery tool, which may explain why they lead to better representations in practice.

My review reflects more from the compressive sensing perspective, instead that of deep learners.
In general, I find many of the observations in this paper interesting. However, this paper is not strong enough as a theory paper; rather, the value lies perhaps in its fresh perspective.
The paper studies text embeddings through the lens of compressive sensing theory. The authors proved that, for the proposed embedding scheme, certain LSTMs with random initialization are at least as good as the linear classifiers; the theorem is almost a direction application of the RIP of random Rademacher matrices. Several simplifying assumptions are introduced, which rendered the implication of the main theorem vague, but it can serve as a good start for the hardcore statistical learning-theoretical analysis to follow.
The second contribution of the paper is the (empirical) observation that, in terms of sparse recovery of embedded words, the pretrained embeddings are better than random matrices, the latter being the main focus of compressive sensing theory. Partial explanations are provided, again using results in compressive sensing theory. In my personal opinion, the explanations are opaque and unsatisfactory. An alternative route is suggested in my detailed review.
Finally, extensive experiments are conducted and they are in accordance with the theory.
My most criticism regarding this paper is the narrow scope on compressive sensing, and this really undermines the potential contribution in Section 5.
Specifically, the authors considered only Basis Pursuit estimators for sparse recovery, and they used the RIP of design matrices as the main tool to argue what is explainable by compressive sensing and what is not. This seems to be somewhat of a tunnel-visioning for me: There are a variety of estimators in sparse recovery problems, and there are much less restrictive conditions than RIP of the design matrices that guarantee perfect recovery.
In particular, in Section 5, instead of invoking [Donoho&Tanner 2005], I believe that a more plausible approach is through [Chandrasekaran et al. 2012]. There, a simple deterministic condition (the null space property) for successful recovery is proved. It would be of direct interest to check whether such condition holds for a pretrained embedding (say GloVe) given some BoWs. Furthermore, it is proved in the same paper that Restricted Strong Convexity (RSC) alone is enough to guarantee successful recovery; RIP is not required at all. While, as the authors argued in Section 5.2, it is easy to see that pretrained embeddings can never possess RIP, they do not rule out the possibility of RSC.
Exactly the same comments above apply to many other common estimators (lasso, Dantzig selector, etc.) in compressive sensing which might be more tolerant to noise.
Several minor comments:
1. Please avoid the use of “information theory”, especially “classical information theory”, in the current context. These words should be reserved to studies of Channel Capacity/Source Coding `a la Shannon. I understand that in recent years people are expanding the realm of information theory, but as compressive sensing is a fascinating field that deserves its own name, there’s no need to mention information theory here.
2. In Theorem 4.1, please be specific about how the l2-regularization is chosen.
3. In Section 4.1, please briefly describe why you need to extend previous analysis to the Lipschitz case. I understood the necessity only through reading proofs.
4. Can the authors briefly comment on the two assumptions in Section 4, especially the second one (on n- cooccurrence)? Is this practical?
5. Page 1, there is a typo in the sentence preceding [Radfors et al., 2017].
6. Page 2, first paragraph of related work, the sentence “Our method also closely related to ...” is incomplete.
7. Page 2, second paragraph of related work, “Pagliardini also introduceD a linear ...”
8. Page 9, conclusion, the beginning sentence of the second paragraph is erroneous.
[1] Venkat Chandrasekaran, Benjamin Recht, Pablo A. Parrilo, Alan S. Willsky, “The Convex Geometry of Linear Inverse Problems”, Foundations of Computational Mathematics, 2012.


Previous work has demonstrated the benefits of incorporating additional linguistic annotations such as syntactic trees into neural machine translation. However the cost of obtaining those syntactic annotations is expensive for many languages and the quality of unsupervised learning linguistic structures is too poor to be helpful. In this work, we aim to improve neural machine translation via source side dependency syntax but without explicit annotation. We propose a set of models that learn to induce dependency trees on the source side and learn to use that information on the target side. Importantly, we also show that our dependency trees capture important syntactic features of language and improve translation quality on two language pairs En-De and En-Ru.

This paper describes a method to induce source-side dependency structures in service to neural machine translation. The idea of learning soft dependency arcs in tandem with an NMT objective is very similar to recent notions of self-attention (Vaswani et al., 2017, cited) or previous work on latent graph parsing for NMT (Hashimoto and Tsuruoka, 2017, cited). This paper introduces three innovations: (1) they pass the self-attention scores through a matrix-tree theorem transformation to produce marginals over tree-constrained head probabilities; (2) they explicitly specify how the dependencies are to be used, meaning that rather than simply attending over dependency representations with a separate attention, they select a soft word to attend to through the traditional method, and then attend to that word’s soft head (called Shared Attention in the paper); and (3) they gate when attention is used. I feel that the first two ideas are particularly interesting. Unfortunately, the results of the NMT experiments are not particularly compelling, with overall gains over baseline NMT being between 0.6 and 0.8 BLEU. However, they include a useful ablation study that shows fairly clearly that both ideas (1) and (2) contribute equally to their modest gains, and that without them (FA-NMT Shared=No in Table 2), there would be almost no gains at all. Interesting side-experiments investigate their accuracy as a dependency parser, with and without a hard constraint on the system’s latent dependency decisions.
This paper has some very good ideas, and asks questions that are very much worth asking. In particular, the question of whether a tree constraint is useful in self-attention is very worthwhile. Unfortunately, this is mostly a negative result, with gains over “flat attention” being relatively small. I also like the “Shared Attention” - it makes a lot of sense to say that if the “semantic” attention mechanism has picked a particular word, one should also attend to that word’s head; it is not something I would have thought of on my own. The paper is also marred by somewhat weak writing, with a number of disfluencies and awkward phrasings making it somewhat difficult to follow.
In terms of specific criticisms:
I found the motivation section to be somewhat weak. We need a better reason than morphology to want to do source-side dependency parsing. All published error analyses of strong NMT systems (Bentivogli et al, EMNLP 2016; Toral and Sanchez-Cartagena, EACL 2017; Isabelle et al, EMNLP 2017) have shown that morphology is a strength, not a weakness of these systems, and the sorts of head selection problems shown in Figure 1 are, in my experience, handled capably by existing LSTM-based systems.
The paper mentions “significant improvements” in only two places: the introduction and the conclusion. With BLEU score differences being so low, the authors should specify how statistical significance is measured; ideally using a technique that accounts for the variance of random restarts (i.e.: Clark et al, ACL 2011).
Equation (3): I couldn’t find the definition for H anywhere.
Sentence before Equation (5): I believe there is a typo here, “f takes z_i” should be “f takes u_t”.
First section of Section 3: please cite the previous work you are talking about in this sentence.
My understanding was that the dependency marginals in p(z_{i,j}=1|x,\phi) in Equation (11) are directly used as \beta_{i,j}. If I’m correct, that’s probably worth spelling out explicitly in Equation (11): \beta_{i,j} = p(z_{i,j}=1|x,\phi) = …
I don’t don’t feel like the clause between equations (17) and (18), “when sharing attention weights from the decoder with the encoder” is a good description of your clever “shared attention” idea. In general, I found this region of the paper, including these two equations and the text between them, very difficult to follow.
Section 4.4: It’s very very good that you compared to “flat attention”, but it’s too bad for everyone cheering for linguistically-informed syntax that the results weren’t better.
Table 5: I had a hard time understanding Table 5 and the corresponding discussion. What are “production percentages”?
Finally, it would have been interesting to include the FA system in the dependency accuracy experiment (Table 4), to see if it made a big difference there.


An objective of pro-activity in dialog systems is to enhance the usability of conversational agents by enabling them to initiate conversation on their own. While dialog systems have become increasingly popular during the last couple of years, current task oriented dialog systems are still mainly reactive and users tend to initiate conversations. In this paper, we propose to introduce the paradigm of contextual bandits as framework for pro-active dialog systems. Contextual bandits have been the model of choice for the problem of reward maximization with partial feedback since they fit well to the task description. As a second contribution, we introduce and explore the notion of memory into this paradigm. We propose two differentiable memory models that act as parts of the parametric reward estimation function. The first one, Convolutional Selective Memory Networks, uses a selection of past interactions as part of the decision support. The second model, called Contextual Attentive Memory Network, implements a differentiable attention mechanism over the past interactions of the agent. The goal is to generalize the classic model of contextual bandits to settings where temporal information needs to be incorporated and leveraged in a learnable manner. Finally, we illustrate the usability and performance of our model for building a pro-active mobile assistant through an extensive set of experiments.

The paper "CONTEXTUAL MEMORY BANDIT FOR PRO-ACTIVE DIALOG ENGAGEMENT" proposes to address the problem of pro-active dialog engagement by the mean of a bandit framework that selects dialog situations w.r.t. to the context of the system. Authors define a neural archiecture managing memory with the mean of a contextual attention mechanism.
My main concern about this paper is that the proposal is not enough well described. A very large amount of technical details are missing for allowing the reader to understand the model (and reproduce the experiments). The most important ones are about the exploration policies which are not described at all, while it is a very central point of the paper. The only discussion given w.r.t. the exploration policy is a very general overview about Thompson Sampling. But nothing is said about how it is implemented in the case of the proposed model. How is estimated p(\Theta|D) ? Ok  authors give p(\Theta|D) as a product between prior and likelihood. But it is not sufficient to get p(\Theta|D), the evidence should also been considered (for instance by using variational inference). Also, what is the prior of the parameters ? How is distributed r given a,x and \Theta ?
Also, not enough justification is given about the general idea of the model. Authors should give more intuitions about the mechanism they propose. Figure 2 should be able to help, but no reference to this figure is given in the text, so it is very difficult to extract any information from it. Authors only (roughly) describe  the architecture without justifying their choices.
At last, the experiments really fail at demonstrating the relevance of the approach, as only questionable artificial data is used. On the first hand it appears mandatory to me to consider some (even minimal) experiments on real data for such proposal. On the other one, the simulated data used there cannot correspond to cues to validate the approach since they appear very far from real scenarios: the trajectories do not depend on what is recommended. Ok only the recommended places reveal some reward but it appears not as a sufficiently realistic scenario to me. Also, very too few baselines are considered: only different versions of the proposal and a random baseline are considered. A classical contextual bandit instance (such as LinUCB) would have been a minimum.
Other remarks:
      - the definition of q is not given
      - user is part of the context x  in the bandit section but not after where it is denoted as u.
      - the notion of time window should be more formally described
      - How is built the context is not clear in the experiments section


ON THE STATE OF THE ART OF EVALUATION IN NEURAL LANGUAGE MODELS
Ongoing innovations in recurrent neural network architectures have provided a steady influx of apparently state-of-the-art results on language modelling benchmarks. However, these have been evaluated using differing codebases and limited computational resources, which represent uncontrolled sources of experimental variation. We reevaluate several popular architectures and regularisation methods with large-scale automatic black-box hyperparameter tuning and arrive at the somewhat surprising conclusion that standard LSTM architectures, when properly regularised, outperform more recent models. We establish a new state of the art on the Penn Treebank and Wikitext-2 corpora, as well as strong baselines on the Hutter Prize dataset.

The authors perform a comprehensive validation of LSTM-based word and character language models, establishing that recent claims that other structures can consistently outperform the older stacked LSTM architecture result from failure to fully explore the hyperparameter space. Instead, with more thorough hyperparameter search, LSTMs are found to achieve state-of-the-art results on many of these language modeling tasks.
This is a significant result in language modeling and a milestone in deep learning reproducibility research. The paper is clearly motivated and authoritative in its conclusions but it's somewhat lacking in detailed model or experiment descriptions.
Some further points:
- There are several hyperparameters set to the "standard" or "default" value, like Adam's beta parameter and the batch size/BPTT length. Even if it would be prohibitive to include them in the overall hyperparameter search, the community is curious about their effect and it would be interesting to hear if the authors' experience suggests that these choices are indeed reasonably well-justified.
- The description of the model is ambiguous on at least two points. First, it wasn't completely clear to me what the down-projection is (if it's simply projecting down from the LSTM hidden size to the embedding size, it wouldn't represent a hyperparameter the tuner can set, so I'm assuming it's separate and prior to the conventional output projection). Second, the phrase "additive skip connections combining outputs of all layers" has a couple possible interpretations (e.g., skip connections that jump from each layer to the last layer or (my assumption) skip connections between every pair of layers?).
- Fully evaluating the "claims of Collins et al. (2016), that capacities of various cells are very similar and their apparent
differences result from trainability and regularisation" would likely involve adding a fourth cell to the hyperparameter sweep, one whose design is more arbitrary and is neither the result of human nor machine optimization.
- The reformulation of the problem of deciding embedding and hidden sizes into one of allocating a fixed parameter budget towards the embedding and recurrent layers represents a significant conceptual step forward in understanding the causes of variation in model performance.
- The plot in Figure 2 is clear and persuasive, but for reproducibility purposes it would also be nice to see an example set of strong hyperparameters in a table. The history of hyperparameter proposals and their perplexities would also make for a fantastic dataset for exploring the structure of RNN hyperparameter spaces. For instance, it would be helpful for future work to know which hyperparameters' effects are most nearly independent of other hyperparameters.
- The choice between tied and clipped (Sak et al., 2014) LSTM gates, and their comparison to standard untied LSTM gates, is discussed only minimally, although it represents a significant difference between this paper and the most "standard" or "conventional" LSTM implementation (e.g., as provided in optimized GPU libraries). In addition to further discussion on this point, this result also suggests evaluating other recently proposed "minor changes" to the LSTM architecture such as multiplicative LSTM (Krause et al., 2016)
- It would also have been nice to see a comparison between the variational/recurrent dropout parameterization "in which there is further sharing of masks between gates" and the one with "independent noise for the gates," as described in the footnote. There has been some confusion in the literature as to which of these parameterizations is better or more standard; simply justifying the choice of parameterization a little more would also help.


Experimental evidence indicates that simple models outperform complex deep networks on many unsupervised similarity tasks. Introducing the concept of an optimal representation space, we provide a simple theoretical resolution to this apparent paradox. In addition, we present a straightforward procedure that, without any retraining or architectural modifications, allows deep recurrent models to perform equally well (and sometimes better) when compared to shallow models. To validate our analysis, we conduct a set of consistent empirical evaluations and introduce several new sentence embedding models in the process. Even though this work is presented within the context of natural language processing, the insights are readily applicable to other domains that rely on distributed representations for transfer tasks.

------ updates to review: ---------
I think the paper is much improved. It is much more clear and the experiments are more focused and more closely connected to the earlier content in the paper. Thanks to the authors for trying to address all of my concerns.
I now better understand in what sense the representation space is optimal.  I had been thinking (or perhaps "hoping" is a better word) that the term "optimal" implied maximal in terms of some quantifiable measure, but it's more of an empirical "optimality".  This makes the paper an empirical paper based on a reasonable and sensible intuition, rather than a theoretical result.  This was a little disappointing to me, but I do still think the paper is marginally above the acceptance threshold and have increased my score accordingly.
------ original review below: --------
This paper is about rethinking how to use encoder-decoder architectures for representation learning when the training objective contains a similarity between the decoder output and the encoding of something else. For example, for the skip-thought RNN encoder-decoder that encodes a sentence and decodes neighboring sentences: rather than use the final encoder hidden state as the representation of the sentence, the paper uses some function of the decoder, since the training objective is to maximize each dot product between a decoder hidden state and the embedding of a context word. If dot product (or cosine similarity) is going to be used as the similarity function for the representation, then it makes more sense, the paper argues, to use the decoder hidden state(s) as the representation of the input sentence. The paper considers both averaging and concatenating hidden states. One difficulty here is that the neighboring sentences are typically not available in downstream tasks, so the paper runs the decoder to produce a predicted sentence one word-at-a-time, using the predicted words as inputs to the decoder RNNs. Then those decoder RNN hidden states are used via averaging or concatenation as the representation of a sentence in downstream tasks.
This paper is a source of contributions, but I think in its current form it is not yet ready for publication.
Pros:
I think it makes sense to pay attention to the training objective when deciding how to use the model for downstream tasks.
I like the empirical investigation of combining RNN and BOW encoders and decoders.
The experimental results show that a single encoder-decoder model can be trained and then two different functions of it can be used at test time for different kinds of tasks (RNN-RNN for supervised transfer and RNN-RNN-mean for unsupervised transfer). I think this is an interesting result.
Cons:
I have several concerns. The first relate to the theoretical arguments and their empirical support.
Regarding the theoretical arguments:
First, the paper discusses the notion of an "optimal representation space" and describes the argument as theoretical, but I don't see much of a theoretical argument here.
As far as I can tell, the paper does not formally define its terms or define in what sense the representation space is "optimal". I can only find heuristic statements like those in the paragraph in Sec 3.2 that begins "These observations...".  What exactly is meant formally by statements like "any model where the decoder is log-linear with respect to the encoder" or "that distance is optimal with respect to the model’s objective"?  It seems like the paper may want to start with formal definitions of an encoder and a decoder, then define what is meant by a "decoder that is log-linear with respect to the encoder", and define what it means for a distance to be optimal with respect to a training objective. That seems necessary in order to provide the foundation to make any theoretical statement about choices for encoders, decoders, and training objectives.  I am still not exactly sure what that theoretical statement might look like, but maybe defining the terms would help the authors get started in heading toward the goal of defining a statement to prove.
Second, the paper's theoretical story seems to diverge almost immediately from the choices used in the model and experimental procedure.
For example, in Sec. 3.2, it is stated that cosine similarity "is the appropriate similarity measure in the case of log-linear decoders."  But the associated footnote (footnote 2) seems to admit a contradiction here by noting that actually the appropriate similarity measure is dot product: "Evidently, the correct measure is actually the dot product."  This is a bit confusing.
It also raises a question: If cosine similarity will be used later for computing similarity, then why not try using cosine similarity in place of dot product in the model?  That is, replace "u_w \cdot h_i" in Eq. (2) with "cos(u_w, h_i)".  If the paper's story is correct (and if I understand the ideas correctly), training with cosine similarity should work better than training with dot product, because the similarity function used during training is more similar to that used in testing.  This seems like a natural experiment to try.  Other natural experiments would be to vary both the similarity function used in the model during training and the similarity function used at test time.  The authors' claims could be validated if the optimal choices always use the same choice for the training and test-time similarity functions.  That is, if Euclidean distance is used during training, then will Euclidean distance be the best choice at test time?
Another example of the divergence lies in the use of the skip-thought decoder on downstream tasks. Since the decoder hidden states depend on neighboring sentences and these are considered to be unavailable at test time, the paper "unrolls" the decoder for several steps by using it to predict words which are then used as inputs on the next time step. To me, this is a potentially very significant difference between training and testing. Since much of the paper is about reconciling training and testing conditions in terms of the representation space and similarity function, this difference feels like a divergence from the theoretical story. It is only briefly mentioned at the end of Sec. 3.3 and then discussed again later in the experiments section. I think this should be described in more detail in Section 3.3 because it is an important note about how the model will be used in practice.
It would be nice to be able to quantify the impact (of unrolling the decoder with predicted words) by, for example, using the decoder on a downstream evaluation dataset that has neighboring sentences in it. Then the actual neighboring sentences can be used as inputs to the decoder when it is unrolled, which would be closer to the training conditions and we could empirically see the difference. Perhaps there is an evaluation dataset with ordered sentences so that the authors could empirically compare using real vs predicted inputs to the decoder on a downstream task?
The above experiments might help to better connect the experiments section with the theoretical arguments.
Other concerns, including more specific points, are below:
Sec. 2:
When describing inferior performance of RNN-based models on unsupervised sentence similarity tasks, the paper states: "While this shortcoming of SkipThought and RNN-based models in general has been pointed out, to the best of our knowledge, it has never been systematically addressed in the literature before."
The authors may want to check Wieting & Gimpel (2017) (and its related work) which investigates the inferiority of LSTMs compared to word averaging for unsupervised sentence similarity tasks. They found that averaging the encoder hidden states can work better than using the final encoder hidden state; the authors may want to try that as well.
Sec. 3.2:
When describing FastSent, the paper includes "Due to the model's simplicity, it is particularly fast to train and evaluate, yet has shown state-of-the-art performance in unsupervised similarity tasks (Hill et al., 2015)."
I don't think it makes much sense to cite the SimLex-999 paper in this context, as that is a word similarity task and that paper does not include any results of FastSent. Maybe the Hill et al (2016) FastSent citation was meant instead? But in that case, I don't think it is quite accurate to make the claim that FastSent is SOTA on unsupervised similarity tasks. In the original FastSent paper (Hill et al., 2016), FastSent is not as good as CPHRASE or "DictRep BOW+embs" on average across the unsupervised sentence similarity evaluations. FastSent is also not as good as sent2vec from Pagliardini et al (2017) or charagram-phrase from Wieting et al. (2016).
Sec. 3.3:
In describing skip-thought, the paper states: "While computationally complex, it is currently the state-of-the-art model for supervised transfer tasks (Hill et al., 2016)."
I don't think it is accurate to state that skip-thought is still state-of-the-art for supervised transfer tasks, in light of recent work (Conneau et al., 2017; Gan et al., 2017).
Sec. 3.3:
When discussing averaging the decoder hidden states, the paper states: "Intuitively, this corresponds to destroying the word order information the decoder has learned." I'm not sure this strong language can be justified here. Is there any evidence to suggest that averaging the decoder hidden states will destroy word order information? The hidden states may be representing word order information in a way that is robust to averaging, i.e., in a way such that the average of the hidden states can still lead to the reconstruction of the word order.
Sec. 4:
What does it mean to use an RNN encoder and a BOW decoder?  This seems to be a strongly-performing setting and competitive with RNN-mean, but I don't know exactly what this means.
Minor things:
Sec. 3.1:
When defining v_w, it would be helpful to make explicit that it's in \mathbb{R}^d.
Sec. 4:
For TREC question type classification, I think the correct citation should be Li & Roth (2002) instead of Vorhees (2002).
Sec. 5:
I think there's a typo in the following sentence: "Our results show that, for example, the raw encoder output for SkipThought (RNN-RNN) achieves strong performance on supervised transfer, whilst its mean decoder output (RNN-mean) achieves strong performance on supervised transfer."  I think "unsupervised" was meant in the latter mention.
References:
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017). Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. EMNLP.
Gan, Z., Pu, Y., Henao, R., Li, C., He, X., & Carin, L. (2017). Learning generic sentence representations using convolutional neural networks. EMNLP.
Li, X., & Roth, D. (2002). Learning question classifiers. COLING.
Pagliardini, M., Gupta, P., & Jaggi, M. (2017). Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features. arXiv preprint arXiv:1703.02507.
Wieting, J., Bansal, M., Gimpel, K., & Livescu, K. (2016). Charagram: Embedding words and sentences via character n-grams. EMNLP.
Wieting, J., & Gimpel, K. (2017). Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings. ACL.