Published as a conference paper at ICLR 2018 LEARNING GENERAL PURPOSE DISTRIBUTED SEN- TENCE REPRESENTATIONS VIA LARGE SCALE MULTI- TASK LEARNING
A lot of the recent success in natural language processing (NLP) has been driven by distributed vector representations of words trained on large amounts of text in an unsupervised manner. These representations are typically used as general purpose features for words across a range of NLP problems. However, extending this success to learning representations of sequences of words, such as sentences, remains an open problem. Recent work has explored unsupervised as well as supervised learning techniques with different training objectives to learn general purpose fixed-length sentence representations. In this work, we present a simple, effective multi-task learning framework for sentence representations that combines the inductive biases of diverse training objectives in a single model. We train this model on several data sources with multiple training objectives on over 100 million sentences. Extensive experiments demonstrate that sharing a single recurrent sentence encoder across weakly related tasks leads to consistent improvements over previous methods. We present substantial improvements in the context of transfer learning and low-resource settings using our learned general-purpose representations.
---- updates: ----
I had a ton of comments and concerns, and I think the authors did an admirable job in addressing them.  I think the paper represents a solid empirical contribution to this area and is worth publishing in ICLR. 
---- original review follows: ----
This paper is about learning sentence embeddings by combining a bunch of training signals: predicting the next & previous sentences (skip-thought), predicting the sentence's translation, classifying entailment relationships between two sentences, and predicting the constituent parse of a sentence. This is a simple idea that combines a bunch of things from prior work into one framework and yields strong results, outperforming most prior work on most tasks. 
I think this paper is impressive in how it scales up training to use so many tasks and such large training sets for each task.  That and its strong experimental results make it worthy of publication. It's not very surprising that adding more tasks and data improves performance on average across downstream tasks, but it is nice to see the experimental results in detail. While many people would think of this idea, few would have the resources and expertise necessary to do it justice.  I also like how the authors move beyond the standard sentence tasks to evaluate also on the Quora question duplicate task with different amounts of training data and also consider the sentence characteristic / syntactic property tasks.  It would be great if the authors could release their pretrained sentence representation model so that other researchers could use it. 
I do have some nitpicks here and there with the presentation and exposition, and I am concerned that at times the paper appears to be minimizing its weaknesses, but I think these are things that can be addressed in the next revision. I understand that sometimes it's tempting to minimize one's weaknesses in order to get a paper accepted because the reviewers may not understand the area very well and may get hung up on the wrong things. I understand the area well and so all the feedback I offer below comes from a place of desiring this paper's publication while also desiring it to be as accurate and helpful for the community as possible. 
Below I'll discuss my concerns with the experiments and description of the results.
Regarding the results in Table 2:
The results in Table 2 seem a little bit unstable, as it is unclear which setting to use for the classification tasks; maybe it depends on the kind of classification being performed. One model seems best for the sentiment tasks ("+2L +STP") while other models seem best for SUBJ and MPQA.  Adding parsing as a training task hurts performance on the sentence classification tasks while helping performance on the semantic tasks, as the authors note.  It is unclear which is the best general model.  In particular, when others write papers comparing to the results in this paper, which setting should they compare to?  It would be nice if the authors could discuss this. 
The results reported for the CNN-LSTM of Gan et al. do not exactly match those of any single row from Gan et al, either v1 or v2 on arxiv or the published EMNLP version. How were those specific numbers selected? 
The caption of Table 2 states "All results except ours are taken from Conneau et al. (2017)." However, Conneau et al (neither the latest arxiv version nor the published EMNLP version) does not include many of the results in the table, such as CNN-LSTM and DiscSent mentioned in the following sentence in the caption. Did the authors replicate the results of those methods themselves, or report them from other papers?
What does bold and underlining indicate in Table 2?  I couldn't find this explained anywhere. 
At the bottom of Table 2, in the section with approaches trained from scratch on these tasks, I'd suggest including the 89.7 SST result of Munkhdalai and Yu (2017) and the 96.1 TREC result of Zhou et al. (2016) (as well as potentially other results from Zhou et al, since they report results on others of these datasets). The reason this is important is because readers may observe that the paper's new method achieves higher accuracies on SST and TREC than all other reported results and mistakenly think that the new method is SOTA on those tasks.  I'd also suggest adding the results from Radford et al. (2017) who report 86.9 on MR and 91.4 on CR.  For other results on these datasets, including stronger results in non-fixed-dimensional-sentence-embedding transfer settings, see results and references in McCann et al. (2017).  While the methods presented in this paper are better than prior work in learning general purpose, fixed-dimensional sentence embeddings, they still do not produce state-of-the-art results on that many of these tasks, if any.  I think this is important to note. 
For all tasks for which there is additional training, there's a confound due to the dimensionality of the sentence embeddings across papers. Using higher-dimensional sentence embeddings leads to more parameters in the linear model being trained on the task data. So it is unclear if the increase in hidden units in rows with "+L" is improving the results because of providing more weights for the linear model or whether it is learning a better sentence representation. 
The main sentence embedding results are in Table 2, and use the SentEval framework. However, not all tasks are included. The STS Benchmark results are included, which use an additional layer trained on the STS Benchmark training data just like the SICK tasks. But the other STS results, which use cosine similarity on the embedding space directly without any retraining, are only included in the appendix (in Table 7). The new approach does not do very well on those unsupervised tasks. On two years of data it is better than InferSent and on two years it is worse. Both are always worse than the charagram-phrase results of Wieting et al (2016a), which has 66.1 on 2012, 57.2 on 2013, 74.7 on 2014, and 76.1 on 2015.  Charagram-phrase trains on automatically-generated paraphrase phrase pairs, but these are generated automatically from parallel text, the same type of resource used in the "+Fr" and "+De" models proposed in this submission, so I think it should be considered as a comparable model. 
The results in the bottom section of Table 7, reported from Arora et al (2016), were in turn copied from Wieting et al (2016b), so I think it would make sense to also cite Wieting et al (2016b) if those results are to be included. Also, it doesn't seem appropriate to designate those as "Supervised Approaches" as they only require parallel text, which is a subset of the resources required by the new model. 
There are some other details in the appendix that I find concerning:
Section 8 describes how there is some task-specific tuning of which function to compute on the encoder to produce the sentence representation for the task.  This means that part of the improvement over prior work (especially skip-thought and InferSent) is likely due to this additional tuning. So I suppose to use these sentence representations in other tasks, this same kind of tuning would have to be done on a validation set for each task?  Doesn't that slightly weaken the point about having "general purpose" sentence representations?
Section 9 provides details about how the representations are created for different training settings. I am confused by the language here. For example, the first setting ("+STN +Fr +De") is described as "A concatenation of the representations trained on these tasks with a unidirectional and bidirectional GRU with 1500 hidden units each." I'm not able to parse this. I think the authors mean "The sentence representation h_x is the concatenation of the final hidden vectors from a forward GRU (with 1500-dimensional hidden vectors) and a bidirectional GRU (also with 1500-dimensional hidden vectors)". Is this correct? 
Also in Sec 9: I found it surprising how each setting that adds a training task uses the concatenation of a representation with that task and one without that task. What is the motivation for doing this? This seems to me to be an important point that should be discussed in Section 3 or 4. And when doing this, are the concatenated representations always trained jointly from scratch with the special task only updating a subset of the parameters, or do you use the fixed pretrained sentence representation from the previous row and just concatenate it with the new one?  To be more concrete, if I want to get the encoder for the second setting ("+STN +Fr +De +NLI"), do I have to train two times or can I just train once?  That is, the train-once setting would correspond to only updating the NLI-specific representation parameters when training on NLI data; on other data, all parameters would be updated. The train-twice setting would first train a representation on "+STN +Fr +De", then set it aside, then train a separate representation on "+STN +Fr +De +NLI", then finally concatenate the two representations as my sentence representation.  Do you use train-once or train-twice? 
Regarding the results in Table 3:
What do bold and underline indicate?
What are the embeddings corresponding to the row labeled "Multilingual"?
In the caption, I can't find footnote 4. 
The caption includes the sentence "our embeddings have 1040 pairs out of 2034 for which atleast one of the words is OOV, so a comparison with other embeddings isn't fair on RW."  How were those pairs handled?  If they were excluded, then I think the authors should not report results on RW.  I suspect that most of the embeddings included in the table also have many OOVs in the RW dataset but still compute results on it using either an unknown word embedding or some baseline similarity of zero for pairs with an OOV. I think the authors should find some way (like one of those mentioned, or some other way) of computing similarity of those pairs with OOVs. It doesn't make much sense to me to omit pairs with OOVs. 
There are much better embeddings on SimLex than the embeddings whose results are reported in the table. Wieting et al. (2016a) report SimLex correlation of 0.706 and Mrkšić et al. (2017) report 0.751.  I'd suggest adding the results of some stronger embeddings to better contextualize the embeddings obtained by the new method.  Some readers may mistakenly think that the embeddings are SOTA on SimLex since no stronger results are provided in the table. 
The points below are more minor/specific:
Sec. 2:
In Sec. 2, the paper discusses its focus on fixed-length sentence representations to distinguish itself from other work that produces sentence representations that are not fixed-length. I feel the motivation for this is lacking. Why should we prefer a fixed-length representation of a sentence? For certain downstream applications, it might actually be easier for practitioners to use a representation that provides a representation for each position in a sentence (Melamud et al., 2016; Peters et al., 2017; McCann et al., 2017) rather than an opaque sentence representation. Some might argue that since sentences have different lengths, it would be appropriate for a sentence representation to have a length proportional to the length of the sentence.  I would suggest adding some motivation for the focus on fixed-length representations. 
Sec. 4.1:
"We take a simpler approach and pick a new task to train on after every parameter update sampled uniformly. An NLI minibatch is interspersed after every ten parameter updates on sequence-to-sequence tasks"
These two sentences seem contradictory. Maybe in the first sentence "pick a new task" should be changed to "pick a new sequence-to-sequence task"?
Sec. 5.1:
typo: "updating the parameters our sentence" --> "updating the parameters of our sentence"
Sec. 5.2:
typo in Table 4 caption: "and The" --> ". The"
typo: "parsing improvements performance" --> "parsing improves performance"
In general, there are many missing citations for the tasks, datasets, and prior work on them. I understand that the authors are pasting in numbers from many places and just providing pointers to papers that provide more citation info, but I think this can lead to mis-attribution of methods. I would suggest including citations for all datasets/tasks and methods whose results are being reported. 
References:
McCann, Bryan, James Bradbury, Caiming Xiong, and Richard Socher. "Learned in translation: Contextualized word vectors." CoRR 2017.
Melamud, Oren, Jacob Goldberger, and Ido Dagan. "context2vec: Learning Generic Context Embedding with Bidirectional LSTM." CoNLL 2016.
Mrkšić, Nikola, Ivan Vulić, Diarmuid Ó. Séaghdha, Ira Leviant, Roi Reichart, Milica Gašić, Anna Korhonen, and Steve Young. "Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints." TACL 2017.
Munkhdalai, Tsendsuren, and Hong Yu. "Neural semantic encoders." EACL 2017. 
Pagliardini, Matteo, Prakhar Gupta, and Martin Jaggi. "Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features." arXiv preprint arXiv:1703.02507 (2017).
Peters, Matthew E., Waleed Ammar, Chandra Bhagavatula, and Russell Power. "Semi-supervised sequence tagging with bidirectional language models." ACL 2017.
Radford, Alec, Rafal Jozefowicz, and Ilya Sutskever. "Learning to generate reviews and discovering sentiment." arXiv preprint arXiv:1704.01444 2017.
Wieting, John, Mohit Bansal, Kevin Gimpel, and Karen Livescu. "Charagram: Embedding words and sentences via character n-grams." EMNLP 2016a.
Wieting, John, Mohit Bansal, Kevin Gimpel, and Karen Livescu. "Towards universal paraphrastic sentence embeddings." ICLR 2016b.
Zhou, Peng, Zhenyu Qi, Suncong Zheng, Jiaming Xu, Hongyun Bao, and Bo Xu. "Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling." COLING 2016.