AN EFFICIENT FRAMEWORK FOR LEARNING SENTENCE REPRESENTATIONS
In this work we propose a simple and efficient framework for learning sentence representations from unlabelled data. Drawing inspiration from the distributional hypothesis and recent work on learning sentence representations, we reformulate the problem of predicting the context in which a sentence appears as a classification problem. Given a sentence and the context in which it appears, a classifier distinguishes context sentences from other contrastive sentences based on their vector representations. This allows us to efficiently learn different types of encoding functions, and we show that the model learns high-quality sentence representations. We demonstrate that our sentence representations outperform stateof-the-art unsupervised and supervised representation learning methods on several downstream NLP tasks that involve understanding sentence semantics while achieving an order of magnitude speedup in training time.
==Update==
I appreciate the response, and continue to recommend acceptance. The evaluation metric used in this paper (SentEval) represents an important open problem in NLP—learning reusable sentence representations—and one of the problems in NLP best suited to presentation at IC*LR*. Because of this, I'm willing to excuse the fact that the paper is only moderately novel, in light of the impressive reported results.
While I would appreciate a direct (same codebase, same data) comparison with some outside baselines, this paper meets or exceeds the standards for rigor that were established by previous published work in the area, and the existing results are sufficient to support some substantial conclusions.
==========
This paper proposes an alternative formulation of Kiros's SkipThought objective for training general-purpose sentence encoder RNNs on unlabeled data. This formulation replaces the decoder in that model with a second encoder, and yields substantial improvements to both speed and model performance (as measured on downstream transfer tasks). The resulting model is, for the first time, reasonably competitive even with models that are trained end-to-end on labeled data for the downstream tasks (despite the requirement, imposed by the evaluation procedure, that only the top layer classifier be trained for the downstream tasks here), and is also competitive with models trained on large labeled datasets like SNLI. The idea is reasonable, the topic is important, and the results are quite strong. I recommend acceptance, with some caveats that I hope can be addressed.
Concerns:
A nearly identical idea to the core idea of this paper was proposed in an arXiv paper this spring, as a commenter below pointed out. That work has been out for long enough that I'd urge you to cite it, but it was not published and it reports results that are far less impressive than yours, so that omission isn't a major problem.
I'd like to see more discussion of how you performed your evaluation on the downstream tasks. Did you use the SentEval tool from Conneau et al., as several related recent papers have? If not, does your evaluation procedure differ from theirs or Kiros's in any meaningful way?
I'm also a bit uncomfortable that the paper doesn't directly compare with any baselines that use the exact same codebase, word representations, hyperparameter tuning procedure, etc.. I would be more comfortable with the results if, for example, the authors compared a low-dimensional version of their model with a low-dimensional version of SkipThought, trained in the *exact* same way, or if they implemented the core of their model within the SkipThought codebase and showed strong results there.
Minor points:
The headers in Table 1 don't make it all that clear which additions (vectors, UMBC) are cumulative with what other additions. This should be an easy fix. 
The use of the check-mark as an output in Figure 1 doesn't make much sense, since the task is not binary classification.
"Instead of training a model to reconstruct the surface form of the input sentence or its neighbors, our formulation attempts to focus on the semantic aspects of sentences. The meaning of a sentence is the property that creates bonds between a sequence of sentences and makes it logically flow." – It's hard to pin down exactly what this means, but it sounds like you're making an empirical claim here: semantic information is more important than non-semantic sources of variation (syntactic/lexical/morphological factors) in predicting the flow of a text. Provide some evidence for this, or cut it.
You make a similar claim later in the same section: "In figure 1(a) however, the reconstruction loss forces the model to predict local structural information about target sentences that may be irrelevant to its meaning (e.g., is governed by grammar rules)." This is a testable prediction: Are purely grammatical (non-semantic) variations in sentence form helpful for your task? I'd suspect that they are, at least in some cases, as they might give you clues as to style, dialect, or framing choices that the author made when writing that specific passage.
"Our best BookCorpus model (MC-QT) trains in just under 11hrs, compared to skip-thought model’s training time of 2 weeks." –  If you say this, you need to offer evidence that your model is faster. If you don't use the same hardware and low-level software (i.e., CuDNN), this comparison tells us nearly nothing. The small-scale replication of SkipThought described above should address this issue, if performed.