NEURAL SPEED READING WITH STRUCTURAL-JUMP- LSTM
Recurrent neural networks (RNNs) can model natural language by sequentially "reading" input tokens and outputting a distributed representation of each token. Due to the sequential nature of RNNs, inference time is linearly dependent on the input length, and all inputs are read regardless of their importance. Efforts to speed up this inference, known as "neural speed reading", either ignore or skim over part of the input. We present Structural-Jump-LSTM: the first neural speed reading model to both skip and jump text during inference. The model consists of a standard LSTM and two agents: one capable of skipping single words when reading, and one capable of exploiting punctuation structure (sub-sentence separators (,:), sentence end symbols (.!?), or end of text markers) to jump ahead after reading a word. A comprehensive experimental evaluation of our model against all five state-of-the-art neural reading models shows that Structural-Jump-LSTM achieves the best overall floating point operations (FLOP) reduction (hence is faster), while keeping the same accuracy or even improving it compared to a vanilla LSTM that reads the whole text.
The paper presents a novel model for neural speed reading. In this new model, the authors combined several existing ideas in a nice way, namely, the new reader has the ability to skip a word or to jump a sequence of words at once. The reward of the reader is mixed of the final prediction correctness and the amount of text been skipped. The problem is formulated as a reinforcement learning problem. The results compared with the existing techniques on several benchmark datasets show consistently good improvements.
In my view, one important (also a little surprising) finding of the paper is that the reader can make jump choices successfully with the help of punctuations. And, blindly jumping a sequence of words without even lightly read them can still make very good predictions.
The basic idea of the paper, the concepts of skip and jump, and the reinforcement learning formulation are not completely new, but the paper combined them in an effective way. The results show good improvements majorly in FLOPS.
The way of defining state, rewards and value function are not very clear to me. Two value estimates are defined separately for the skip agent and the jump agent. Why not define a common value function for a shared state? Two values will double count the rewards from reading. Also, the state of the jump agent may not capture all available information. For example, how many words until the end of the sentence if you make a jump. Will this make the problem not a MDP? 
Overall, this is a good paper.
I read the authors' response. The paper should in its final version add the precise explanation of how the two states interact and how a joint state definition differs from the current one.