Plan, Attend, Generate: Planning for Sequence-to-Sequence Models
We investigate the integration of a planning mechanism into sequence-to-sequence models using attention. We develop a model which can plan ahead in the future when it computes its alignments between input and output sequences, constructing a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan. This mechanism is inspired by the recently proposed strategic attentive reader and writer (STRAW) model for Reinforcement Learning. Our proposed model is end-to-end trainable using primarily differentiable operations. We show that it outperforms a strong baseline on character-level translation tasks from WMT'15, the algorithmic task of finding Eulerian circuits of graphs, and question generation from the text. Our analysis demonstrates that the model computes qualitatively intuitive alignments, converges faster than the baselines, and achieves superior performance with fewer parameters.
The paper introduces the planning mechanism into the encoder-decoder model with attention. The key idea is that instead of computing the attention weights on-the-fly for each decoding time step, an attention template is generated for the next k time steps based on the current decoding hidden state s_{t-1}, and a commitment vector is used to decide whether to follow this alignment plan or to recompute it. 
Overall, the paper is well written, and technical discussion are clear. The effect of the planning mechanism is demonstrated by Figure 2, where planning mechanism tends to generate smoother alignments compared to the vanilla attention approach. In this paper, the attention template (matrix A) is computed as eq(3). It would be interesting to see if some other type of features could be used, and if they could control the language generation, and produce more diverse and controllable responses. This might be evaluated on a conversation task, however, I understand that it is out of the scope of this paper.
In general, this is an interesting idea to me. A couple of minor questions:
-- In Table 1, why the Baseline + layer norm is missing? PAG and rPAG seem only outperform the baseline when layer norm is used.
--You used k=10 for all the experiments. Is there any motivation behind that? 10 time steps only correspond to 1 to 2 English words for character MT.
--Do you have any results with word-level MT? Does the planning approach help?
--Can you say anything about the computation complexity? Does it slow down training a lot?