POSTERIOR ATTENTION MODELS FOR SEQUENCE TO SEQUENCE LEARNING
Modern neural architectures critically rely on attention for mapping structured inputs to sequences. In this paper we show that prevalent attention architectures do not adequately model the dependence among the attention and output tokens across a predicted sequence. We present an alternative architecture called Posterior Attention Models that after a principled factorization of the full joint distribution of the attention and output variables, proposes two major changes. First, the position where attention is marginalized is changed from the input to the output. Second, the attention propagated to the next decoding stage is a posterior attention distribution conditioned on the output. Empirically on five translation and two morphological inflection tasks the proposed posterior attention models yield better BLEU score and alignment accuracy than existing attention models.
This paper proposes a new sequence to sequence model where attention is treated as a latent variable, and derive novel inference procedures for this model. The approach obtains significant improvements in machine translation and morphological inflection generation tasks. An approximation is also used to make hard attention more efficient by reducing the number of softmaxes that have to be computed.  
Strengths:
- Novel, principled sequence to sequence model.
- Strong experimental results in machine translation and morphological inflection.
Weaknesses:
- Connections can be made with previous closely related architectures.
- Further ablation experiments could be included. 
The derivation of the model would be more clear if it is first derived without attention feeding: The assumption that output is dependent only on the current attention variable is then valid. The Markov assumption on the attention variable should also be stated as an assumption, rather than an approximation: Given that assumption, as far as I can tell the (posterior) inference procedure that is derived is exact: It is indeed equivalent to the using the forward computation of the classic forward-backward algorithm for HMMs to do inference. 
The model’s overall distribution can then be defined in a somewhat different way than the authors’ presentation, which I think makes more clear what the model is doing:
p(y | x) = \sum_a \prod_{t=1}^n p(y_t | y_{<t}, x, a_t) p(a_t | y_{<t}, x_ a_{t-1}).  
The equations derived in the paper for computing the prior and posterior attention is then just a dynamic program for computing this distribution, and is equivalent to using the forward algorithm, which in this context is:
 \alpha_t(a) = p(a_t = a, y_{<=t}) = p(y_t | s_t, a_t =a) \sum_{a’} \alpha_{t-1}(a’) p(a_t = a | s_t, a_{t-1} = a’) 
The only substantial difference in the inference procedure is then that the posterior attention probability is fed into the decoder RNN, which means that the independence assumptions are not strictly valid any more, even though the structural assumptions are still encoded through the way inference is done. 
[1] recently proposed a model with a similar factorization, although that model did not feed the attention distribution, and performed EM-like inference with the forward-backward algorithm, while this model is effectively computing forward probabilities and performing inference through automatic differentiation.
The Prior-Joint variant, though its definition is not as clear as it should be, seems to be assuming that the attention distribution at each time step is independent of the previous attention (similar to the way standard soft attention is computed) - the equations then reduce to a (neural) version of IBM alignment model 1, similar to another recently proposed model [2]. These papers can be seen as concurrent work, and this paper provides important insights, but it would strengthen rather than weaken the paper to make these connections clear. 
The results clearly show the advantages of the proposed approach over soft and sparse attention baselines. However, the difference in BLEU score between the variants of the prior or posterior attention models is very small across all translation datasets, so to make claims about which of the variants are better, at a minimum statistical significance testing should be done. Given that the “Prior-Joint” model performs competitively, is it computationally more efficient that the full model? 
The main missing experiment is not doing attention feeding at all. The other experiment that is not included (as I understood it) is to compute prior and posterior attention, but feed the prior attention rather than the posterior attention. 
The paper is mostly written very clearly, there are just a few typos and grammatical errors in sections 4.2 and 4.3. 
Overall, I really like this paper and would like to see it accepted, although I hope that a revised version would make the assumptions the model is making clearer and make connections to related models clearer. 
 
[1] Neural Hidden Markov Model for Machine Translation, Wang et al, ACL 2018. 
[2] Hard Non-Monotonic Attention for Character-Level Transduction, Wu, Shapiro and Cotterell, EMNLP 2018.