Auto-regressive sequence-to-sequence models with attention mechanism have achieved state-of-the-art performance in many tasks such as machine translation and speech synthesis. These models can be difficult to train. The standard approach, teacher forcing, guides a model with reference output history during training. The problem is that the model is unlikely to recover from its mistakes during inference, where the reference output is replaced by generated output. Several approaches deal with this problem, largely by guiding the model with generated output history. To make training stable, these approaches often require a heuristic schedule or an auxiliary classifier. This paper introduces attention forcing, which guides the model with generated output history and reference attention. This approach can train the model to recover from its mistakes, in a stable fashion, without the need for a schedule or a classifier. In addition, it allows the model to generate output sequences aligned with the references, which can be important for cascaded systems like many speech synthesis systems. Experiments on speech synthesis show that attention forcing yields significant performance gain. Experiments on machine translation show that for tasks where various re-orderings of the output are valid, guiding the model with generated output history is challenging, while guiding the model with reference attention is beneficial.
This paper proposes an alternative mechanism of training the attention values of a sequence to sequence learning model as applied to tasks like speech synthesis and translation.  During training they compute two forms of attention: (1) the standard soft-attention from a decoder fed with teacher forced output, and (2) the inference-time attention from a decoder fed with predicted outputs.  Their training objective consists of two terms: The first is the token-wise cross entropy loss but by conditioning on the predicted output  but with teacher-forced attention.  The second is a KL distance between the above two types of attention distributions.   Experiments with mechanical  turks indicate that their attention forcing mechanism is strongly preferred over the existing teacher forced output and attention model.  On translation their method provides little or no improvement.
I am inclined towards rejecting the paper because the experiment and related work section still requires a lot of work before 1. The claimed utility of the idea is established, and 2. The novelty over the many existing attention architectures is established.   I elaborate on each of these next.
Related work: Recently, many papers have directly or indirectly handled the problem of exposure bias that this paper attempts to address.  The paper does not discuss most of these.  Here are some that are missed from the paper:
1.   Sequence level training with recurrent neural networks
MA Ranzato, S Chopra, M Auli, W Zaremba, 2015.
This paper shows that the scheduled sampling method (discussed in the paper) is much worse than a reinforce-based training mechanism of handling exposure bias.  
2. An actor-critic algorithm for sequence prediction
D Bahdanau, P Brakel, K Xu, A Goyal, R Lowe
3.  Posterior Attention Models for Sequence to Sequence Learning
S Shankar, S Sarawagi - 2019
4. Latent Alignment and Variational Attention
Yuntian Deng, Yoon Kim, Justin Chiu, Demi Guo, Alexander M. Rush 2018
5. Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings
Shaohui Kuang, Junhui Li, António Branco, Weihua Luo, Deyi Xiong
Experiments:  Their experiments are rather sketchy and limited.
The TTS experiments are only on one dataset.  Their method is compared only with the standard seq2seq learning approach.  Even the scheduled sampling or professor forcing methods are not compared with.  In addition, state of the art TTS methods have gained significantly from hierarchical attention.  As such as far as the TTS task is concerned the significance of the improved quality over a baseline seq2seq method is limited.
For translation they consider only the English-Vietnamese task whereas there are tens of other translation tasks that are used in recent literature.
Overall, the idea proposed seems quite incremental, experiments are limited, and related work discussion incomplete.
*********
I read the author response but I do not think the paper is ready for publication yet without the thorough comparison with related work.