Text generation is ubiquitous in many NLP tasks, from summarization, to dialogue and machine translation. The dominant parametric approach is based on locally normalized models which predict one word at a time. While these work remarkably well, they are plagued by exposure bias due to the greedy nature of the generation process. In this work, we investigate un-normalized energy-based models (EBMs) which operate not at the token but at the sequence level. In order to make training tractable, we first work in the residual of a pretrained locally normalized language model and second we train using noise contrastive estimation. Furthermore, since the EBM works at the sequence level, we can leverage pretrained bi-directional contextual representations, such as BERT and RoBERTa. Our experiments on two large language modeling datasets show that residual EBMs yield lower perplexity compared to locally normalized baselines. Moreover, generation via importance sampling is very efficient and of higher quality than the baseline models according to human evaluation.
The authors make good points, starting from the exposure bias and label bias suffered by the mainstream neural auto-regressive models.
Residual EBMs are defined and trained using NCE. Experiments on two large language modeling datasets show that residual EBMs yield lower perplexity and generation via importance sampling is of higher quality, compared to locally normalized baselines.
In generally, the paper is well motivated and interesting. But I have some concerns.
1. Missing important relevant references.
EBMs (a.k.a. un-normalized models, random fields) have been successfully developed in language modeling in recent years. A large body of this paper has been studied in [5,6], including the model and the NCE estimation method. The model Eq.(2) is exactly the model in [5], defining the model in the form of exponential tilting of a reference distribution.
Connecting and comparing to these previous works are needed.
[1] R. Rosenfeld, S. F. Chen, and X. Zhu, “Whole-sentence exponential language models: a vehicle for linguistic-statistical integration,” Computer Speech & Language,  2001.
[2] B. Wang, Z. Ou, and Z. Tan, “Trans-dimensional random fields for language modeling,” ACL, 2015.
[3] B. Wang, Z. Ou, and Z. Tan, “Learning transdimensional random fields with applications to language modeling,” IEEE transactions on pattern analysis and machine intelligence, 2018.
[4] B. Wang and Z. Ou, “Language modeling with neural trans-dimensional random fields,” IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2017.
[5] B. Wang and Z. Ou, “Learning neural trans-dimensional random field language models with noise-contrastive estimation,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018.
[6] B. Wang and Z. Ou, “Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation,” IEEE Spoken Language Technology Workshop (SLT), 2018.
2. I am a little bit concerned that the theoretical contribution seems weak. 
Though Eq. (4) and (5) seem to be novel, I am not sure whether such a contribution is substantial enough to motivate acceptance.
I'm happy to adjust the score if the paper can be better placed in the literature and the authors take efforts to improve the paper.
--------update after reading the response-----------
Being well-placed in the literature and properly claiming contribution with respect to prior work is one of the key questions in reviewing a paper. The first version of the paper clearly lacks in this respect. That's the main concern when I gave a 1.
I appreciate the authors' response. The updated paper has been improved to address my main concern, although the added discussions presented in the updated paper is not as clear as the authors' clarifications in the response. I suggest to polish the main text incorporating these clarifications.
Generally, it is nice to see the successful application of energy-based/random-field-based models in text generation, besides in speech recognition. I update the score to 6 (Weak Accept).
It would have been better that the following can be further clarified.
"the partition function estimated via importance sampling would lead to bias favoring the random field language model" --- this comment is not clear to me. 
Both Eq.4 and Eq.5 give estimates for perplexity. It would be better to clarify different uses of the two equations. If the perplexities are estimated using Eq.4 (as in Table 1), then what is the purpose of developing Eq.5?
How to calculate the lower and upper bounds of the step-wise perplexity gain at each position in Figure 1?
Under Figure 1, "At each position the lower and upper bounds (see Eq. 4) are estimated using 20,000 samples." But in the main text, it is said that "We therefore break down perplexity per position in the generated sequences as in Eq. 5" at page 8. It is confusing.