Appendix Non Monotonic Sequential Text Generation
Oracle For p* annealed, b is linearly annealed from 1.0 to 0.0 at a rate of 0.05 each epoch, after a burn in period of 20 epochs in which b is not decreased. We use greedy decoding when p* coaching is selected at a roll in step; we did not observe significant performance variations with stochastically sampling from p* coaching. These settings are based on a grid search of brate {0.01, 0.05} bburn in {0, 20} coaching rollin {greedy, stochastic} using the model selected in the Model section above.