Deep Learning, Artificial Intelligence, Neural Networks, Word Prediction, Language Model.
perplexity over the test set is equivalent to that of the one trained with exact gradient at its overfitting point (18 epochs, 113 days). Surprisingly enough, if letting the sampling-trained model converge, it starts to overfit at epoch 18 — as for classical training — but with a lower test