well, i haven’t worked on language modeling before…so can’t say… one more thing, I read perplexity = 2 ^ cross_entropy_loss, my loss is around 7 but perplexity is showing ~ 700 (according to the code used in pytorch examples word level language model), --> they are using e instead of 2… i guess this is the standard