Hey, did you find an answer to your question?
What is the right way (if there is a need) to normalize the perplexity number based on sentence length? Should I divide by the number of tokens  ? I have a reason to believe that they must already be doing it on the inside in the loss computation. Not sure though