So perplexity for unidirectional models is: after feeding c_0 … c_n, the model outputs a probability distribution p over the alphabet and perplexity is exp(-p(c_{n+1}), where we took c_{n+1} from the ground truth, you take and you take the expectation / average over your validation set.
Now you take a bidirectional model, feed ??? and the model outputs a probability distribution ??? and take the probability of ???.
I don’t think the perplexity makes much sense for bidirectional models, because there is no next character when you feed from both directions.
As a practical example, when I last looked fast.ai trained separate forward and backward LMs and then evaluated the perplexity on either.
Somewhat related: https://www.scribendi.ai/can-we-use-bert-as-a-language-model-to-assign-score-of-a-sentence/ 67
Best regards
Thomas
P.S.: I don’t think it’s a good habit to cross post questions to stack overflow without linking that here.