Hi, it’s me again with questions about the language model example here 10.
When using a regular, unidirectional LSTM without weight tying, I’m getting perplexities of around 100, which is expected. Just by changing the model to a bidirectional LSTM (and its related changes), I’m getting perplexities around 1 for the test set, which doesn’t make any sense.
Anyone might have an idea why is this happening? If it was overfitting, I’d expect the test perplexities to be much higher. but they are also around 1…
Thanks!