I would try to narrow down the problem a bit and e.g. start with a single layer.
Once you get the same outputs (up to floating point precision) for the embedding layer, I would try to match the output of the next LSTM layer and so on.