Wanted to update that i was able to get the same results as Keras.
Not sure exactly what i’ve done wrong, but i tried to get the same embedding for the same input in both Keras and PyTorch (as mentioned above), there i saw i was getting different results!
So using the weight matrix (pre-training embedding one), i use in Keras, i plugged it in pretrained_embeddings and i was able to get the same result.
@tom - it would be great if you could respond to my previous message and outline how you’d go about debugging, i can definitely learn from your suggestion.
Thanks everybody who helped. I also use @ptrblck initialization (haven’t done ablation of not using it, yet)