Maybe there are some minor differences between your PyTorch and Keras/TF code.
Could you post the Keras code and also some dummy input and target tensors, i.e. data = torch.randn(...) so that we could compare the results and debug the code?