It looks like your Keras model uses a dropout layer directly for the input.
Could you add it to your PyTorch model and check the accuracy again?
The f1_prev computation should work on the GPU after a few changes.
Try to replace all numpy methods with PyTorch ones and pass CUDA tensors to this method.