I trained my model on GPU and saved the model with
params = {k:v.cpu() for k, v in self.state_dict().items()}
torch.save(params, open('model.pt', 'wb'))

Then, when evaluating the model, I load it like this:
model = m(...)
params = torch.load('model.pt')
model.load_state_dict(params)

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)

However, I am getting much worse results on GPU, as on CPU. It seems like on CPU its performing correctly (I get similar results as in the paper), but on GPU its about ~20% worse.
I wrote all my code device agnostic, so apart from saving/loading a model, I never specify the device. In my forward pass, when I needed a new tensor, I usually used input_tensor.new_*(...) to create a new tensor on the same device as the input.
Anyone got an idea where the different performances might come from?