Yeah, unfortunate as it is, something isn’t terribly robust about the model because errors of that size might happen even when switching backends within PyTorch.
You could try to train the PyTorch model for a few steps to give answers that more closely match the torch7 ones.