Solved by st-vincent1 in post #3 
              
              
                Thanks @ptrblck . I was about to do this after reading this suggestion of yours somewhere else, when miraculously it worked now (touch wood). 
The only thing I changed was the way I save and load the model like this (I’m using DataParallel): 
model = MyModel(config, vocab)

model.to(config.device)
p…