Solved by Miles_Cranmer in post #2 
              
              
                Are you using nn.DataParallel both for the CPU and GPU models? I had a similar issue and that fixed it for me. 
Edit: Doesn’t look like you are. Try that adding nn.DataParallel(.) to wrap your model before loading the state dict.