Solved by ptrblck in post #2 
              
              
                I can’t find any obvious differences. 
Sometimes an unwanted broadcasting takes place in the loss calculation, if you don’t pass an output and target in the same shape to nn.MSELoss, which should raise a warning but if often overlooked. 
Could you double check it in your PyTorch code?