Solved by ptrblck in post #2 
              
              
                You are saving the nn.DataParallel state_dict, which will include the .module keys as described here. You could either wrap the model first into nn.DataParallel and load the state_dict afterwards or store the net.module.state_dict() instead.