Solved by ttoosi in post #11 
              
              
                I could successfully bypass loss.backward() in a simple network for both MSE loss and CrossEntropy loss and I replaced it with a backward network. For future reference, I am sharing the code. Thanks albanD! 
import torch.nn as nn
import torch.optim as optim
import torchvision

import numpy as np
imp…