Solved by mobius in post #8 
              
              
                If you have parameters with requires_grad = False backward() does not compute a gradient on those. 
So you could try something like: 
def train():
    for batch_idx, (data, target) in enumerate(train_loader):
        optimizer.zero_grad()
        output = model(data)

        loss = criterion(output…