Just one more thing I have notice that
let’s say that I change the code to the following
when I first time do backward on my regular loss function
loss = torch.mean((f_z-z)**2)
loss.backward(der_shape,retain_graph=True) # first time loss

Then,  do the variance control
    var_opt.zero_grad()
    var_loss = torch.mean((gr_a+gr_b-loss)**2). # use first loss value so retain_graph=True for the first time

please ignore the crazy loss func that doesn’t make sense; I’m simply playing around to make myself familiar with the torch system
Which makes sense; however, this time; it NO longer requires me to do
gr_b = torch.tensor(b.grad.clone(), requires_grad=True)
gr_a = torch.tensor(a.grad.clone(), requires_grad=True)

Instead, the code can work even with simply
    gr_a = a.grad.clone()
    a.grad.data.zero_()

    gr_b = b.grad.clone()
    b.grad.data.zero_()


But since my second time backward(), it simply incorporate the first loss func, its two loss function together; pytorch no longer requires to do the trick you taught me; makes me think that second time backward(); pytorch only did differentiation on first loss function; and for gr_a, gr_b; they simply treated as constant here and never been treated as part of variance loss function (the second loss func)?
to be more clear
when the second time backward()
    var_loss = torch.mean((gr_a+gr_b-loss)) # not requires gr_a gr_b to be requires_grad=True
    # does gr_a and gr_b treated as const here; how to make the backward() differentiate gr_a and gr_b?

    #var_loss = torch.mean(gr_a+gr_b) # must have gr_a gr_b to be requires_grad=True
   

Here is the full code
a = Variable(torch.Tensor([-2.]),requires_grad=True)
b = Variable(torch.Tensor([-2.]),requires_grad=True)
z = 2*x+10
zz = 3*x + 10
nn = Control_Variate() # a simple neural network


var_opt = torch.optim.Adam(nn.parameters(), lr=0.1)
for i in range(5000):
 

    f_z = nn(a*x+b)
    
    der_shape = torch.ones(f_z.size())
    loss = torch.mean((f_z-z)**2)
    loss.backward(der_shape,retain_graph=True)
    
    
    gr_a = a.grad.clone()
    a.grad.data.zero_()

    gr_b = b.grad.clone()
    b.grad.data.zero_()
    g_a,m_a,v_a = Adam_Optim(gr_a)
    g_b,m_b,v_b = Adam_Optim(gr_b)
    
    b.data.sub_(g_b)
    a.data.sub_(g_a)

    

    var_opt.zero_grad()
    
    var_loss = torch.mean((gr_a+gr_b-loss)) # not requires gr_a gr_b to be requires_grad=True
    #var_loss = torch.mean(gr_a+gr_b) # must have gr_a gr_b to be requires_grad=True
    
    var_loss.backward()
    var_opt.step()
    print(loss)
    


print("done")
    

because I kind of want the second time backward() do differentiate on gr_a and gr_b; now it looks like it doesn’t because here gr_a and gr_b not even variable but constant