why didn’t u wrap avg_param is a Variable with autograd set to false? As in something like:
W = Variable(w_init, requires_grad=True)
W_avg = Variable(torch.FloatTensor(W).type(dtype), requires_grad=False)
for i in range(nb_iterations):
    #some GD stuff...
    W_avg = (1/nb_iter)*W + W_avg