Solved by chenglu in post #3 
              
              
                The core reason of “those variables not updated” is because they are not registed in the optimizer. If you look at any optimizer’s constructor, there will be a parameter named params, which is expected to hold the parameters you want to automatically updated by grad. 
I’m not sure how you initialize…