Solved by albanD in post #2 
              
              
                Hi, 
A leaf Variable is a variable that is at the beginning of the graph. That means that no operation tracked by the autograd engine created it. 
This is what you want when you optimize neural networks as it is usually your weights or input. 
So to be able to give weights to the optimizer, they sho…