Solved by KFrank in post #2 
              
              
                Hi Snowball! 

Yes. 

Not exactly. 
Let w be a Parameter (or for than matter, just a Tensor that 
has requires_grad = True, but is not wrapped in a Parameter), 
and let L be a scalar (that is, a tensor with a single element) that 
has been calculated from w (and a bunch of other Parameters 
and non-P…