Solved by KFrank in post #2 
              
              
                Hi L! 

There are a number of ways to do this.  I would probably use two 
separate optimizers: 
optModel = torch.optim.SGD (myModel.parameters(), lr = 0.1)
optAlpha = torch.optim.SGD ([alpha], lr = 0.1)

(This assumes that alpha is not one of myModel's parameters.) 
Now you can call optModel.step() …