Solved by albanD in post #2 
              
              
                Hi, 
The nn.Parameter() always require gradients, no need to specify it. 
Also when you do .to() you get a new Tensor (which is not a leaf Tensor !) Didn’t you get a warning when you did the optimizer step? 
You should change the definition to self.mult = nn.Parameter(torch.rand(3, device=device))