Dear All,
I am trying to add the trainable weights of my custom loss function (which is a weighted sum of 2 loss functions) to the optimizer. The weights are initialized as
w = nn.Parameter(torch.Tensor([1]).cuda())

And I’ve successfully registered the weights to the optimizer’s param group, but I keep getting the ‘can’t optimize a non-leaf Tensor’ error. It would be fine if I just initialize the weights as tensors
w = torch.Tensor([1]).cuda()
w.detach().requires_grad_() 

but in this way, even if the weights have been added to the optimizer param group, it does not update over epochs.
How can I solve this problem?
Thanks