Thank you for your reply. the transformation that you did in forward() as @ptrblck mentioned, makes it a non-leaf tensor. I am wondering how can we learn a parameter which should be constrained. For example, we have a custom loss function which is the combination of cross-entropy and MSE loss.
def __init__():
           my_param= nn.Parameter(torch.empty(1).cuda(), requires_grad=True)

def forward():
          gamma = torch.sigmoid(my_param)

         total_loss = gamma * cross_entropy_loss + (1- gamma) * MSE()
         return total_loss

So in this way, my_param can not be trained meaningfully. right?