I am reproducing the paper " Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics". The loss function is defined as

This means that W and σ are the learned parameters of the network. We are the weights of the network while σ are used to calculate the weights of each task loss and also to regularize this task loss wight.
It is easy to implement the L1 and L2 (assume they are L1 loss)
loss1 = nn.L1Loss()
loss2 = nn.L1Loss()
input1 = torch.randn(3, 5, requires_grad=True)
input2 = torch.randn(3, 5, requires_grad=True)
target= torch.randn(3, 5)
loss_total = 1/(2*sigma1^2)*loss1(input1, target) +  1/(2*sigma2^2)* loss2(input2, target) + log(sigma1*sigma2)
loss_total.backward()

However, the weight σ also learned. How can I make the σ learnable in the combined loss