Hi any reason you can’t do gradient descent over the parameter as well? Say the optimizer over your network is opt, make another optimizer for the parameter in question:
opt_sig = torch.optim.SGD([sigma],lr=1e-1)

and whenever you do a optimizer step for your network, also do a step with opt_sig:
opt.zero_grad()
opt_sig.zero_grad()
loss.backward()
opt.step()
opt_sig.step()