Hi everyone,
I am trying to implement MetaSGD, where learning rates are learnable. I am having trouble getting gradients for them.
The code:
        for j in range(iterations):
            self.optim.zero_grad()
            inner_output = self.model(inner_x)
            inner_loss = self.criterion(inner_output, inner_y)
            grads = torch.autograd.grad(inner_loss, self.model.parameters(), create_graph=True)
            for i, param in enumerate(self.model.parameters()):
                param.grad = self.lrs[i] * grads[i]
             self.optim.step()

        outer_output = self.model(outer_x)
        outer_loss = self.criterion(outer_output, outer_y)
        
        # These lines should create the grads for the learning rates and update them accordingly.
        outer_loss.backward()
        # lrs.grad = None
        other_optimizer.step()

What happens is that lrs.grad = None. Meaning that param.grad = self.lrs[i] * grads[i] probably doesnt add to the computation graph for calculating the gradients.
And if I change the code to:
        for j in range(iterations):
            self.optim.zero_grad()
            inner_output = self.model(inner_x)
            inner_loss = self.criterion(inner_output, inner_y)
            grads = torch.autograd.grad(inner_loss, self.model.parameters(), create_graph=True)
            for i, param in enumerate(self.model.parameters()):
                param -= self.lrs[i] * grads[i]

        outer_output = self.model(outer_x)
        outer_loss = self.criterion(outer_output, outer_y)
        
        # These lines should create the grads for the learning rates and update them accordingly.
        outer_loss.backward()
        other_optimizer.step()

Then param -= self.lrs[i] * grads[i] gives a RunTimeError because of an in place operation. Which are not allowed for variables that requires grad.
<edit>
param = param - self.lrs[i] * grads[i], here no learning happens as well because the first param actually a new variable probably. And self.lrs.grad = None as well.
</edit>
Does anyone know a solution, i.e. getting gradients for the learning rates?