Hi,
I am trying to extend the size of a nn.parameter, but without creating a new one. Is it possible?
Here is a simplified code of what I want to do:
import torch

w = torch.nn.Parameter(torch.randn((2,5)),requires_grad=True)

l = torch.sum(w)
l.backward()

# if "del l" is added here then the code runs 

ext_w = torch.randn((1,5),requires_grad=True) # now the shape is (3,5)
w.data = torch.cat((w.data, ext_w), dim=0)
w.requires_grad = True
w.grad= None

l = torch.sum(w)
l.backward()

if I run this code I get this error:
RuntimeError: Function SumBackward0 returned an invalid gradient at index 0 - got [3, 5] but expected shape compatible with [2, 5]
my question is why is this happening? which object is expecting a shape [2,5]?
The weird thing is that, I tried deleting the l variable (del l) before extending the w parameter, and in that case it seems to work
thanks a lot in advance