i tried the snippet i am pasting and i think i will have to execute a stack or cat after each optimizer.step call. In the snippet below the changes in the leaf tensors dont seem to be communicated to the cat/stack, while changes to the stack/cat are communicated to the leaf:
a = torch.randn(1,2)
b = torch.randn(1,2)
c = torch.cat([a,b],0)

print(a)
a.data = a.data+1
print(a)
print('*'*50)
print(c)
print('\n\n')
c.data[0,:] = c.data[0,:] *10
print(c,a)


tensor([[ 0.5121, -0.8800]])
tensor([[1.5121, 0.1200]])
**************************************************
tensor([[ 0.5121, -0.8800],
[-0.6470,  0.2288]])


tensor([[ 5.1207, -8.7999],
[-0.6470,  0.2288]]) tensor([[1.5121, 0.1200]])