Hi,



 Chuong_Vo:

Note: The new tensor’s values are just copied from the old tensor.


Could you deeply explain what are you doing?
What I’ve understood is that you have a tensor T, you convert it to numpy (T_numpy) but then the network keeps using T.
If that is the case you don’t need to struggle, you can keep using T as if nothing happened.
import torch

a=torch.rand(10).requires_grad_()

b=a.sqrt().mean()

c=b.detach()
b.backward()
print(b.grad_fn)
print(c.grad_fn)

<MeanBackward0 object at 0x7fba8eefdcc0>
None

In case you want to modify T according to what you have done in numpy, the easiest way is to reimplement that in pytorch.
Else, you can create a nn.Module which implements a proper backward.
The last thing you can do is to in-place modify values of T.
import torch

a=torch.rand(10).requires_grad_()

b=a.sqrt().mean()
print(b)
b.data=torch.tensor(5).float()
print(b)
c=b.detach()
b.backward()
print(b.grad_fn)
print(c.grad_fn)

Which would go against DL theory