Thanks, this is very informative. Indeed I use data to overcome inplace correctness checks.
Didn’t know data does a shallow copy.
You say we can just avoid the correctness checks under torch.no_grad?
One thing I want to fully understand, the aliasing you mentioned in nn.Parameter
a = torch.randn(1)
b = nn.Parameter(a)

When we do b.data is it exactly like calling a.data? Is that what you mean? is it a problem?