Made a summary here. Hope it is helpful:

Best practice: Avoid inplace operations if it is not necessary as it changes the state of tensors silently. Non-inplace operations will make a copy before doing the operation. Thus, if an operation is inplace within a function, it affects the tensor’s state outside of the function while the non-inplace operation does not change the state unless you reassign it outside of the function.

e.g.
def inplace_op(X):
	X += 1
	return X

X = torch.rand(4, 2)
inplace_op(X) # X is changed without re-asigned to X

Summary of inplace operations:

x *= 3
X[…]  = …
X.add_(1)

Common examples of inplace operations:

x += 1, x *= 3, …
x[2] = 2, X[0, 0] = 2
x[:, 3] = 3
X[2] /= 5
X[:, 3] /= 4
X[:, 3] = X[:, 3] / 4

These are not inplace operations:

x = x + 1
y = x.clone; y[0] += 100