Inplace operations are not allowed where the original input is needed to calculate the gradients.
E.g. in a vanilla CNN you could try to use the inplace nn.ReLU as long as PyTorch doesn’t raise an error.
If it does, then the input needs to be unchanged for Autograd.