inplace means that it will not allocate new memory and change tensors inplace. But from the autograd point of view, you have two different tensors (even though they actually share the same memory). One is the output of conv (or batchnorm for resnet) and one is the output of the relu.