I saw torch vision use nn.ReLU(inplace=True) after a CONV layer (https://github.com/pytorch/vision/blob/1aef87d01eec2c0989458387fa04baebcc86ea7b/torchvision/models/vgg.py#L74 2). However, based on the description for here (Why relu(inplace=True) does not give error in official resnet.py but it gives error in my code? 1). I think the output of the CONV layer is modified. I wonder if this affects the backward propagation on the CONV layer. Please feel free to correct me, if I am wrong.
Thanks.