The following is my code. If I remove inplace=True in relu, I do not have error. However, if I leave it, I will have RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation error

class BasicBlock(nn.Module):
def __init__(self, inplanes, planes, use_prelu, use_se_block, anchor=None):
    super(BasicBlock, self).__init__()
    self.anchor = anchor
    self.conv1 = conv3x3(inplanes, planes, stride=1)
    nn.init.xavier_normal_(self.conv1.weight)
    self.conv2 = conv3x3(planes, planes)
    nn.init.normal_(self.conv2.weight, mean=0, std=0.01)

    self.use_prelu = use_prelu
    if self.use_prelu:
        self.prelu1 = nn.PReLU(planes)
        self.prelu2 = nn.PReLU(planes)
    else:
        self.relu = nn.ReLU(inplace=True)

    self.use_se_block = use_se_block
    if self.use_se_block:
        self.se = SEBlock(planes)



def forward(self, x):
    if self.anchor is not None:
        x = self.anchor(x)

    residual = x

    x = self.conv1(x)
    if self.use_prelu:
        x = self.prelu1(x)
    else:
        x = self.relu(x)

    x = self.conv2(x)
    if self.use_prelu:
        x = self.prelu2(x)
    else:
        x = self.relu(x)

    if self.use_se_block:
        x = self.se(x)

    x += residual

    return x


I consulted the usage from https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py 23 and cannot understand why there is no such error when used in resnet but I encounter the error in my code.
The usage in resnet.py is also weird. It is initialized with inplace=True. However, it is used with x = self.relu(x) in forward. Doesn’t this defeat the purpose of inplace=True? Is it equivalent to just writing self.relu(x) without reassignment?