Hello,



 MrRobot:

If I write a conditional in the model definition that looks like following. I am not sure this conditional will be executed


Yes conditionals in the the forward function are executed since PyTorch builds the computation graph dynamically (at runtime) from the operations in the forward function. For example, here is the forward function of a BasicBlock (code 10) in PyTorch’s implementation of ResNet:
    def forward(self, x):
        identity = x

        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)

        out = self.conv2(out)
        out = self.bn2(out)

        if self.downsample is not None:
            identity = self.downsample(x)

        out += identity
        out = self.relu(out)

    return out