In the resnet50 model from torchvision, the bottlneck layer is given as below.
Why isnt there a nonlinearity (ReLU) after each Conv2d (or BatchNorm2d)?
I see only one nonlinearity after all the 3 Convolutions.
Thanks
(layer1): Sequential(
(0): Bottleneck(
(conv1): Conv2d (64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)
(conv2): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)
(conv3): Conv2d (64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)
(relu): ReLU(inplace)
(downsample): Sequential(
(0): Conv2d (64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)
)
)
This is the model from torchvision.