The layers of my network are of the form:
        self.layer11 = nn.Sequential(
            nn.Conv2d(in_channels=1024, out_channels=1024, kernel_size=1),
            nn.BatchNorm2d(1024, track_running_stats=False),
            nn.ReLU()
        )

with some max pooling layers.
My forward function looks like this:
    def forward(self, x):
        x = self.layer1(x)
        x = self.layer2(x)
        ...
        x = self.layern(x)
        return x