Something seems strange about the output shape after a conv layer then. Can you separate the
x = self.activation(self.bn1(self.conv1(x)))

to
x = self.conv1(x)
x = self.bn1(x)
...

and print the shape of x in between to debug? It looks like the shape of x is smaller than expected for the batchnorm.