You are creating the new self.conv and self.bn layers inside the forward pass without specifying the device, so they will be created on the CPU by default.
To properly push them to the GPU, you could use:
    def reset_parameters(self, x):
        self.conv = nn.Conv2d(x.size(1), self.num_classes, kernel_size=1).to(x.device)
        self.bn = nn.BatchNorm2d(self.num_classes).to(x.device)

Additionally, you could also check the lazy modules, e.g. LazyConv2d 3, which perform a similar approach.