You can’t test it like this. As I said, the eps is right in initialization, but in the forward function, they are all set to 1e-3.
Code is like this:
class BNPReLU(nn.Module):
    def __init__(self, nIn):
        super().__init__()
        self.bn = nn.BatchNorm2d(nIn, eps=1e-3)
        self.acti = nn.PReLU(nIn)

    def forward(self, input):
        output = self.bn(input)
        output = self.acti(output)

        return output


class Main_net(nn.Module):
    def __init__(self, classes=2):
        super().__init__()
        self.bn_prelu_1 = BNPReLU(32)
        self.decode1 = nn.Sequential(nn.Conv2d(67, 16, kernel_size=1, stride=1, bias=False),
                                     nn.BatchNorm2d(16, eps=1e-5),
                                     nn.ReLU())
         print('self in initialization:', self)

     def forward(self, input):
        print('self in forward:', self)
        output0 = self.bn_prelu_1(input)
        output = self.decode1(output0)

In the main.py,  I set like this:
    model = Main_net(class=2)
    model.train()
    output = model(input)

The result: because i am a new user and only upload one picture, so I merge these results in one

image1418×1008 180 KB

By the way, I also set the eps other value, such as 1e-4 in BNPReLU and nn.BatchNorm2d. But in the forward process, the eps is all set to 1e-3. Very weird.