Hi Peter
Many thanks for your recommandation.
In my code like that:
def __init__(self):
    super(NetDSpace, self).__init__()
    self.conv1 = nn.Conv2d(1, 6, 3)
    self.pool = nn.MaxPool2d(2, 2)
    self.conv2 = nn.Conv2d(6, 16, 3)
    self.conv2_bn = nn.BatchNorm2d(16)
    self.fc1 = nn.Linear(16 * 3 * 3, 120)
    self.fc1_bn = nn.BatchNorm1d(120)
    self.fc2 = nn.Linear(120, 84)
    self.fc3 = nn.Linear(84, 10)

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = x.view(-1, 16 * 3 * 3)
    x = F.relu(self.fc1_bn(self.fc1(x)))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x

So It’s correct?
I have tested and I have no more nan values but the loss values are very variable some times decrease and some times increase. (not stable)
Question please What did you mean with ‘And when indeed that helps you can use normalise over your whole training set.’