Now,
Starting from last experiment, I turn to True the track_running_stats  of the first Batch Norm layer (bn1), the other BN have track_running_stats=False

self.bn1 = nn.BatchNorm2d(16,track_running_stats=True)
…
def forward(self, x, reddening):
out = F.relu(self.bn1(self.conv1(x)))


I start to see a difference between the test loss and the train loss… more to come.