Thanks for the reply! You are awsome!
The running stats looks to handle the differences between different areas quite well in the training so I think that could be a good thing also for evaluation. The model will be used over larger areas and each area have similar properties so running stats could be used in production per area. This network only have batch normalization that depend on the mode train/eval so it could be used in training mode.
The settings for the layers are in a library so I need to change the library or modify the layers afterwards.
Is there a drawback using the running stats for data normalization in production (data leak)?
(Normalization is difficult and this normalization occur deep in the network.)
I will also try to see if I can change the momentum and get a network that is more robust and don’t need the running stats.
Thanks again!