Hi,
all my inputs are normalized. I compute the mean and standard deviation over the dataset and normalize all images with that. There are no nans or infs in the data.
How could this issue be related to my sampling strategy? Oversampling some cases may skew the running stats but not to an extend where everything collapses and especially the models recovering back to normal towards the end of the training (when the lr gets lower) contradicts that.
I don’t quite know how the learning rate interacts with batch norm layers. As far as I remember, the gradient is not stopped for mean and var computation (via something like detach()). Maybe there is something there?
Best,
Fabian