OK. Then it’s at least not my code doing it.  I had tried to see if I could get the Welford variance update to be negative, but it seems to not be easy (you would need x - avg and x-(avg + (x-avg)/n) to be of different signs).
More seriously, you could try to see if you can avoid the problem by calling batch norm in a with torch.backends.cudnn.flags(enabled=False): block (but I didn’t try if it works or helps).
Best regards
Thomas