kedar:

Is this correct behaviour that when track_running_stats = False the training mode of a module is compulsory?


Yes, since the batchnorm layer cannot use the running stats to normalize the input (they are disabled) and needs to compute the input batch stats.