There is some cases that in training we don’t want to track the statistics.
In evaluation, if track_running_stats=False, then this statement will make training=True and the batch_norm will compute statistic as I understand. This is not correct because in evaluation we want to use the pre-computed statistic rather than compute this in evaluation.