But I wanna ask that here, does the n is the .num_batches_tracked in the BatchNorm parameters?
and why is that my batch num is not 1 and still get nan,
pretrain_dict['featureExtract.12.num_batches_tracked']
Out[52]: tensor(8638, device='cuda:0')