Is there any new update for this question. I am also want to keep two running_mean, and variance. One set is for source domain, other one is target domain. however pytorch can only keep one running_mean, and variance for both source and target domain.