the PR to make BN work correctly with QAT+DDP is here: https://github.com/pytorch/pytorch/pull/38478 18 .  This enables SyncBatchNorm to be swapped in to a fused QAT Conv-BN.  I will update the issue.  There were also a couple of bug fixes landed, such as https://github.com/pytorch/pytorch/pull/38368 6