Thank you so much! That works!
Also, I think it might be related to the way the following code is executed
net = torch.nn.DataParallel(net).cuda()

When I changed it to
net = net.cuda()
net = torch.nn.DataParallel(net)

The running mean and var did get updated.