I have a module like this:
class Block(nn.Module):
    def __init__(self, net):
        super(Block, self).__init__()
        self.net = net
        self.net_copy = copy.deepcopy(net)

    def forward(self, x):
        self.net_copy.load_state_dict(self.net.state_dict())
        return self.net(x)

The net is an nn.Sequential() module. When I use Pytorch>=1.5 and use nn.DataParallel in multi-GPUs, It shows that net_copy.state_dict().keys() is different with net.state_dict().keys(). However, when I use Pytorch==1.4 or single-GPU, this problem doesn’t appear. How can I make sure that net and net_copy is exactly the same?