This is probably due to this PR: https://github.com/pytorch/pytorch/pull/33907 5
In v1.5, parameters on replicated models are no longer considered as leaves, as they shouldn’t be. If you really need to access those replicated parameters, you probably can get them from _former_parameters and manually add them into the stat_dict?

  
      github.com
  
  
    pytorch/pytorch/blob/c93e96fbd9903e576c6c1aa2fe12d8d548ae2d5b/torch/nn/parallel/replicate.py#L148 2

            replica._parameters[key] = None
    else:
        param_idx = param_indices[param]
        for j in range(num_replicas):
            replica = module_copies[j][i]
            param = param_copies[j][param_idx]
            # parameters in replicas are no longer leaves,
            # so setattr them as non-parameter attributes
            setattr(replica, key, param)
            # expose the parameter for DDP
            replica._former_parameters[key] = param
for key, buf in module._buffers.items():
    if buf is None:
        for j in range(num_replicas):
            replica = module_copies[j][i]
            replica._buffers[key] = None
    else:
        if buf.requires_grad and not detach:
            buffer_copies = buffer_copies_rg
            buffer_idx = buffer_indices_rg[buf]
        else:



  
  
    
    
  
  


cc @ngimel please correct me if I am wrong. And any thoughts on whether we should make state_dict() consistent between v1.4 vs v1.5?