How to reach model attributes wrapped by nn.DataParallel?
What is the way to reach model attributes inside DataParallel?
erogol
i come across the same problem as u  met,how do u solve it,thank u
andyhx
You can access them by using <SCODE>model.module.rnn
<ECODE> but it’s not exposed in the public API, so it might change in the future.
fmassa
oneTaken
What if I chage the attribute by this way? Will the models on other GPUs change synchronously?
MrTuo
<SCODE>    device =  "cuda:0"

    graph = Text2Mel().to(device)
    # set the training flag
    graph.train()
<ECODE> i tried ‘dataparallel’ but i’m noob in pytorch and alway i have this error: <SCODE>AttributeError: 'DataParallel' object has no attribute 'attention'
<ECODE> I would appreciate any help, thanks!
Victor_Maldonado
<SCODE># simple fix for dataparallel that allows access to class attributes
class MyDataParallel(torch.nn.DataParallel):
    def __getattr__(self, name):
        try:
            return super().__getattr__(name)
        except AttributeError:
            return getattr(self.module, name)
        
#     def __setattr__(self, name, value):
#         try:
#             return super().__setattr__(name, value)
#         except AttributeError:
#             return setattr(self.module, name, value)

def load_weights(base_model_name, model, epoch):
    """
    Loads previously trained weights into a model given an epoch and the model itself
    :param base_model_name: name of base model in training session
    :param model: the model to load weights into
    :param epoch: what epoch of training to load
    :return: the model with weights loaded in
    """

    pretrained_dict = torch.load('{}/{}_{}_{}.pt'.format(args.model_path, base_model_name, epoch, args.lr))['state_dict']
    model_dict = model.state_dict()
    pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
    model_dict.update(pretrained_dict)
    model.load_state_dict(model_dict)

    return model

model = r2plus1d50()
model = MyDataParallel(model, device_ids=[0, 1])

# Copying the layer from epoch 13 to model
epoch = 13
head_model = r2plus1d50()
head_model = MyDataParallel(head_model, device_ids=[0, 1])
head_model = load_weights(base_model_name, head_model, epoch)
# head1 is the name of the attribute that we want to copy over
head = getattr(head_model, "head1")

# set attribute to change the attribute
setattr(model.module, "head1", head)

<ECODE>
hobi