Solved by NoobCoder in post #5 
              
              
                Thanks for your responses. 
I will put the code maybe it is useful for others: 
# Unwrap from DataParallel
    if torch.cuda.device_count() > 1:
        model = model.module

    model.classifier = Identity()

    if torch.cuda.device_count() > 1:
        print("Wrap Again with DataParallel with ", …