Hi Ptr,
The network I am using has Dropout, Pooling layers… So when I use weight initialization as follows:
    def weights_initnot(m):
        xavier=torch.nn.init.xavier_uniform_(m.weight.data,init.calculate_gain('relu'))
        classname = m.__class__.__name__
        if classname.find('Conv') != -1:
            xavier(m.weight.data)
    #         print 'come xavier'
            #xavier(m.bias.data)
        elif classname.find('BatchNorm') != -1:
            m.weight.data.normal_(1.0, 0.02)
            m.bias.data.fill_(0)
        elif classname.find('Linear')!=-1:
            m.weight.data.normal_(0.0, 0.02)
            m.bias.data.fill_(0)
        # elif classname.find('Dropout') != -1:

I get errors like there is no weight attribute in Dropout or Pool class. which makes sense.
Then I used the following weight init using isinstance… it kinda solved the problem because I am getting similar weights in specific layer weight index with the same seed.
seed = 1
torch.cuda.manual_seed(seed)


def weights_init(m):
    if isinstance(m, nn.Conv3d):
        torch.nn.init.xavier_uniform_(m.weight.data, init.calculate_gain('relu'))
        # torch.nn.init.xavier_uniform_(m.bias.data)
    elif isinstance(m, nn.BatchNorm3d):
        m.weight.data.normal_(mean=1.0, std=0.02)
        m.bias.data.fill_(0)
    elif isinstance(m, nn.Linear):
        m.weight.data.normal_(0.0, 0.02)
        m.bias.data.fill_(0)

    Net = ResNet3D().to(device)
    Net.apply(weights_init)

On another note:
Do seed_torch and manual_seed work the same way? I have seen some codes use seed_torch.
Seeking Explanation: I was talking to one of my friends about the purpose of fix weight initialization. He said it has no big impact because the data we are feeding in are random because of the DataLoader. And we are saving the best-performing weights anyway so fixed weights are not a biggy.
What do you think about it?