I would recommend to call model.apply with your weight init method and remove the loop inside xavier_init:
def xavier_init(ms):
    if isinstance(m, nn.Linear) or isinstance(m, nn.Conv2d):
        nn.init.xavier_uniform(m.weight,gain=nn.init.calculate_gain('relu'))
        m.bias.data.zero_()

model.apply(xavier_init)

Also, your instantiation of net looks as if you are importing the model directly from torchvision.models instead of your custom ResNet definition.