Solved by ptrblck in post #2 
              
              
                You could load the pretrained weights and just call the desired init method on the last two layers: 
model = ...
model.load_state_dict(torch.load(...))  #  or your equivalent code using torch.load
with torch.no_grad():
    torch.nn.init.xavier_uniform_(model.fc1.weight)
    torch.nn.init.zeros_(mode…