There’re so many ways to set different learning rates for different layers.
By changing the "params" in the optimizer,we could get what we want.
I would like to know how this whole thing work.
By this tutorial:
How to use an optimizer 36
It seems that we pass the tensor and specify the learning rate for these parameters,so I write some code for testing:
    large_lr_layers = nn.Sequential(*list(model.children())[:-4]).parameters()  
    small_lr_layers = nn.Sequential(*list(model.children())[-4:]).parameters()  

    optimizer = optim.SGD([{"params": large_lr_layers,"lr":0.1},
                           {"params":small_lr_layers,"lr":0.01},
                           {"params": torch.randn((3,64,64))}],lr = args.lr*1000000000000000.,momentum=0.85)

The last line is that I create a tensor randomly and start training ,and the whole process is not ruined by it,How does Pytorch tell the randomly created tensor should be ignored?Thanks