Thx juan,
But in this way,
optimizer = torch.optim.SGD(model_parameters,lr=LR, weight_decay=WEIGTH_DECAY) 

The lr in the SGD()  seems not funtion, right? since you already pre set all the lr for each param

,lr=LR,