Thanks, it worked for me, and I also tried it in torch 1.4.0. The results are just like your results.
But it’s worth noting that when this code was run in torch 1.2.0, the results were as follows
Epoch 1, lr1 0.001, lr2 0.001
Epoch 2, lr1 0.001, lr2 0.001
Epoch 3, lr1 0.001, lr2 0.001
Epoch 4, lr1 0.001, lr2 0.001
Epoch 5, lr1 0.0001, lr2 0.0001
Epoch 6, lr1 0.0001, lr2 0.0001
Epoch 7, lr1 0.0001, lr2 0.0001
Epoch 8, lr1 0.0001, lr2 0.0001
Epoch 9, lr1 0.0001, lr2 0.0001
Epoch 10, lr1 1.0000000000000003e-05, lr2 1.0000000000000003e-05
Epoch 11, lr1 1.0000000000000003e-05, lr2 1.0000000000000003e-05
Epoch 12, lr1 1.0000000000000003e-05, lr2 1.0000000000000003e-05
Epoch 13, lr1 1.0000000000000003e-05, lr2 1.0000000000000003e-05
Epoch 14, lr1 1.0000000000000003e-05, lr2 1.0000000000000003e-05