Solved.
I used nn.Dataparallel to train my model on multi-gpu machine, it calculated the gradient on the main gpu, so my function in class to update the parameters is incorrect.
When I choose to use on gpu or DDP, it works well.