Solved by Tiankai_Hang in post #8 
              
              
                Solved. 
I used nn.Dataparallel to train my model on multi-gpu machine, it calculated the gradient on the main gpu, so my function in class to update the parameters is incorrect. 
When I choose to use on gpu or DDP, it works well.