Solved by SSARCandy in post #3 
              
              
                Ok, I figured out why. 
I’m training DQN, and there have two network (q and target_q) when training. 
in the training process, only update one of the net (q), and the other net (target_q) will periodically copy parameters from the q net. 
The performance problem is because I only save the q net para…