Try not to hard copy them. Q-Learning tends to diverge fast if the target is moving too fast.
Also you have a really deep neural network. Try a more shallow network. use only 2-3 layers. As mentioned in the OFENET paper deep RL networks tend to diverge also.
Beside does it work now?