I just changed the state from the image difference to the original state of the game environment. After tuning the learning rate, everything is fine now. It seems the image input is difficult to train.