To be clear, you are training an agent to act in an environment in which it occasionally receives rewards for its actions, right?
In such an environment it is common to use DQN, A3C or any number of other training techniques.
I believe that a model suitable for A3C needs to produce two outputs at each step. One suggesting actions (the actor), and another predicting the value of said actions (the critic). Your model seems to be just predicting the actions. In which case DQN or one of its variants seems more appropriate than A3C.