Some evident erros, there might be more:


What’s this? n-step learning?
  for index, (current_state, action, reward, new_current_state, done) in enumerate(batch):
         done = False
         if not done:
             max_future_q = np.max(future_qs_list[index])
             new_q = reward + self.gamma * max_future_q
         else:
             new_q = reward



No direct call of Forward when you want to calculate gradients:
preds = self.policy_net.forward(...)

You should do:
preds = self.policy_net(...)