@ptrblck Here is the output.
print("policy_net(state_batch).shape: {}".format(policy_net(state_batch).shape))
print("state_batch.shape: {}".format(state_batch.shape))
print("action_batch: shape= {}, max= {}, min= {}".format(action_batch.shape, action_batch.max(), action_batch.min()))
state_action_values = policy_net(state_batch).gather(1, action_batch)

output:
policy_net(state_batch).shape: torch.Size([128, 2])
state_batch.shape: torch.Size([128, 3, 180, 320])
action_batch: shape= torch.Size([128, 1]), max= 6, min= 0

I have a total of 7 actions. Action values 0 to 6 are mapped for the following drone movements: FORWARDS, BACKWARDS, STRAFE_LEFT, ;STRAFE_RIGHT, UP, DOWN, STOP.
There are a total of 8 observations. x, y, z ,r, p, y, sonar_value, collision

  
      github.com
  
  
    edowson/alphapilot_openai_ros/blob/master/ardrone_race_track/src/ardrone_v1_goto_task_env.py#L39-L40 1

number_actions = rospy.get_param('/drone/n_actions')
self.action_space = spaces.Discrete(number_actions)



  
  
    
    
  
  



  
      github.com
  
  
    edowson/alphapilot_openai_ros/blob/master/ardrone_race_track/src/ardrone_v1_goto_task_env.py#L158-L180 1

if action == 0: #FORWARDS
    linear_speed_vector.x = self.linear_forward_speed
    self.last_action = "FORWARDS"
elif action == 1: #BACKWARDS
    linear_speed_vector.x = -1*self.linear_forward_speed
    self.last_action = "BACKWARDS"
elif action == 2: #STRAFE_LEFT
    linear_speed_vector.y = self.linear_forward_speed
    self.last_action = "STRAFE_LEFT"
elif action == 3: #STRAFE_RIGHT
    linear_speed_vector.y = -1*self.linear_forward_speed
    self.last_action = "STRAFE_RIGHT"
elif action == 4: #UP
    linear_speed_vector.z = self.linear_forward_speed
    self.last_action = "UP"
elif action == 5: #DOWN
    linear_speed_vector.z = -1*self.linear_forward_speed
    self.last_action = "DOWN"
elif action == 6: #STOP
    linear_speed_vector.x = 0.0


  This file has been truncated. show original

  
  
    
    
  
  


I’m running it on the cpu, to debug, and it gives the following error:
  File "/project/ros-kinetic-alphapilot/catkin_ws/src/alphapilot_openai_ros/ardrone_race_track/src/ardrone_v1_ddqn.py", line 446, in optimize_model
    state_action_values = policy_net(state_batch).gather(1, action_batch)
RuntimeError: Invalid index in gather at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:457