Answers.Body: <p>You could make the actions</p>
  <ol>
  <li>move up</li>
  <li>move down</li>
  <li>move left</li>
  <li>move right</li>
  <li>shoot</li>
  </ol>
  <p>Then you can declare the speed of this movement. If you want to go more in depth you can do</p>
  <ol>
  <li>Accelerate up</li>
  <li>Accelerate down</li>
  <li>Accelerate left</li>
  <li>Accelerate right</li>
  <li>Shoot</li>
  </ol>
  <p>This will give control of the speed to the neural network, but is harder to train, and you should give the speed of the mouse as an input.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-10T10:43:13.197Z
Answers.Text: I also thought about that. But since the score is based on the time between two hits I see some problems. Because the main goal is that the agent gets as fast as possible. But when can can only move x much pixels it will lack precision and speed I think...
Answers.UserDisplayName: Voß
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-10T11:14:50.017Z
Answers.Text: I don't think it will lack precision. And if you use the acceleration the speed purely depends on how much you accelerate it on every step, and how many frames you wait to make the next step
Answers.UserDisplayName: Lustwelpintje
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-10T10:40:13.257Z
Answers.LastActivityDate: 2022-06-04T08:03:26.223Z
Answers.LastEditDate: 2022-06-04T08:03:26.223Z
Answers.UserDisplayName: Lustwelpintje
Body: <p>I'm building a game environment (see the picture below) where an agent should position the mouse on the screen (see the coordinates on the upper right corner) and then click to shoot a cannonball. If the goal (left) is hit. The agent gets a reward based on the elapsed time between this strike and the last one. If three shots are missed, the game is done and the environment will reset.</p>
  <p><a href="https://i.sstatic.net/ZMK3H.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ZMK3H.gif" alt="enter image description here" /></a></p>
  <p>The env is done so far. But now I wonder what the action space should look like. How can I make the agent choose some x and y coordinates? And how can I combine this with a &quot;shoot&quot; action?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-10T11:07:18.430Z
Comments.Text: you can learn the mean $\mu$ of normal distribution for $x$ coordinate position and for $y$ coordinate position. Then the action would be sampled coordinates from the normal distribution. You can also learn standard deviation $\sigma$ or it can be fixed. You don't need to model shoot action, agent can shoot immediately  to given coordinates. You would need to use policy gradient/actor critic method for this
Comments.UserDisplayName: Brale
Tags: <reinforcement-learning><deep-rl><environment><action-spaces>
Title: How should I design the action space of an agent that needs to choose a 2d point and then shoot a cannonball?
UserDisplayName: Voß
ThreadText: # How should I design the action space of an agent that needs to choose a 2d point and then shoot a cannonball?
  
  (Asked by: username_0 on 2020-03-10)
  
  I'm building a game environment (see the picture below) where an agent should position the mouse on the screen (see the coordinates on the upper right corner) and then click to shoot a cannonball. If the goal (left) is hit. The agent gets a reward based on the elapsed time between this strike and the last one. If three shots are missed, the game is done and the environment will reset.
  
  [![enter image description here](https://i.sstatic.net/ZMK3H.gif)](https://i.sstatic.net/ZMK3H.gif)
  
  The env is done so far. But now I wonder what the action space should look like. How can I make the agent choose some x and y coordinates? And how can I combine this with a "shoot" action?
  
  **Tags:** reinforcement-learning, deep-rl, environment, action-spaces
  
  > **Comments:**
  > **username_1**: you can learn the mean $\mu$ of normal distribution for $x$ coordinate position and for $y$ coordinate position. Then the action would be sampled coordinates from the normal distribution. You can also learn standard deviation $\sigma$ or it can be fixed. You don't need to model shoot action, agent can shoot immediately  to given coordinates. You would need to use policy gradient/actor critic method for this  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-03-10. Score: 0 | Accepted answer
  
  You could make the actions
  
  1. move up
  2. move down
  3. move left
  4. move right
  5. shoot
  
  Then you can declare the speed of this movement. If you want to go more in depth you can do
  
  1. Accelerate up
  2. Accelerate down
  3. Accelerate left
  4. Accelerate right
  5. Shoot
  
  This will give control of the speed to the neural network, but is harder to train, and you should give the speed of the mouse as an input.
  
  > **Comments:**
  > **username_0**: I also thought about that. But since the score is based on the time between two hits I see some problems. Because the main goal is that the agent gets as fast as possible. But when can can only move x much pixels it will lack precision and speed I think...  
  > **username_2**: I don't think it will lack precision. And if you use the acceleration the speed purely depends on how much you accelerate it on every step, and how many frames you wait to make the next step  
  
  ---
  
  