Answers.Body: <p>Yes this is indeed a very plausible approach !
  Mind however, that RL is not very sample efficient, so you need a lot of runs for try and error.
  So it essentially boils down to the question whether you can provide/simulate a sufficient amount of samples to train the RL agent. If so, then I would recommend you to go with a very simple Q-learning approach. No deep neural networks are needed here.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-18T09:44:53.993Z
Answers.Text: I could provide samples as I am playing. I was thinking basically to start with random throws and adjust the aim and velocity as I go accordingly to the Q-table.
Answers.UserDisplayName: zaxunobi
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-02T21:30:17.580Z
Answers.LastActivityDate: 2024-08-02T21:30:17.580Z
Answers.UserDisplayName: Juhuuu
Body: <p>I am trying to automate the throw of a ball in a game in order to get &quot;Excellent Throw&quot;.</p>
  <p>To achieve an &quot;Excellent&quot; throw, you need to hit the center of the shrinking target circle when it's at its smallest size. The mechanics involve:</p>
  <p>Target Circle: When you encounter a Pokémon, a target circle appears around it, which shrinks and expands. The size of this circle determines the throw bonus:</p>
  <p>Large Circle: &quot;Nice&quot; throw</p>
  <p>Medium Circle: &quot;Great&quot; throw</p>
  <p>Small Circle: &quot;Excellent&quot; throw</p>
  <p>Timing: The key to getting an &quot;Excellent&quot; throw is to release the Poké Ball when the target circle is at its smallest.</p>
  <p>Accuracy: You need to aim for the center of the target circle to hit the Pokémon accurately.</p>
  <p>Examples:</p>
  <p><a href="https://i.sstatic.net/f5gxLoQ6.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/f5gxLoQ6.jpg" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/Tpx6EAzJ.webp" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Tpx6EAzJ.webp" alt="enter image description here" /></a></p>
  <p>What I would normally do is:</p>
  <pre><code>1. Initialize game screen capture.
  2. Detect and track the target circle's size.
  3. Calculate the optimal release time when the circle is at its smallest.
  4. Aim for the center of the circle.
  5. Simulate the throw with appropriate speed and trajectory.
  </code></pre>
  <p>The thing is I would need to adjust these for each pokemon, since each one has a different circle. So I was wondering if ML would be preferable in this case scenario and more specifically if RL would be the more preferable choice.</p>
  <p>The reward would be of course reading from the screen capture &quot;Excellent!&quot; after the throw.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-02T23:47:14.450Z
Comments.Text: Why are you worried about need to adjust your algo for each pokemon? The same algo should apply to each appeared pokemon universally once it detects its appearance on the screen. Also note your reward is sparse and binary at the very end of throw assuming the algo only has one chance to throw for each pokemon.
Comments.UserDisplayName: cinch
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-18T09:39:08.780Z
Comments.Text: @cinch Because different pokemon have different circles size and distance. The problem is not so much with the circle size but with the distance of the pokemon, so I would need to adjust the velocity of the swipe. Not necessarily binary the reward since I could also provide also rewards with lesser values.
  Miss: -1
  Normal throw: 0
  Nice throw: 1
  Great throw: 2
  Excellent throw: 3
Comments.UserDisplayName: zaxunobi
Tags: <reinforcement-learning><machine-learning><deep-rl><ai-design><reward-functions>
Title: Should I use reinforcement learning to automate the throw of a ball in a Pokemon game?
UserDisplayName: zaxunobi
ThreadText: # Should I use reinforcement learning to automate the throw of a ball in a Pokemon game?
  
  (Asked by: username_0 on 2024-08-02)
  
  I am trying to automate the throw of a ball in a game in order to get "Excellent Throw".
  
  To achieve an "Excellent" throw, you need to hit the center of the shrinking target circle when it's at its smallest size. The mechanics involve:
  
  Target Circle: When you encounter a Pokémon, a target circle appears around it, which shrinks and expands. The size of this circle determines the throw bonus:
  
  Large Circle: "Nice" throw
  
  Medium Circle: "Great" throw
  
  Small Circle: "Excellent" throw
  
  Timing: The key to getting an "Excellent" throw is to release the Poké Ball when the target circle is at its smallest.
  
  Accuracy: You need to aim for the center of the target circle to hit the Pokémon accurately.
  
  Examples:
  
  [![enter image description here](https://i.sstatic.net/f5gxLoQ6.jpg)](https://i.sstatic.net/f5gxLoQ6.jpg)
  
  [![enter image description here](https://i.sstatic.net/Tpx6EAzJ.webp)](https://i.sstatic.net/Tpx6EAzJ.webp)
  
  What I would normally do is:
  
  ```
  1. Initialize game screen capture.
  2. Detect and track the target circle's size.
  3. Calculate the optimal release time when the circle is at its smallest.
  4. Aim for the center of the circle.
  5. Simulate the throw with appropriate speed and trajectory.
  
  ```
  
  The thing is I would need to adjust these for each pokemon, since each one has a different circle. So I was wondering if ML would be preferable in this case scenario and more specifically if RL would be the more preferable choice.
  
  The reward would be of course reading from the screen capture "Excellent!" after the throw.
  
  **Tags:** reinforcement-learning, machine-learning, deep-rl, ai-design, reward-functions
  
  > **Comments:**
  > **username_1**: Why are you worried about need to adjust your algo for each pokemon? The same algo should apply to each appeared pokemon universally once it detects its appearance on the screen. Also note your reward is sparse and binary at the very end of throw assuming the algo only has one chance to throw for each pokemon.  
  > **username_0**: @username_1 Because different pokemon have different circles size and distance. The problem is not so much with the circle size but with the distance of the pokemon, so I would need to adjust the velocity of the swipe. Not necessarily binary the reward since I could also provide also rewards with lesser values. Miss: -1 Normal throw: 0 Nice throw: 1 Great throw: 2 Excellent throw: 3  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-08-02. Score: 0
  
  Yes this is indeed a very plausible approach !
  Mind however, that RL is not very sample efficient, so you need a lot of runs for try and error.
  So it essentially boils down to the question whether you can provide/simulate a sufficient amount of samples to train the RL agent. If so, then I would recommend you to go with a very simple Q-learning approach. No deep neural networks are needed here.
  
  > **Comments:**
  > **username_0**: I could provide samples as I am playing. I was thinking basically to start with random throws and adjust the aim and velocity as I go accordingly to the Q-table.  
  
  ---
  
  