Answers.Body: <p>The core part of RL is the reward function. It belongs to the environment and it is the only way the agent can explore the world given a state.</p>
  <p>By constructing an appropriate reward function we can make an agent do exactly what we want.</p>
  <p><a href="https://github.com/karray/UnityReacherRL" rel="nofollow noreferrer">Here</a> is an example of a double-jointed arm that can move to target locations in the Unity environment.</p>
  <p>A reward of +0.1 is provided for each step that the agent's hand is in the goal location. So, the goal of the agent is to maintain its position at the target location for as many time steps as possible.</p>
  <p>Initially, the agent performs random actions (random forces on the joints) and eventually learns to follow the target. A high-level idea is described in this <a href="https://www.youtube.com/watch?v=pVIFc72VYH8" rel="nofollow noreferrer">video</a>. And here is also related DeepMind's <a href="https://deepmind.com/blog/article/producing-flexible-behaviours-simulated-environments" rel="nofollow noreferrer">article</a> and <a href="https://arxiv.org/pdf/1707.02286.pdf" rel="nofollow noreferrer">paper</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-06T14:46:26.143Z
Answers.LastActivityDate: 2022-09-06T14:46:26.143Z
Answers.UserDisplayName: Aray Karjauv
Body: <p>I'm working on a simulation of a motor that is attached to a wing (Later, this will also have a real-life counterpart once I'll assemble all the components in our lab), and I can control the forces/torques that the motor applies. I want to use RL and find an optimal action in terms of</p>
  <blockquote>
  <p>&quot;<em>what force should the motor apply to maximize lift</em>&quot;.</p>
  </blockquote>
  <p>To make things clearer, check out the following figure</p>
  <p>                                              <a href="https://i.sstatic.net/RWhxa.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/RWhxa.png" alt="enter image description here" /></a></p>
  <p>So for example, I can find <span class="math-container">$\phi$</span> at every single time step <span class="math-container">$t$</span> (using some 1st-year physics equations and python integrators) and this will be my state <span class="math-container">$s$</span>.</p>
  <p>Are you familiar with approaches that deal with this problem?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-06T13:09:43.917Z
Comments.Text: Hey, check this [answers](https://ai.stackexchange.com/questions/18425/how-can-i-implement-the-reward-function-for-an-8-dof-robot-arm-with-trpo/18436#18436). Also, take a look at these: [one](https://ai.stackexchange.com/questions/28291/how-would-you-shape-a-reward-function-if-there-was-four-quantities-to-optimize/28298#28298), [two](https://ai.stackexchange.com/questions/24157/how-should-i-define-the-reward-function-to-solve-the-wumpus-game-with-deep-q-lea/24164#24164). Did they answer your question?
Comments.UserDisplayName: Aray Karjauv
Tags: <reinforcement-learning><deep-rl><proximal-policy-optimization>
Title: How to use RL on a robotic moving arm?
UserDisplayName: Hadar Sharvit
ThreadText: # How to use RL on a robotic moving arm?
  
  (Asked by: username_0 on 2022-09-04)
  
  I'm working on a simulation of a motor that is attached to a wing (Later, this will also have a real-life counterpart once I'll assemble all the components in our lab), and I can control the forces/torques that the motor applies. I want to use RL and find an optimal action in terms of
  
  > "*what force should the motor apply to maximize lift*".
  
  To make things clearer, check out the following figure
  
  [![enter image description here](https://i.sstatic.net/RWhxa.png)](https://i.sstatic.net/RWhxa.png)
  
  So for example, I can find $\phi$ at every single time step $t$ (using some 1st-year physics equations and python integrators) and this will be my state $s$.
  
  Are you familiar with approaches that deal with this problem?
  
  **Tags:** reinforcement-learning, deep-rl, proximal-policy-optimization
  > **Comments:**
  > **username_1**: Hey, check this [answers](https://ai.stackexchange.com/questions/18425/how-can-i-implement-the-reward-function-for-an-8-dof-robot-arm-with-trpo/18436#18436). Also, take a look at these: [one](https://ai.stackexchange.com/questions/28291/how-would-you-shape-a-reward-function-if-there-was-four-quantities-to-optimize/28298#28298), [two](https://ai.stackexchange.com/questions/24157/how-should-i-define-the-reward-function-to-solve-the-wumpus-game-with-deep-q-lea/24164#24164). Did they answer your question?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-09-06. Score: 0
  
  The core part of RL is the reward function. It belongs to the environment and it is the only way the agent can explore the world given a state.
  
  By constructing an appropriate reward function we can make an agent do exactly what we want.
  
  [Here](https://github.com/karray/UnityReacherRL) is an example of a double-jointed arm that can move to target locations in the Unity environment.
  
  A reward of +0.1 is provided for each step that the agent's hand is in the goal location. So, the goal of the agent is to maintain its position at the target location for as many time steps as possible.
  
  Initially, the agent performs random actions (random forces on the joints) and eventually learns to follow the target. A high-level idea is described in this [video](https://www.youtube.com/watch?v=pVIFc72VYH8). And here is also related DeepMind's [article](https://deepmind.com/blog/article/producing-flexible-behaviours-simulated-environments) and [paper](https://arxiv.org/pdf/1707.02286.pdf).
  
  
  ---
  
  