Answers.Body: <p>The most important part of the RL is the reward function. If we want an agent to do something speciﬁc, we must provide rewards to it in such a way that it will achieve the goal. It is thus very important that the reward function accurately indicates the exact behavior.</p>
  <p>Assume, the robot's goal is to reach the desired position as fast as possible. You can construct your reward function so, that it will take into account the Euclidean distance to the position. If the arm moves to the position directly, you will reward the agent with a positive value, otherwise, you will punish it with a deviation from the direct line. You probably have other parameters of the joints, such as position and velocity. It can be also included in your reward function, in order to find optimal movements.</p>
  <p><a href="https://i.sstatic.net/newNu.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/newNu.png" alt="enter image description here" /></a></p>
  <p>Check out this <a href="https://www.youtube.com/watch?v=pVIFc72VYH8" rel="nofollow noreferrer">video</a> from the free udacity overview course on RL and this paper <a href="https://arxiv.org/pdf/1803.07067.pdf" rel="nofollow noreferrer">&quot;Setting up a Reinforcement Learning Task with a Real-World Robot&quot;</a></p>
  <p>Here is also related DeepMind's <a href="https://deepmind.com/blog/article/producing-flexible-behaviours-simulated-environments" rel="nofollow noreferrer">article</a> and <a href="https://arxiv.org/pdf/1707.02286.pdf" rel="nofollow noreferrer">paper</a></p>
  <p>I also have a <a href="https://github.com/karray/neuroracer" rel="nofollow noreferrer">project on github</a>, where I implemented custom Gazebo environment for OpenAI Gym. This allows you to run the test even on a Jupyter Notebook. Check out <a href="https://github.com/karray/neuroracer/blob/master/q_learning.ipynb" rel="nofollow noreferrer">my example</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-05T11:47:11.183Z
Answers.LastActivityDate: 2022-09-06T14:43:57.627Z
Answers.LastEditDate: 2022-09-06T14:43:57.627Z
Answers.UserDisplayName: Aray Karjauv
Body: <p>I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to write the code for the reward functions and the algorithm for the joint space motion. </p>
  
Tags: <reinforcement-learning><gym><reward-design><reward-functions><trust-region-policy-optimization>
Title: How can I implement the reward function for an 8-DOF robot arm with TRPO?
UserDisplayName: user1690356
ThreadText: # How can I implement the reward function for an 8-DOF robot arm with TRPO?
  
  (Asked by: username_0 on 2020-03-05)
  
  I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to write the code for the reward functions and the algorithm for the joint space motion.
  
  **Tags:** reinforcement-learning, gym, reward-design, reward-functions, trust-region-policy-optimization
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-03-05. Score: 1
  
  The most important part of the RL is the reward function. If we want an agent to do something speciﬁc, we must provide rewards to it in such a way that it will achieve the goal. It is thus very important that the reward function accurately indicates the exact behavior.
  
  Assume, the robot's goal is to reach the desired position as fast as possible. You can construct your reward function so, that it will take into account the Euclidean distance to the position. If the arm moves to the position directly, you will reward the agent with a positive value, otherwise, you will punish it with a deviation from the direct line. You probably have other parameters of the joints, such as position and velocity. It can be also included in your reward function, in order to find optimal movements.
  
  [![enter image description here](https://i.sstatic.net/newNu.png)](https://i.sstatic.net/newNu.png)
  
  Check out this [video](https://www.youtube.com/watch?v=pVIFc72VYH8) from the free udacity overview course on RL and this paper ["Setting up a Reinforcement Learning Task with a Real-World Robot"](https://arxiv.org/pdf/1803.07067.pdf)
  
  Here is also related DeepMind's [article](https://deepmind.com/blog/article/producing-flexible-behaviours-simulated-environments) and [paper](https://arxiv.org/pdf/1707.02286.pdf)
  
  I also have a [project on github](https://github.com/karray/neuroracer), where I implemented custom Gazebo environment for OpenAI Gym. This allows you to run the test even on a Jupyter Notebook. Check out [my example](https://github.com/karray/neuroracer/blob/master/q_learning.ipynb)
  
  
  ---
  
  