Answers.Body: <p>See <a href="https://gymnasium.farama.org/environments/mujoco/humanoid_standup/#rewards" rel="nofollow noreferrer">https://gymnasium.farama.org/environments/mujoco/humanoid_standup/#rewards</a> for a description of how to get a humanoid ragdoll to stand up!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-16T11:09:39.093Z
Answers.Text: Note that I have got this far so far myself...  https://www.youtube.com/watch?v=ISZ_xgybazQ  (it reliably rolls over onto it's front)
Answers.UserDisplayName: ChrisSim
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-17T14:28:58.970Z
Answers.Text: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-16T11:06:12.450Z
Answers.LastActivityDate: 2023-04-16T11:06:12.450Z
Answers.UserDisplayName: ChrisSim
Body: <p>I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest amount of time.</p>
  
  <p>But I'm having trouble in regards to reward shaping. I've tried multiple different reward functions, but they all end up the same way: the agent quickly learns to sit (i.e. lifting its torso), but then gets stuck on this local optimum forever.</p>
  
  <p>Any ideas or advice on how to best design a good reward function for this scenario?</p>
  
  <p>A few reward functions I've tried so far:</p>
  
  <ul>
  <li><em>current_height / goal_height</em></li>
  <li><em>current_height / goal_height - 1</em></li>
  <li><em>current_height / goal_height - reward_prev_timestep</em></li>
  <li><em>(current_height / goal_height)^N (tried multiple different values of N)</em></li>
  <li>...</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-20T17:14:26.127Z
Comments.Text: In section 6.2.2 of [this](https://arxiv.org/pdf/1506.02438.pdf) paper they provide some reward functions so you can try with these.
Comments.UserDisplayName: Brale
Tags: <reinforcement-learning><reward-functions><reward-design><reward-shaping>
Title: How define a reward function for a humanoid agent whose goal is to stand up from the ground?
UserDisplayName: Tirafesi
ThreadText: # How define a reward function for a humanoid agent whose goal is to stand up from the ground?
  
  (Asked by: username_0 on 2019-05-17)
  
  I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest amount of time.
  
  But I'm having trouble in regards to reward shaping. I've tried multiple different reward functions, but they all end up the same way: the agent quickly learns to sit (i.e. lifting its torso), but then gets stuck on this local optimum forever.
  
  Any ideas or advice on how to best design a good reward function for this scenario?
  
  A few reward functions I've tried so far:
  
  * *current\_height / goal\_height*
  * *current\_height / goal\_height - 1*
  * *current\_height / goal\_height - reward\_prev\_timestep*
  * *(current\_height / goal\_height)^N (tried multiple different values of N)*
  * ...
  
  **Tags:** reinforcement-learning, reward-functions, reward-design, reward-shaping
  
  > **Comments:**
  > **username_1**: In section 6.2.2 of [this](https://arxiv.org/pdf/1506.02438.pdf) paper they provide some reward functions so you can try with these.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-04-16. Score: 0
  
  See <https://gymnasium.farama.org/environments/mujoco/humanoid_standup/#rewards> for a description of how to get a humanoid ragdoll to stand up!
  
  > **Comments:**
  > **username_2**: Note that I have got this far so far myself...  https://www.youtube.com/watch?v=ISZ_xgybazQ  (it reliably rolls over onto it's front)  
  > **username_3**: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  
  ---
  
  