Answers.Body: <p>There is no one-size-fits-all answer to this question, as the best reward function for encouraging exploration will vary depending on the specifics of the AI system. However, some possible reward functions that could encourage exploration include giving the AI a small reward for each new state it visits, or giving the AI a large reward for discovering new states that are particularly valuable.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-16T21:46:14.487Z
Answers.LastActivityDate: 2022-10-16T21:46:14.487Z
Answers.UserDisplayName: Faizy
Body: <p>I am new to Reinforcement Learning. I am trying to train PPO agent for <a href="https://www.aicrowd.com/challenges/neurips-2022-citylearn-challenge" rel="nofollow noreferrer">citylearn</a>. The goal is to lower two environmental variables from observations. The default reward function is</p>
  <p><code>reward = (variable1 + variable2)*-1</code></p>
  <p>The agent is providing actions with values that are close to -1 for higher iterations. Instead of attempting to improve the environment, it only tries to avoid high negative rewards. I've tried normalizing observations and rewards, but it's still been ineffective.</p>
  <p>Is there a reward function that would encourage exploration rather than the same behavior?</p>
  
Tags: <reinforcement-learning><proximal-policy-optimization><reward-functions><reward-design>
Title: Is there a reward function that would encourage exploration in this case?
UserDisplayName: Sai Dinesh Pola
ThreadText: # Is there a reward function that would encourage exploration in this case?
  
  (Asked by: username_0 on 2022-10-14)
  
  I am new to Reinforcement Learning. I am trying to train PPO agent for [citylearn](https://www.aicrowd.com/challenges/neurips-2022-citylearn-challenge). The goal is to lower two environmental variables from observations. The default reward function is
  
  `reward = (variable1 + variable2)*-1`
  
  The agent is providing actions with values that are close to -1 for higher iterations. Instead of attempting to improve the environment, it only tries to avoid high negative rewards. I've tried normalizing observations and rewards, but it's still been ineffective.
  
  Is there a reward function that would encourage exploration rather than the same behavior?
  
  **Tags:** reinforcement-learning, proximal-policy-optimization, reward-functions, reward-design
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-10-16. Score: 2
  
  There is no one-size-fits-all answer to this question, as the best reward function for encouraging exploration will vary depending on the specifics of the AI system. However, some possible reward functions that could encourage exploration include giving the AI a small reward for each new state it visits, or giving the AI a large reward for discovering new states that are particularly valuable.
  
  
  ---
  
  