Answers.Body: <p>The environment needs to return immediate reward only, from the <code>step</code> function. It's the agent's task to sum received rewards, the environment should not normally do that.</p>
  <p>In some environments it may make sense to add to something that accumulated on actions and influences reward. E.g. perhaps you want to simulate investments where value accumulates over time. You would want to store that as an instance variable for the environment, and reset on episode start. However, rather than call that <code>reward</code>, name it after whatever it represents within your environment model. Convert it to immediate reward received from the <code>step</code> function as needed.</p>
  <p>In short, the second approach you show, where reward is reset and calculated on every call to <code>step</code> is the standard approach that you should follow in most cases.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-30T08:40:41.057Z
Answers.LastActivityDate: 2024-01-30T08:40:41.057Z
Answers.UserDisplayName: Neil Slater
Body: <p>I am using PPO to train my environment which I created using stable baselines 3. I am confused if I should make the reward = 0 in the step function or not.</p>
  <p>Initially, I used to have self.reward = 0 in the <strong>init</strong>() and reset() function and I used to calculate reward in the step function like this.</p>
  <pre><code>def __init__():
      self.reward = 0
  def reset():
      self.reward = 0
  def step():
      self.reward = self.reward + slope + calculation
  </code></pre>
  <p>But I have seen some implementation where people do this</p>
  <pre><code>    def __init__():
        
      def reset():
          
      def step():
          reward = 0
          reward = slope + calculation
  </code></pre>
  <p>What is the right way of doing it and how does it work?</p>
  
Tags: <reinforcement-learning><python><proximal-policy-optimization>
Title: How does reward work while training a Reinforcement Learning agent?
UserDisplayName: user79474
ThreadText: # How does reward work while training a Reinforcement Learning agent?
  
  (Asked by: username_0 on 2024-01-29)
  
  I am using PPO to train my environment which I created using stable baselines 3. I am confused if I should make the reward = 0 in the step function or not.
  
  Initially, I used to have self.reward = 0 in the **init**() and reset() function and I used to calculate reward in the step function like this.
  
  ```
  def __init__():
      self.reward = 0
  def reset():
      self.reward = 0
  def step():
      self.reward = self.reward + slope + calculation
  
  ```
  
  But I have seen some implementation where people do this
  
  ```
      def __init__():
        
      def reset():
          
      def step():
          reward = 0
          reward = slope + calculation
  
  ```
  
  What is the right way of doing it and how does it work?
  
  **Tags:** reinforcement-learning, python, proximal-policy-optimization
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-01-30. Score: 1
  
  The environment needs to return immediate reward only, from the `step` function. It's the agent's task to sum received rewards, the environment should not normally do that.
  
  In some environments it may make sense to add to something that accumulated on actions and influences reward. E.g. perhaps you want to simulate investments where value accumulates over time. You would want to store that as an instance variable for the environment, and reset on episode start. However, rather than call that `reward`, name it after whatever it represents within your environment model. Convert it to immediate reward received from the `step` function as needed.
  
  In short, the second approach you show, where reward is reset and calculated on every call to `step` is the standard approach that you should follow in most cases.
  
  
  ---
  
  