Body: <p>I am using a reinforcement learning model for some tasks.
  and for the model, I am using stable_baselin3 and for the environment, I am using the gym.</p>
  <p>I made a small change in the environment and the results became poor now when I rolled back that change the results didn't match the previous results even though I had made no change in the parameters or data.</p>
  <p>The seed of the model is fixed so there is no chance that this could be due to random initialization and I have tested this on my model before by running it multiple times to see if gives the same results for the same seed and it did.</p>
  <p>I checked that version of NumPy and the torch I used didn't change during my two runs.
  and I also tried changing them manually and the results were the same for all the versions.
  I am sure that the change I made is rolled back properly because it was a very small change.</p>
  <p>Can anyone suggest a list of things that can cause results to change that will help me debug the problem?</p>
  <p>I am setting seed for NumPy, torch(used by stable_baseline3), and random. I am not running anything in parallel. I am using the PPO model of stable_baseline3.</p>
  <p>I am using below-given function for seed, it is provided by the stable_baseline3 library.</p>
  <pre><code>def set_random_seed(seed: int, using_cuda: bool = False) -&gt; None:
      &quot;&quot;&quot;
      Seed the different random generators.
  
      :param seed:
      :param using_cuda:
      &quot;&quot;&quot;
      # Seed python RNG
      random.seed(seed)
      # Seed numpy RNG
      np.random.seed(seed)
      # seed the RNG for all devices (both CPU and CUDA)
      th.manual_seed(seed)
  
      if using_cuda:
          th.backends.cudnn.deterministic = True
          th.backends.cudnn.benchmark = False
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-17T12:37:54.697Z
Comments.Text: If you use NumPy, PyTorch and Regular python, you may well have 3 (or more) seeds needing setting to manage all the PRNGs. You don't say which seeds you are setting? Also, some libraries cope badly with RNG seeds in multi-threaded code, so worth mentioning if you are using multi-threading or multi-process to run environments in parallel.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-17T13:02:51.403Z
Comments.Text: @NeilSlater Thanks for the reply and suggestions. The thing is I checked running the code multiple times before, and results never changed for the same data and parameters, and I am not able to re-create this again it just happened once.
Comments.UserDisplayName: Maunish Dave
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-18T13:17:12.157Z
Comments.Text: This seems to be a programming issue, so it's off-topic here. We focus on the theoretical aspects of AI, which also includes RL, of course. See https://ai.stackexchange.com/help/on-topic for more details.
Comments.UserDisplayName: nbro
Tags: <machine-learning><reinforcement-learning><python>
Title: The results changed even though seed is fixed
UserDisplayName: Maunish Dave
ThreadText: # The results changed even though seed is fixed
  
  (Asked by: username_0 on 2022-01-17)
  
  I am using a reinforcement learning model for some tasks.
  and for the model, I am using stable_baselin3 and for the environment, I am using the gym.
  I made a small change in the environment and the results became poor now when I rolled back that change the results didn't match the previous results even though I had made no change in the parameters or data.
  The seed of the model is fixed so there is no chance that this could be due to random initialization and I have tested this on my model before by running it multiple times to see if gives the same results for the same seed and it did.
  I checked that version of NumPy and the torch I used didn't change during my two runs.
  and I also tried changing them manually and the results were the same for all the versions.
  I am sure that the change I made is rolled back properly because it was a very small change.
  Can anyone suggest a list of things that can cause results to change that will help me debug the problem?
  I am setting seed for NumPy, torch(used by stable_baseline3), and random. I am not running anything in parallel. I am using the PPO model of stable_baseline3.
  I am using below-given function for seed, it is provided by the stable_baseline3 library.
  def set_random_seed(seed: int, using_cuda: bool = False) -> None:
      """
      Seed the different random generators.
  
      :param seed:
      :param using_cuda:
      """
      # Seed python RNG
      random.seed(seed)
      # Seed numpy RNG
      np.random.seed(seed)
      # seed the RNG for all devices (both CPU and CUDA)
      th.manual_seed(seed)
  
      if using_cuda:
          th.backends.cudnn.deterministic = True
          th.backends.cudnn.benchmark = False
  
  **Tags:** machine-learning, reinforcement-learning, python
  > **Comments:**
  > **username_1**: If you use NumPy, PyTorch and Regular python, you may well have 3 (or more) seeds needing setting to manage all the PRNGs. You don't say which seeds you are setting? Also, some libraries cope badly with RNG seeds in multi-threaded code, so worth mentioning if you are using multi-threading or multi-process to run environments in parallel.  
  > **username_0**: @NeilSlater Thanks for the reply and suggestions. The thing is I checked running the code multiple times before, and results never changed for the same data and parameters, and I am not able to re-create this again it just happened once.  
  > **username_2**: This seems to be a programming issue, so it's off-topic here. We focus on the theoretical aspects of AI, which also includes RL, of course. See https://ai.stackexchange.com/help/on-topic for more details.  
  
  ---
  
  ## 0 answers
  
  