Answers.Body: <p>This depends on your definition of robust.<br />
  Robust to what exactly?</p>
  <p>Testing different random seeds will test the robustness of the algorithm on stochasticity of the environment and the algorithm's optimization procedure.<br />
  Trying different hyperparameters would test the robustness of the algorithm to hyperparameter changes.</p>
  <p>Some RL benchmarks have their own definition of robustness:<br />
  The L2RPN benchmark (<a href="https://competitions.codalab.org/competitions/25426" rel="nofollow noreferrer">https://competitions.codalab.org/competitions/25426</a>) defines robustness as a policy that is able to respond properly when unexpected events or adversarial attacks happen. In benchmarks such as Atari or Procgen, which have multiple tasks, a robust algorithm is one that can solve all the tasks.</p>
  <p>If you mean robust, as in, the algorithm do indeed learn some inherent pattern on how to solve the task, as opposed to superficially memorizing sequences of actions, you could add noise to the observations (e.g. simple gaussian noise), try to add some sort of adversarial attacks, or try the sticky action idea used in Atari benchmarks.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-08T15:40:44.693Z
Answers.LastActivityDate: 2021-06-08T15:40:44.693Z
Answers.UserDisplayName: João Schapke
Body: <p>I have used the stable-baseline3 implementation of the SAC algorithm to train policies in a custom gym environment. So far the results look promising. However, I would like to test the robustness of the results. What are common ways to test robustness? So far, I have considered testing different seeds. Which other tests are recommended?</p>
  
Tags: <deep-rl><testing>
Title: How to test the robustness of an agent in a custom reinforcement learning environment?
UserDisplayName: Manuel
ThreadText: # How to test the robustness of an agent in a custom reinforcement learning environment?
  
  (Asked by: username_0 on 2021-06-08)
  
  I have used the stable-baseline3 implementation of the SAC algorithm to train policies in a custom gym environment. So far the results look promising. However, I would like to test the robustness of the results. What are common ways to test robustness? So far, I have considered testing different seeds. Which other tests are recommended?
  
  **Tags:** deep-rl, testing
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-06-08. Score: 1 | Accepted answer
  
  This depends on your definition of robust.  
  Robust to what exactly?
  
  Testing different random seeds will test the robustness of the algorithm on stochasticity of the environment and the algorithm's optimization procedure.  
  Trying different hyperparameters would test the robustness of the algorithm to hyperparameter changes.
  
  Some RL benchmarks have their own definition of robustness:  
  The L2RPN benchmark (<https://competitions.codalab.org/competitions/25426>) defines robustness as a policy that is able to respond properly when unexpected events or adversarial attacks happen. In benchmarks such as Atari or Procgen, which have multiple tasks, a robust algorithm is one that can solve all the tasks.
  
  If you mean robust, as in, the algorithm do indeed learn some inherent pattern on how to solve the task, as opposed to superficially memorizing sequences of actions, you could add noise to the observations (e.g. simple gaussian noise), try to add some sort of adversarial attacks, or try the sticky action idea used in Atari benchmarks.
  
  
  ---
  
  