Answers.Body: <p>I don't think there's a strategy that applies to all cases. In some cases, the reward function may need to be carefully designed (e.g. a self-driving car), but, in other cases, the reward function might be quickly designed (e.g. chess) and other parts of the RL system may require more care.</p>
  <p><a href="https://github.com/williamFalcon/DeepRLHacks" rel="nofollow noreferrer">Here</a> you can find tips to approach an RL problem. For example, one tip that I find useful, although probably obvious once you hear of it, is to compare your policy with a random policy. See also <a href="https://github.com/Stable-Baselines-Team/stable-baselines/blob/master/docs/guide/rl_tips.rst" rel="nofollow noreferrer">these tips</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-16T16:23:16.603Z
Answers.Text: I wanted to infer the amount of time spent for each component by some literature, but this seems not to work out then. Anyway, the links you provided are very useful for designing the task, thank you very much!
Answers.UserDisplayName: kitaird
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-13T09:38:46.647Z
Answers.LastActivityDate: 2021-12-13T09:38:46.647Z
Answers.UserDisplayName: nbro
Body: <p>I'm working on my thesis concerning a reinforcement learning problem and am trying to prioritise my time on different components of it:</p>
  <ul>
  <li>Formalising the agent environment (like the design of state-, action-space and reward-structure)</li>
  <li>Selection of learning algorithm</li>
  <li>Selection of network architecture and size</li>
  <li>Design of the training setup</li>
  </ul>
  <p>It is an agent in a 3D environment with simulated physics (in Unity), the domain being a real-time strategical game. It is an environment with constraint training data, so sample efficiency is very important.</p>
  <p>Now my question: I do anticipate that the design of the state- and action space will have a big impact on the training result, especially in this environment with little training data.</p>
  <p>However, is there a way one can clearly prioritise what components will be the most important ones for an RL setting?</p>
  <p>Time is limited, and, for me, as a beginner, it seems to be quite difficult to determine what component will be the most important one and needs the most focus. Testing only the hyper-parameters of a learning algorithm thoroughly will take in itself a long time. And obviously disregarding any component will result in bad results.</p>
  <p>Is there a way to know on which component one should focus more?</p>
  
Tags: <reinforcement-learning><ai-design><hyperparameter-optimization><hyper-parameters><environment>
Title: What components of reinforcement learning influence the result the most?
UserDisplayName: kitaird
ThreadText: # What components of reinforcement learning influence the result the most?
  
  (Asked by: username_0 on 2021-12-12)
  
  I'm working on my thesis concerning a reinforcement learning problem and am trying to prioritise my time on different components of it:
  
  * Formalising the agent environment (like the design of state-, action-space and reward-structure)
  * Selection of learning algorithm
  * Selection of network architecture and size
  * Design of the training setup
  
  It is an agent in a 3D environment with simulated physics (in Unity), the domain being a real-time strategical game. It is an environment with constraint training data, so sample efficiency is very important.
  
  Now my question: I do anticipate that the design of the state- and action space will have a big impact on the training result, especially in this environment with little training data.
  
  However, is there a way one can clearly prioritise what components will be the most important ones for an RL setting?
  
  Time is limited, and, for me, as a beginner, it seems to be quite difficult to determine what component will be the most important one and needs the most focus. Testing only the hyper-parameters of a learning algorithm thoroughly will take in itself a long time. And obviously disregarding any component will result in bad results.
  
  Is there a way to know on which component one should focus more?
  
  **Tags:** reinforcement-learning, ai-design, hyperparameter-optimization, hyper-parameters, environment
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-12-13. Score: 1 | Accepted answer
  
  I don't think there's a strategy that applies to all cases. In some cases, the reward function may need to be carefully designed (e.g. a self-driving car), but, in other cases, the reward function might be quickly designed (e.g. chess) and other parts of the RL system may require more care.
  
  [Here](https://github.com/williamFalcon/DeepRLHacks) you can find tips to approach an RL problem. For example, one tip that I find useful, although probably obvious once you hear of it, is to compare your policy with a random policy. See also [these tips](https://github.com/Stable-Baselines-Team/stable-baselines/blob/master/docs/guide/rl_tips.rst).
  
  > **Comments:**
  > **username_0**: I wanted to infer the amount of time spent for each component by some literature, but this seems not to work out then. Anyway, the links you provided are very useful for designing the task, thank you very much!  
  
  ---
  
  