Answers.Body: <p>Theoretically video games and robotics problems are also about optimization(getting maximum reward). So, just like other reinforcement learning problems, I would expect PPO to be the most efficient in your case too. I don't think a "goal" is necessary for rl, all you need is the rewards.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-02T20:02:41.687Z
Answers.Text: In RL, the reward defines the goal. RL isn't the most appropriate approach to solve many optimization problems, because it can be very expensive to train good agents for real-world problems. Games are actually a very limited scenario. Even games like Go, which are combinatorially difficult to solve, are limited, in the sense that there aren't many external sources of noise, etc.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-02T05:49:12.257Z
Answers.LastActivityDate: 2020-04-02T19:54:57.113Z
Answers.LastEditDate: 2020-04-02T19:54:57.113Z
Answers.UserDisplayName: Joel Joseph
Body: <p>My task is to solve an optimization problem with deep reinforcement learning. I read about several algorithms like DQN, PPO, DDPG, and A2C/A3C but use cases always seem to be problems like video games (sparse rewards, etc.) or robotics (continuous action spaces, etc.). Since my problem is an optimization issue, I wonder which algorithm is appropriate for my setting:</p>
  
  <ul>
  <li>limited number of discrete actions (like 20)</li>
  <li>high-dimensional states (like 250 values)</li>
  <li>instant reward after every single action (not only at the end of an episode)</li>
  <li>a single action can affect the state quite a lot</li>
  </ul>
  
  <p>There's no "goal" like in a video game, an episode ends after a certain number of actions. I'm not quite sure which algorithm is appropriate for my use case.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-30T13:12:42.770Z
Comments.Text: How long is the average episode? And on how many episodes will you train?
Comments.UserDisplayName: BlindKungFuMaster
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-30T14:48:26.993Z
Comments.Text: Getting feedback from the environment is pretty slow so there's a max. number of actions in each episode. The number greatly depends on the hyperparameters so I can't really tell for now.
Comments.UserDisplayName: annow
Tags: <reinforcement-learning><optimization><deep-rl>
Title: Which deep reinforcement learning algorithm is appropriate for my problem?
UserDisplayName: annow
ThreadText: # Which deep reinforcement learning algorithm is appropriate for my problem?
  
  (Asked by: username_0 on 2020-03-30)
  
  My task is to solve an optimization problem with deep reinforcement learning. I read about several algorithms like DQN, PPO, DDPG, and A2C/A3C but use cases always seem to be problems like video games (sparse rewards, etc.) or robotics (continuous action spaces, etc.). Since my problem is an optimization issue, I wonder which algorithm is appropriate for my setting:
  
  * limited number of discrete actions (like 20)
  * high-dimensional states (like 250 values)
  * instant reward after every single action (not only at the end of an episode)
  * a single action can affect the state quite a lot
  
  There's no "goal" like in a video game, an episode ends after a certain number of actions. I'm not quite sure which algorithm is appropriate for my use case.
  
  **Tags:** reinforcement-learning, optimization, deep-rl
  
  > **Comments:**
  > **username_1**: How long is the average episode? And on how many episodes will you train?  
  > **username_0**: Getting feedback from the environment is pretty slow so there's a max. number of actions in each episode. The number greatly depends on the hyperparameters so I can't really tell for now.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-04-02. Score: 1
  
  Theoretically video games and robotics problems are also about optimization(getting maximum reward). So, just like other reinforcement learning problems, I would expect PPO to be the most efficient in your case too. I don't think a "goal" is necessary for rl, all you need is the rewards.
  
  > **Comments:**
  > **username_3**: In RL, the reward defines the goal. RL isn't the most appropriate approach to solve many optimization problems, because it can be very expensive to train good agents for real-world problems. Games are actually a very limited scenario. Even games like Go, which are combinatorially difficult to solve, are limited, in the sense that there aren't many external sources of noise, etc.  
  
  ---
  
  