<p>Theoretically video games and robotics problems are also about optimization(getting maximum reward). So, just like other reinforcement learning problems, I would expect PPO to be the most efficient in your case too. I don't think a "goal" is necessary for rl, all you need is the rewards.</p>

CC BY-SA 4.0
2020-04-02T20:02:41.687Z
In RL, the reward defines the goal. RL isn't the most appropriate approach to solve many optimization problems, because it can be very expensive to train good agents for real-world problems. Games are actually a very limited scenario. Even games like Go, which are combinatorially difficult to solve, are limited, in the sense that there aren't many external sources of noise, etc.
nbro
CC BY-SA 4.0
2020-04-02T05:49:12.257Z
2020-04-02T19:54:57.113Z
2020-04-02T19:54:57.113Z
Joel Joseph
<p>My task is to solve an optimization problem with deep reinforcement learning. I read about several algorithms like DQN, PPO, DDPG, and A2C/A3C but use cases always seem to be problems like video games (sparse rewards, etc.) or robotics (continuous action spaces, etc.). Since my problem is an optimization issue, I wonder which algorithm is appropriate for my setting:</p>

<ul>
<li>limited number of discrete actions (like 20)</li>
<li>high-dimensional states (like 250 values)</li>
<li>instant reward after every single action (not only at the end of an episode)</li>
<li>a single action can affect the state quite a lot</li>
</ul>

<p>There's no "goal" like in a video game, an episode ends after a certain number of actions. I'm not quite sure which algorithm is appropriate for my use case.</p>

CC BY-SA 4.0
2020-03-30T13:12:42.770Z
How long is the average episode? And on how many episodes will you train?
BlindKungFuMaster
CC BY-SA 4.0
2020-03-30T14:48:26.993Z
Getting feedback from the environment is pretty slow so there's a max. number of actions in each episode. The number greatly depends on the hyperparameters so I can't really tell for now.
annow
<reinforcement-learning><optimization><deep-rl>
Which deep reinforcement learning algorithm is appropriate for my problem?
annow
# Which deep reinforcement learning algorithm is appropriate for my problem?

(Asked by: username_0 on 2020-03-30)

My task is to solve an optimization problem with deep reinforcement learning. I read about several algorithms like DQN, PPO, DDPG, and A2C/A3C but use cases always seem to be problems like video games (sparse rewards, etc.) or robotics (continuous action spaces, etc.). Since my problem is an optimization issue, I wonder which algorithm is appropriate for my setting:

* limited number of discrete actions (like 20)
* high-dimensional states (like 250 values)
* instant reward after every single action (not only at the end of an episode)
* a single action can affect the state quite a lot

There's no "goal" like in a video game, an episode ends after a certain number of actions. I'm not quite sure which algorithm is appropriate for my use case.

**Tags:** reinforcement-learning, optimization, deep-rl

> **Comments:**
> **username_1**: How long is the average episode? And on how many episodes will you train?  
> **username_0**: Getting feedback from the environment is pretty slow so there's a max. number of actions in each episode. The number greatly depends on the hyperparameters so I can't really tell for now.  

---

## 1 answers

### Answer by username_2 on 2020-04-02. Score: 1

Theoretically video games and robotics problems are also about optimization(getting maximum reward). So, just like other reinforcement learning problems, I would expect PPO to be the most efficient in your case too. I don't think a "goal" is necessary for rl, all you need is the rewards.

> **Comments:**
> **username_3**: In RL, the reward defines the goal. RL isn't the most appropriate approach to solve many optimization problems, because it can be very expensive to train good agents for real-world problems. Games are actually a very limited scenario. Even games like Go, which are combinatorially difficult to solve, are limited, in the sense that there aren't many external sources of noise, etc.  

---

