Answers.Body: <p>I was just reading Sutton and Barto last night and a passage stuck with me which I will paraphrase.  RL is marked by the evaluation of an action, not the instruction of an action.  So I would say if the correct mixture of units is not known before hand and must be learned by evaluation then I would say that RL is correct.</p>
  <p>The first decision point I'd consider on determining which would be reasonable choice in methods would be the input type/range.</p>
  <p>I assume the enemy must deploy a whole, positive number of units (ie integer).  In this case Q-table might be okay.  The decision point would be how many values these inputs can take on.  A Q-Table stores the Q value for every state, so as the number of states grow, so does memory requirements.</p>
  <p>If the range of the inputs gets significantly large (or the inputs are real valued), one should consider a function approximator (chiefly NN in DQN/PPO but there are alternatives).  Which exact function approximator and how to train it will be along ideological/practical lines.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-20T11:53:26.000Z
Answers.Text: Great deduction, especially on the DQN/PPO part. What I can't  get my head around is the time aspect, which I associate with the training. In my problem the time would solely pass in the environment (for each composition). Does this still fit the RL frame ?
Answers.UserDisplayName: Astraeus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-21T02:24:54.220Z
Answers.Text: Are you saying that the composition is chosen at the start of the round, time passes until the end of the round and a winner is chosen?  But the agent can only act/perceive between rounds?
Answers.UserDisplayName: foreverska
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-21T08:36:25.737Z
Answers.Text: Yes exactly, kinda a one shot round.
Answers.UserDisplayName: Astraeus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-21T13:53:11.183Z
Answers.Text: Sure, rock, paper, scissors except you must determine how much of each to deploy.  Might start there, I think Kaggle has an  exercise for rock, paper scissors.
Answers.UserDisplayName: foreverska
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-21T13:55:54.530Z
Answers.Text: You might look at portfolio allocation examples for chatter on how to do this mixture of resources thing.
Answers.UserDisplayName: foreverska
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-21T14:10:10.220Z
Answers.Text: Thank you for the pointer mate, will have a look.
Answers.UserDisplayName: Astraeus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-14T18:18:08.150Z
Answers.LastActivityDate: 2024-02-14T18:18:08.150Z
Answers.UserDisplayName: foreverska
Body: <p>I want to use RL to solve the problem of Unit Allocation.</p>
  <p>SETTING: So for example in a game you can use units of type A and units of type B and you know that your opposing player has units of type C and type D. Now given the knowledge of the opposing units the neural net should give me the composition of my units to deploy against them. Let's say x of type A and y of type B.</p>
  <p>PROBLEM Since I have an environment to play and score the outcome my idea was to use Reinforcement Learning. Resulting in my first question, would this be the right choice of method ?</p>
  <p>I searched for resource allocation problems and all I found was some Communication Problems, where I can't fit my problem.</p>
  <p>What type of RL would be the best choice here? DQL/Q-Learning or PPO?</p>
  
Tags: <reinforcement-learning><ai-design><policy-gradients>
Title: Should I use RL for this allocation problem and, if yes, which RL approach?
UserDisplayName: Astraeus
ThreadText: # Should I use RL for this allocation problem and, if yes, which RL approach?
  
  (Asked by: username_0 on 2024-02-09)
  
  I want to use RL to solve the problem of Unit Allocation.
  
  SETTING: So for example in a game you can use units of type A and units of type B and you know that your opposing player has units of type C and type D. Now given the knowledge of the opposing units the neural net should give me the composition of my units to deploy against them. Let's say x of type A and y of type B.
  
  PROBLEM Since I have an environment to play and score the outcome my idea was to use Reinforcement Learning. Resulting in my first question, would this be the right choice of method ?
  
  I searched for resource allocation problems and all I found was some Communication Problems, where I can't fit my problem.
  
  What type of RL would be the best choice here? DQL/Q-Learning or PPO?
  
  **Tags:** reinforcement-learning, ai-design, policy-gradients
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-02-14. Score: 3 | Accepted answer
  
  I was just reading Sutton and Barto last night and a passage stuck with me which I will paraphrase. RL is marked by the evaluation of an action, not the instruction of an action. So I would say if the correct mixture of units is not known before hand and must be learned by evaluation then I would say that RL is correct.
  
  The first decision point I'd consider on determining which would be reasonable choice in methods would be the input type/range.
  
  I assume the enemy must deploy a whole, positive number of units (ie integer). In this case Q-table might be okay. The decision point would be how many values these inputs can take on. A Q-Table stores the Q value for every state, so as the number of states grow, so does memory requirements.
  
  If the range of the inputs gets significantly large (or the inputs are real valued), one should consider a function approximator (chiefly NN in DQN/PPO but there are alternatives). Which exact function approximator and how to train it will be along ideological/practical lines.
  
  > **Comments:**
  > **username_0**: Great deduction, especially on the DQN/PPO part. What I can't  get my head around is the time aspect, which I associate with the training. In my problem the time would solely pass in the environment (for each composition). Does this still fit the RL frame ?  
  > **username_1**: Are you saying that the composition is chosen at the start of the round, time passes until the end of the round and a winner is chosen?  But the agent can only act/perceive between rounds?  
  > **username_0**: Yes exactly, kinda a one shot round.  
  > **username_1**: Sure, rock, paper, scissors except you must determine how much of each to deploy.  Might start there, I think Kaggle has an  exercise for rock, paper scissors.  
  > **username_1**: You might look at portfolio allocation examples for chatter on how to do this mixture of resources thing.  
  > **username_0**: Thank you for the pointer mate, will have a look.  
  
  ---
  
  