Answers.Body: <p>A2C is a policy gradient method, that learns a policy <span class="math-container">$\pi(a\mid s)$</span> directly.</p>
  <p>This means that the actions are sampled from a probability distribution. In practice, you can model a discrete action space with a <strong>Categorical distribution</strong> or, in case you experience gradient issues, with a <em>Gumbel-Softmax</em> (also called Concrete) distribution which is its continuous relaxation.</p>
  <p>Lastly, unless <code>NUM_PRICES</code> is very large you shouldn't experience issue about the action space size.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-24T20:37:34.067Z
Answers.LastActivityDate: 2023-05-24T20:37:34.067Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>I am working on a problem where I need to generate a list of 0 - 8 different prices as the action space where generating 0 prices represents doing nothing</p>
  <pre><code>NUM_PRICES = 8
  action_space = [i for i in range(NUM_PRICES + 1)]  # Include 0 as an option
  
  # Sample an action from the action space
  action = np.random.choice(action_space)
  
  if action == 0:
      prices = []
  else:
      prices = np.random.uniform(-10.0, 200, action)
  </code></pre>
  <p>I have something like this where prices would be the generated action.
  Would something like this work with the A2C algorithm?
  I'm unfamiliar with A2C and have only worked with DQN and QLearning
  So I'm worried maybe the action space is too complex</p>
  
Tags: <reinforcement-learning><advantage-actor-critic>
Title: Action space for the A2C algorithm
UserDisplayName: willem12
ThreadText: # Action space for the A2C algorithm
  
  (Asked by: username_0 on 2023-05-16)
  
  I am working on a problem where I need to generate a list of 0 - 8 different prices as the action space where generating 0 prices represents doing nothing
  NUM_PRICES = 8
  action_space = [i for i in range(NUM_PRICES + 1)]  # Include 0 as an option
  
  # Sample an action from the action space
  action = np.random.choice(action_space)
  
  if action == 0:
      prices = []
  else:
      prices = np.random.uniform(-10.0, 200, action)
  
  I have something like this where prices would be the generated action.
  Would something like this work with the A2C algorithm?
  I'm unfamiliar with A2C and have only worked with DQN and QLearning
  So I'm worried maybe the action space is too complex
  
  **Tags:** reinforcement-learning, advantage-actor-critic
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-05-24. Score: 0 | Accepted answer
  
  A2C is a policy gradient method, that learns a policy $\pi(a\mid s)$ directly.
  
  This means that the actions are sampled from a probability distribution. In practice, you can model a discrete action space with a **Categorical distribution** or, in case you experience gradient issues, with a *Gumbel-Softmax* (also called Concrete) distribution which is its continuous relaxation.
  
  Lastly, unless `NUM_PRICES` is very large you shouldn't experience issue about the action space size.
  
  
  ---
  
  