Answers.Body: <p>The expression &quot;number of actions&quot; is being used in the same way in both cases. In fact, the letter <span class="math-container">$m$</span> is used in both cases. The number of actions (in the state <span class="math-container">$s$</span>) is the number of possible actions that you can take in the state <span class="math-container">$s$</span>. So, here, <span class="math-container">$m$</span> does not refer to the <strong>dimensionality</strong> of an action, but to the size of the action set for a state. So, if <span class="math-container">$A(s)$</span> is the set of actions that you can take in <span class="math-container">$s$</span>, then <span class="math-container">$m = |A(s)|$</span>. <a href="https://www.ri.cmu.edu/pub_files/pub1/thrun_sebastian_1993_1/thrun_sebastian_1993_1.pdf" rel="nofollow noreferrer">The paper</a> that they refer to in the first excerpt actually uses (on page 3, in the lemma) <span class="math-container">$n$</span> to refer to <span class="math-container">$m$</span> and states that <span class="math-container">$n$</span> is the number of actions. <a href="https://ai.stackexchange.com/q/9812/2444">Here</a> there's a related question, which you may want to read.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-07T19:07:35.743Z
Answers.LastActivityDate: 2022-01-07T19:07:35.743Z
Answers.UserDisplayName: nbro
Body: <p>In the original DDQN article (<a href="https://arxiv.org/pdf/1509.06461.pdf," rel="nofollow noreferrer">https://arxiv.org/pdf/1509.06461.pdf,</a>) the phrase &quot;number of actions&quot; is used twice;</p>
  <p>First, in the following context:
  <a href="https://i.sstatic.net/78OXt.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/78OXt.png" alt="enter image description here" /></a></p>
  <p>Secondly in Theorem 1.</p>
  <p><a href="https://i.sstatic.net/K9diL.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/K9diL.png" alt="enter image description here" /></a></p>
  <p>I have a hard time understanding the way the phrase is being used or if it is being used in the same way, could anyone care to explain how it is meant to be understood?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-07T19:40:56.997Z
Comments.Text: nbro's answer is correct; I don't think I've ever seen anyone in RL literature referring to the number of actions that have already been taken as just the "number of actions" (they'd have to explicitly add something like "that have already been taken" to make that clear). Normally the latter would just be referred to as the time step $t$ instead.
Comments.UserDisplayName: Dennis Soemers
Tags: <deep-learning><papers><dqn><double-dqn><double-q-learning>
Title: Does "number of actions" refer to the number of actions taken or size of the action space?
UserDisplayName: GeorgeWTrump
ThreadText: # Does "number of actions" refer to the number of actions taken or size of the action space?
  
  (Asked by: username_0 on 2022-01-06)
  
  In the original DDQN article (<https://arxiv.org/pdf/1509.06461.pdf,>) the phrase "number of actions" is used twice;
  
  First, in the following context:
  [![enter image description here](https://i.sstatic.net/78OXt.png)](https://i.sstatic.net/78OXt.png)
  
  Secondly in Theorem 1.
  
  [![enter image description here](https://i.sstatic.net/K9diL.png)](https://i.sstatic.net/K9diL.png)
  
  I have a hard time understanding the way the phrase is being used or if it is being used in the same way, could anyone care to explain how it is meant to be understood?
  
  **Tags:** deep-learning, papers, dqn, double-dqn, double-q-learning
  
  > **Comments:**
  > **username_1**: username_2's answer is correct; I don't think I've ever seen anyone in RL literature referring to the number of actions that have already been taken as just the "number of actions" (they'd have to explicitly add something like "that have already been taken" to make that clear). Normally the latter would just be referred to as the time step $t$ instead.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-01-07. Score: 1 | Accepted answer
  
  The expression "number of actions" is being used in the same way in both cases. In fact, the letter $m$ is used in both cases. The number of actions (in the state $s$) is the number of possible actions that you can take in the state $s$. So, here, $m$ does not refer to the **dimensionality** of an action, but to the size of the action set for a state. So, if $A(s)$ is the set of actions that you can take in $s$, then $m = |A(s)|$. [The paper](https://www.ri.cmu.edu/pub_files/pub1/thrun_sebastian_1993_1/thrun_sebastian_1993_1.pdf) that they refer to in the first excerpt actually uses (on page 3, in the lemma) $n$ to refer to $m$ and states that $n$ is the number of actions. [Here](https://ai.stackexchange.com/q/9812/2444) there's a related question, which you may want to read.
  
  
  ---
  
  