Answers.Body: <p>Let me rephrase it a little - it's a multidimensional continuous space of actions. So, you assign each action some vector from <span class="math-container">$R^{n}$</span>. For intuition -- imagine you have a robot arm with four joints. For every joint you could applied a rotation force from [-1, 1] and thus you get a 4-D vector with float numbers for each possible action.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-01T14:19:36.980Z
Answers.LastActivityDate: 2020-05-01T14:28:46.420Z
Answers.LastEditDate: 2020-05-01T14:28:46.420Z
Answers.UserDisplayName: Kirill Fedyanin
Answers.Body: <p>The question has already been answered by Kirill, but I thought I'll add a good example of a multi-dimensional continuous action space too, namely the one I just encountered in the COBRA paper itself.</p>
  <blockquote>
  <p>In all of our experiments we use a 2-dimensional virtual &quot;touch-screen&quot; environment that contains objects with configurable shape, position, and color. The agent can move any visible object by clicking on the object and clicking in a direction for the object to move. Hence the action space is <strong>continuous and 4-dimensional, namely a pair of clicks</strong>.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-01T14:42:15.907Z
Answers.LastActivityDate: 2021-01-09T22:23:26.697Z
Answers.LastEditDate: 2021-01-09T22:23:26.697Z
Answers.UserDisplayName: stoic-santiago
Body: <p>In the context of Reinforcement Learning, <strong>what does it mean to have a multi-dimensional continuous action space?</strong></p>
  <p>I came across the following in the <a href="https://arxiv.org/abs/1905.09275" rel="nofollow noreferrer">COBRA Paper</a></p>
  <blockquote>
  <p>A method for learning a distribution over a <strong>multi-dimensional continuous action space.</strong> This learned distribution can be sampled efficiently.</p>
  </blockquote>
  <p>and</p>
  <blockquote>
  <p>During the initial exploration phase it explores its environment, in which it can move objects freely with a <strong>continuous action space</strong> but is not rewarded for its actions.</p>
  </blockquote>
  <p>So, what do the multi-dimensionality and the continuity of the action space refer to? It'd be great if someone could provide an explanation with examples!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-01T13:59:36.037Z
Comments.Text: It means $a \in R^n$, where $a$ is action
Comments.UserDisplayName: Brale
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-01T14:00:37.003Z
Comments.Text: Ohh, okay! I don't see why researchers resort to complicated terms in papers.
Comments.UserDisplayName: stoic-santiago
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-01T14:33:25.217Z
Comments.Text: I wouldn't know how to write it simpler. It is just the translation of $a \in \mathbb{R}^n$ into text. Continuous values means (most likely) $\mathbb{R}$ and multidimensional $^n$. If you understand the mathematical expression you are most likely familiar with math and if that is the case then you most likely also know about these two terms. Maybe action space is a term that is quite unfamiliar for people who are learning about reinforcement learning? That just means the set of all possible actions.
Comments.UserDisplayName: alfa
Tags: <reinforcement-learning><terminology><papers><action-spaces><continuous-action-spaces>
Title: What is meant by a multi-dimensional continuous action space?
UserDisplayName: stoic-santiago
ThreadText: # What is meant by a multi-dimensional continuous action space?
  
  (Asked by: username_0 on 2020-05-01)
  
  In the context of Reinforcement Learning, **what does it mean to have a multi-dimensional continuous action space?**
  
  I came across the following in the [COBRA Paper](https://arxiv.org/abs/1905.09275)
  
  > A method for learning a distribution over a **multi-dimensional continuous action space.** This learned distribution can be sampled efficiently.
  
  and
  
  > During the initial exploration phase it explores its environment, in which it can move objects freely with a **continuous action space** but is not rewarded for its actions.
  
  So, what do the multi-dimensionality and the continuity of the action space refer to? It'd be great if someone could provide an explanation with examples!
  
  **Tags:** reinforcement-learning, terminology, papers, action-spaces, continuous-action-spaces
  > **Comments:**
  > **username_1**: It means $a \in R^n$, where $a$ is action  
  > **username_0**: Ohh, okay! I don't see why researchers resort to complicated terms in papers.  
  > **username_2**: I wouldn't know how to write it simpler. It is just the translation of $a \in \mathbb{R}^n$ into text. Continuous values means (most likely) $\mathbb{R}$ and multidimensional $^n$. If you understand the mathematical expression you are most likely familiar with math and if that is the case then you most likely also know about these two terms. Maybe action space is a term that is quite unfamiliar for people who are learning about reinforcement learning? That just means the set of all possible actions.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2020-05-01. Score: 2 | Accepted answer
  
  Let me rephrase it a little - it's a multidimensional continuous space of actions. So, you assign each action some vector from $R^{n}$. For intuition -- imagine you have a robot arm with four joints. For every joint you could applied a rotation force from [-1, 1] and thus you get a 4-D vector with float numbers for each possible action.
  
  
  ---
  
  ### Answer by username_0 on 2020-05-01. Score: 1
  
  The question has already been answered by Kirill, but I thought I'll add a good example of a multi-dimensional continuous action space too, namely the one I just encountered in the COBRA paper itself.
  
  > In all of our experiments we use a 2-dimensional virtual "touch-screen" environment that contains objects with configurable shape, position, and color. The agent can move any visible object by clicking on the object and clicking in a direction for the object to move. Hence the action space is **continuous and 4-dimensional, namely a pair of clicks**.
  
  
  ---
  
  