Answers.Body: <p>The observation space and the state space are not the same in general. There exist problems where the state space cannot be fully observed, which goes by the name <code>Partially Observable Reinforcement Learning</code> (or some variation of it). This is relevant in environments with imperfect information. To me, it does not look like you are dealing with partial observation here, therefore state and observation space are identical.</p>
  <p>Let's say e.g. for a poker game the state and the observation space are different, because the state space is the space of all the cards and money of all players (+ the order of play) and the community cards, while the observation space from one agent is only his own cards + community cards and the money. The observation space does not include the cards of the opposite players.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-01T15:42:28.413Z
Answers.Text: Hey! Thanks for the clarification! It helped me understand better. I was mainly concerned with this as I'm trying to train a DDPG model for such a scenario. I was really confused if my input states to the Actor-network would be the state space or the observation space(as most of the papers and blogs use the observation space). Since they are the same, would you say that my input to the network would be my state space itself?
Answers.UserDisplayName: AlphaBit95
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-01T16:30:41.370Z
Answers.Text: The input to your network should be the states at a given time, not the whole state space. So in your case the inputs should be the concrete values of `l` and `b` as far as I can read from your question.
Answers.UserDisplayName: pythonic833
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-31T12:32:50.427Z
Answers.LastActivityDate: 2023-01-31T12:32:50.427Z
Answers.UserDisplayName: pythonic833
Body: <p>I'm fairly new to reinforcement learning concepts, and I'm trying to implement a simple custom environment. In my custom environment, I have a scenario where I have multiple continuous state spaces, for example, length(l), and breadth(b), from which I calculate say, area(a) = l*b. I calculate the reward based on the area. Here I check if the area lies between the range I'm expecting it to and reward it accordingly.</p>
  <p>Since my reward is based on the area which is f(l, b) = l*b, <strong>should I declare the observation space as the range of values my area can attain? Or should I declare my observation space as the values the length(l) and breadth(b) can attain?</strong> Or is my understanding wrong?</p>
  <pre><code>Area(a) = f(l, b) = l*b
  Reward = +1 (if 15 &lt; a &lt; 20)
           -.1 (otherwise)
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-01T09:46:24.427Z
Comments.Text: Don't put 2 questions in the title. One post should contain only one question. Decide what is your main question and put it in the title. The others can be follow-up questions, but ideally asked in a separate post.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-01T15:44:44.220Z
Comments.Text: Hey! This was the first time I posted on such a forum and I wasn't aware of this. Will keep this in mind. Thank you! :)
Comments.UserDisplayName: AlphaBit95
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-01T20:11:26.730Z
Comments.Text: Please, just do as I suggested for this post too, not just for the future.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><terminology><environment>
Title: When is it necessary to explicitly define both the state and observation space in a custom environment?
UserDisplayName: AlphaBit95
ThreadText: # When is it necessary to explicitly define both the state and observation space in a custom environment?
  
  (Asked by: username_0 on 2023-01-31)
  
  I'm fairly new to reinforcement learning concepts, and I'm trying to implement a simple custom environment. In my custom environment, I have a scenario where I have multiple continuous state spaces, for example, length(l), and breadth(b), from which I calculate say, area(a) = l\*b. I calculate the reward based on the area. Here I check if the area lies between the range I'm expecting it to and reward it accordingly.
  
  Since my reward is based on the area which is f(l, b) = l\*b, **should I declare the observation space as the range of values my area can attain? Or should I declare my observation space as the values the length(l) and breadth(b) can attain?** Or is my understanding wrong?
  
  ```
  Area(a) = f(l, b) = l*b
  Reward = +1 (if 15 < a < 20)
           -.1 (otherwise)
  
  ```
  
  **Tags:** reinforcement-learning, terminology, environment
  
  > **Comments:**
  > **username_1**: Don't put 2 questions in the title. One post should contain only one question. Decide what is your main question and put it in the title. The others can be follow-up questions, but ideally asked in a separate post.  
  > **username_0**: Hey! This was the first time I posted on such a forum and I wasn't aware of this. Will keep this in mind. Thank you! :)  
  > **username_1**: Please, just do as I suggested for this post too, not just for the future.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-01-31. Score: 2
  
  The observation space and the state space are not the same in general. There exist problems where the state space cannot be fully observed, which goes by the name `Partially Observable Reinforcement Learning` (or some variation of it). This is relevant in environments with imperfect information. To me, it does not look like you are dealing with partial observation here, therefore state and observation space are identical.
  
  Let's say e.g. for a poker game the state and the observation space are different, because the state space is the space of all the cards and money of all players (+ the order of play) and the community cards, while the observation space from one agent is only his own cards + community cards and the money. The observation space does not include the cards of the opposite players.
  
  > **Comments:**
  > **username_0**: Hey! Thanks for the clarification! It helped me understand better. I was mainly concerned with this as I'm trying to train a DDPG model for such a scenario. I was really confused if my input states to the Actor-network would be the state space or the observation space(as most of the papers and blogs use the observation space). Since they are the same, would you say that my input to the network would be my state space itself?  
  > **username_2**: The input to your network should be the states at a given time, not the whole state space. So in your case the inputs should be the concrete values of `l` and `b` as far as I can read from your question.  
  
  ---
  
  