Answers.Body: <blockquote>
  <p>Does it make a difference how I structure that data?</p>
  </blockquote>
  <p>Since your gym environment creates two values, <code>t</code> and <code>h</code> at one single step, I believe the way you structure your input matters.</p>
  <p>Let's start with this one:</p>
  <pre><code>[
   [t,h,...],
   [t,h,...],
  ]
  </code></pre>
  <p>If I understand correctly, the index of the list is the input at the <code>i</code>-th step, so your input at step 0 is <code>inputs = X[0] = [t,h,...]</code>. <strong>This is the correct way</strong>. Your model sees <code>t</code> and <code>h</code> at the same time, which is what your environment produces.</p>
  <p>If you structure your input in the second way:</p>
  <pre><code>[
   [t,t,...],
   [h,h,...],
  ]
  </code></pre>
  <p>This way, it means your model sees <code>t</code> first, then sees <code>h</code> in the next step. This is technically incorrect from what we assume, since each input should be independent of each other. What's worse, I see that your <code>t</code> and <code>h</code> are on different scales, so it is highly possible that the training will diverge.</p>
  <p>However, depending on the difficulty of the task (which we don't know), you might or might not see a big difference between two structures. However, I recommend sticking with the first one.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-06T08:01:35.697Z
Answers.Text: Thanks for your answer @minh-long-luu. I think my description is a bit ambiguous: The list I am describing is actually for one time-step, meaning that I have readings from multiple sensors per step. I will edit my question to make that more explicit.
Answers.UserDisplayName: 94621
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-03T14:53:30.910Z
Answers.LastActivityDate: 2023-03-03T14:53:30.910Z
Answers.UserDisplayName: Minh-Long Luu
Body: <p>I am currently designing a custom gym environment that is based on sensor data and I struggle a bit with structuring the data and designing the model. Virtually every resource I find online is kind of vision based, in the sense that it takes images as input arrays.</p>
  <h2>Data structure</h2>
  <p>For each time step, I get a couple of readings from each sensor. This would be an example reading from one sensor, with <code>...</code> representing other variables.</p>
  <pre><code>{
    &quot;temperature&quot;:&quot;20.3&quot;,
    &quot;humidity&quot;:&quot;63&quot;,
    ...
  }
  </code></pre>
  <p>Now I see two approaches how to structure the input array for a model (<code>t</code> being temperature, <code>h</code> being humidity):</p>
  <pre><code>[
   [t,h,...],
   [t,h,...],
  ]
  </code></pre>
  <pre><code>[
   [t,t,...],
   [h,h,...],
  ]
  </code></pre>
  <p>Note that this represents the data for one time-step and is not representing a series of time-steps.</p>
  <h2>Question</h2>
  <p>Does it make a difference how I structure that data?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-26T17:37:42.333Z
Comments.Text: I'm gonna mark this for later because I think I can help, but it's late here
Comments.UserDisplayName: Minh-Long Luu
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-02T18:11:19.387Z
Comments.Text: Hi @94621 and welcome to AI Stack Exchange! This stack exchange website seems to be very particular about asking a single question per post. If possible, please condense your questions into a single question. I know that it might be difficult, but this post will most likely be closed if action isn't taken. Thank you for posting, and we hope to see more of your questions on this site!
Comments.UserDisplayName: DeepQZero
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-03T13:28:10.067Z
Comments.Text: Thanks for the hint @DeepQZero. I will edit the question accordingly
Comments.UserDisplayName: 94621
Tags: <reinforcement-learning><deep-rl>
Title: How to structure the input data for non-vision deep reinforcement learning?
UserDisplayName: 94621
ThreadText: # How to structure the input data for non-vision deep reinforcement learning?
  
  (Asked by: username_0 on 2023-02-24)
  
  I am currently designing a custom gym environment that is based on sensor data and I struggle a bit with structuring the data and designing the model. Virtually every resource I find online is kind of vision based, in the sense that it takes images as input arrays.
  Data structure
  For each time step, I get a couple of readings from each sensor. This would be an example reading from one sensor, with ... representing other variables.
  {
    "temperature":"20.3",
    "humidity":"63",
    ...
  }
  
  Now I see two approaches how to structure the input array for a model (t being temperature, h being humidity):
  [
   [t,h,...],
   [t,h,...],
  ]
  
  [
   [t,t,...],
   [h,h,...],
  ]
  
  Note that this represents the data for one time-step and is not representing a series of time-steps.
  Question
  Does it make a difference how I structure that data?
  
  **Tags:** reinforcement-learning, deep-rl
  > **Comments:**
  > **username_1**: I'm gonna mark this for later because I think I can help, but it's late here  
  > **username_2**: Hi @94621 and welcome to AI Stack Exchange! This stack exchange website seems to be very particular about asking a single question per post. If possible, please condense your questions into a single question. I know that it might be difficult, but this post will most likely be closed if action isn't taken. Thank you for posting, and we hope to see more of your questions on this site!  
  > **username_0**: Thanks for the hint @DeepQZero. I will edit the question accordingly  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-03. Score: 0 | Accepted answer
  
  Does it make a difference how I structure that data?
  
  Since your gym environment creates two values, t and h at one single step, I believe the way you structure your input matters.
  Let's start with this one:
  [
   [t,h,...],
   [t,h,...],
  ]
  
  If I understand correctly, the index of the list is the input at the i-th step, so your input at step 0 is inputs = X[0] = [t,h,...]. This is the correct way. Your model sees t and h at the same time, which is what your environment produces.
  If you structure your input in the second way:
  [
   [t,t,...],
   [h,h,...],
  ]
  
  This way, it means your model sees t first, then sees h in the next step. This is technically incorrect from what we assume, since each input should be independent of each other. What's worse, I see that your t and h are on different scales, so it is highly possible that the training will diverge.
  However, depending on the difficulty of the task (which we don't know), you might or might not see a big difference between two structures. However, I recommend sticking with the first one.
  
  > **Comments:**
  > **username_0**: Thanks for your answer @minh-long-luu. I think my description is a bit ambiguous: The list I am describing is actually for one time-step, meaning that I have readings from multiple sensors per step. I will edit my question to make that more explicit.  
  
  ---
  
  