Answers.Body: <p>In general, both methods are valid to train temporal models. The only thing you need to check is that validation and test-set don't overlap with any of your training samples.</p>
  <p>Using the overlapping variant can sometimes increase performance, because your model might benefit from the increased number of different starting points. This can be very relevant for certain datasets. For example, traffic data has periodic characteristics (little traffic at nighttimes / high traffic at daytimes). Let's say you have hourly traffic data and a sequence length of 24. Now your sequences will always start at the exact same day time. Therefore the model might not learn to properly forecast from other starting times.</p>
  <p>I cannot think of any drawback the overlapping variant might have, so I'd opt for the overlapping variant. To be sure which one works best for you, I think you have to find out empirically.</p>
  <blockquote>
  <p>Does it depend on the Model/Training algorithm.</p>
  </blockquote>
  <p>In general it shouldn't, after all, in both cases you are always feeding the model valid sequences - any model and any training algorithm should deal well with either one.</p>
  <blockquote>
  <p>Does it depend on the Data?</p>
  </blockquote>
  <p>Yes, if the sequences have distinct starting points the non-overlapping variant might be better suited. The traffic-data example from above applies here.</p>
  <blockquote>
  <p>Does it depend on the number of data points I have available? (I have hourly data for 6 years)</p>
  </blockquote>
  <p>I'd say yes, you effectively increase the number of data points by using overlapping sequences, this is good if your dataset is small. Your dataset is comparably large.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-11T19:17:05.530Z
Answers.LastActivityDate: 2022-07-11T19:17:05.530Z
Answers.UserDisplayName: Chillston
Body: <p>If I have time dependent data and want to predict the relative change for a future time. Should I separate the data so that the input times don't overlap?</p>
  <p>With an example:<br />
  I have hourly temperature readings and want to predict the temperature change 4 hours after the last input hour. Now I could split my data so that I have <br />
  (overlapping)<br />
  1st data point: 1h, 2h, 3h, ... 10h -&gt; predict for 14h<br />
  2nd data point: 2h, 3h, 4h ... 11h -&gt; predict for 15h<br />
  (not overlapping)<br />
  1st data point: 1h, 2h, 3h, ... 10h -&gt; predict for 14h<br />
  2nd data point: 15h, 16h, 17h ... 25h -&gt; predict for 29h\</p>
  <p>Further questions:<br />
  Does it depend on the Model/Training algorithm.<br />
  Does it depend on the Data? <br />
  Does it depend on the number of data points I have available? (I have hourly data for 6 years)</p>
  
Tags: <neural-networks><datasets><training-datasets>
Title: Should I include overlapping (input) Data in my training data
UserDisplayName: KarlTheGreat
ThreadText: # Should I include overlapping (input) Data in my training data
  
  (Asked by: username_0 on 2022-07-11)
  
  If I have time dependent data and want to predict the relative change for a future time. Should I separate the data so that the input times don't overlap?
  
  With an example:  
  I have hourly temperature readings and want to predict the temperature change 4 hours after the last input hour. Now I could split my data so that I have   
  (overlapping)  
  1st data point: 1h, 2h, 3h, ... 10h -> predict for 14h  
  2nd data point: 2h, 3h, 4h ... 11h -> predict for 15h  
  (not overlapping)  
  1st data point: 1h, 2h, 3h, ... 10h -> predict for 14h  
  2nd data point: 15h, 16h, 17h ... 25h -> predict for 29h\
  
  Further questions:  
  Does it depend on the Model/Training algorithm.  
  Does it depend on the Data?   
  Does it depend on the number of data points I have available? (I have hourly data for 6 years)
  
  **Tags:** neural-networks, datasets, training-datasets
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-11. Score: 1 | Accepted answer
  
  In general, both methods are valid to train temporal models. The only thing you need to check is that validation and test-set don't overlap with any of your training samples.
  
  Using the overlapping variant can sometimes increase performance, because your model might benefit from the increased number of different starting points. This can be very relevant for certain datasets. For example, traffic data has periodic characteristics (little traffic at nighttimes / high traffic at daytimes). Let's say you have hourly traffic data and a sequence length of 24. Now your sequences will always start at the exact same day time. Therefore the model might not learn to properly forecast from other starting times.
  
  I cannot think of any drawback the overlapping variant might have, so I'd opt for the overlapping variant. To be sure which one works best for you, I think you have to find out empirically.
  
  > Does it depend on the Model/Training algorithm.
  
  In general it shouldn't, after all, in both cases you are always feeding the model valid sequences - any model and any training algorithm should deal well with either one.
  
  > Does it depend on the Data?
  
  Yes, if the sequences have distinct starting points the non-overlapping variant might be better suited. The traffic-data example from above applies here.
  
  > Does it depend on the number of data points I have available? (I have hourly data for 6 years)
  
  I'd say yes, you effectively increase the number of data points by using overlapping sequences, this is good if your dataset is small. Your dataset is comparably large.
  
  
  ---
  
  