Body: <p>I have a dataset with financial stock data, some of the features are shared, for example daily gold prices, while the stock price for each individual stock is different, the gold price would be the same for everybody that day.</p>
  <p>When I split 80/10/10 randomly, it's &quot;cheating&quot; and while the result accuracy is great the actual real world live result is bad.</p>
  <p>When I split sequentially, ie first 8 years of data in training, next year in validation, last year in testing.  The result accuracy is bad, and live testing is also bad.</p>
  <p>What I want to ask is, should I do random split between just training and validation on first 9 years of data, then do testing on last year of data separately?</p>
  <p>OR is sequentially as good as it's gonna get and I simply can't predict the future?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-03T23:27:54.743Z
Comments.Text: Welcome to SE:AI!  I think this has gotten close voted per the solo "datasets" tag.  My sense is this is, in fact, a question about AI theory, and the better method of splitting such data?  Is this correct?  Are there other tag you could potentially apply?
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-04T16:04:08.383Z
Comments.Text: It is about theory and I am putting it in practice, at the end, the question boil down to: can training set and validation set share information?  OR is that consider cheating?  It is well known in our industry already that sharing information between training and testing is a big no no, but what about training and validation?  I am not sure what other tag to put this under.
Comments.UserDisplayName: Bill Software Engineer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-04T22:46:45.023Z
Comments.Text: My advice is to edit your question to reinforce the on-topic nature and add a couple more tags.  (We get a lot of pure data set questions, not related to theory, which are off topic.)
Comments.UserDisplayName: DukeZhou
Tags: <datasets>
Title: Split on dataset with some shared features?
UserDisplayName: Bill Software Engineer
ThreadText: # Split on dataset with some shared features?
  
  (Asked by: username_0 on 2021-06-01)
  
  I have a dataset with financial stock data, some of the features are shared, for example daily gold prices, while the stock price for each individual stock is different, the gold price would be the same for everybody that day.
  
  When I split 80/10/10 randomly, it's "cheating" and while the result accuracy is great the actual real world live result is bad.
  
  When I split sequentially, ie first 8 years of data in training, next year in validation, last year in testing. The result accuracy is bad, and live testing is also bad.
  
  What I want to ask is, should I do random split between just training and validation on first 9 years of data, then do testing on last year of data separately?
  
  OR is sequentially as good as it's gonna get and I simply can't predict the future?
  
  **Tags:** datasets
  > **Comments:**
  > **username_1**: Welcome to SE:AI!  I think this has gotten close voted per the solo "datasets" tag.  My sense is this is, in fact, a question about AI theory, and the better method of splitting such data?  Is this correct?  Are there other tag you could potentially apply?  
  > **username_0**: It is about theory and I am putting it in practice, at the end, the question boil down to: can training set and validation set share information?  OR is that consider cheating?  It is well known in our industry already that sharing information between training and testing is a big no no, but what about training and validation?  I am not sure what other tag to put this under.  
  > **username_1**: My advice is to edit your question to reinforce the on-topic nature and add a couple more tags.  (We get a lot of pure data set questions, not related to theory, which are off topic.)  
  
  ---
  
  ## 0 answers
  
  