Answers.Body: <p>This is not advisable. If you train your model with random data your model is not learning anything useful, because there is no information to gain from those examples. Even worse it may (and likely is) trying to generalize off of your incorrect examples, which will lessen the effect your real examples have. Essentially, you are just dampening your training set with noise.</p>
  <p>You are moving in the right direction though. 75 examples will not be enough if your problem has any complexity at all. And unless you know some correlation between the inputs <code>a</code>, <code>b</code> and the output <code>c</code>, you don't want to generate data (and even if you did know some correlation, it is not always suggested to generate data). If it is impossible to get any more data, you might want to consider a statistical model, rather than a neural network.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-01T17:28:23.297Z
Answers.Text: Thank you. Is there any way I can know what a minimum number of samples should be? I used to use a rule of thumb that the examples should be at least 10 times the degree of freedom.
Answers.UserDisplayName: Mohammad
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-01T18:55:08.220Z
Answers.Text: (@Mhmd) I use a slightly different method that I was taught in school. Train your model on 80% of your training set and see how much this hurts your performance. If you have enough data you won't see much of a difference between 80% and 100% of your training set. If you don't have enough data you performance will suffer.
Answers.UserDisplayName: Andrew Butler
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-01T20:17:50.343Z
Answers.Text: Yes, we know it is bad, but we don't know if it will have a negative effect on the resulting networks. It could happen by chance that we create a set of perfectly correct examples.
Answers.UserDisplayName: Andrew Butler
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-01T20:17:54.200Z
Answers.Text: OP has a correlation that says the points are increasing as a and b increase. Sure we could generate data that fit the rules provided and this would bring the error rate down, but error rate isn't the end all be all. Say we want to detect fraudulent transactions and assume 99% of transactions are non-fraudulent. A model could get 99% precision on predicting non-fraudulent transactions by always predicting transactions to be non-fraudulent. This is essentially what we would be doing by generating values. We'd be lowering the error rate by bounding our output, but lose a lot of predictive power.
Answers.UserDisplayName: Andrew Butler
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-04-01T17:02:18.743Z
Answers.LastActivityDate: 2021-12-25T17:43:44.117Z
Answers.LastEditDate: 2021-12-25T17:43:44.117Z
Answers.UserDisplayName: Andrew Butler
Answers.Body: <p>If you add fake samples to the training set, your neural network learns the new dataset that you just made. Your fake samples are estimations, so you add noise to your training set.</p>
  <p>You can use leave one out cross validation for evaluating your model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-04-01T18:42:25.277Z
Answers.LastActivityDate: 2024-05-21T17:33:18.070Z
Answers.LastEditDate: 2024-05-21T17:33:18.070Z
Answers.UserDisplayName: CVDE
Body: <p>I have a neural network with 2 inputs and one output, like so:</p>
  <pre><code>input    | output
  ____________________
   a    | b   |  c       
   5.15 |3.17 | 0.0607
   4.61 |2.91 | 0.1551
  </code></pre>
  <p>etc.</p>
  <p>I have 75 samples and I am using 50 for training and 25 for testing.</p>
  <p>However, I feel that the training samples are not enough. Because I can't provide more real samples (due to time limitation), I would like to train the network using fake data:</p>
  <p>For example, I know that the range for the <code>a</code> parameter is from 3 to 14, and that the <code>b</code> parameter is ~65% of the <code>a</code> parameter. I also know that <code>c</code> is a number between 0 and 1 and that it increases when a &amp; b increase.</p>
  <p>So, what I would like to do is to generate some data using the above restrictions (about 20 samples). For example, assume <code>a = 13</code> , <code>b = 8</code> and <code>c= 0.95</code>, and train the network with these samples before training it with the real samples.</p>
  <p>Has anybody studied the effect of doing this on the neural network? Is it possible to know if the effect will be better or worse on the networks? Are there any recommendations/guidelines if I want to do this?</p>
  
Tags: <neural-networks><reference-request><data-augmentation><training-datasets>
Title: What is the effect of training a neural network with randomly generated fake data that satisfies certain constraints?
UserDisplayName: Mohammad
ThreadText: # What is the effect of training a neural network with randomly generated fake data that satisfies certain constraints?
  
  (Asked by: username_0 on 2018-04-01)
  
  I have a neural network with 2 inputs and one output, like so:
  input    | output
  ____________________
   a    | b   |  c       
   5.15 |3.17 | 0.0607
   4.61 |2.91 | 0.1551
  
  etc.
  I have 75 samples and I am using 50 for training and 25 for testing.
  However, I feel that the training samples are not enough. Because I can't provide more real samples (due to time limitation), I would like to train the network using fake data:
  For example, I know that the range for the a parameter is from 3 to 14, and that the b parameter is ~65% of the a parameter. I also know that c is a number between 0 and 1 and that it increases when a & b increase.
  So, what I would like to do is to generate some data using the above restrictions (about 20 samples). For example, assume a = 13 , b = 8 and c= 0.95, and train the network with these samples before training it with the real samples.
  Has anybody studied the effect of doing this on the neural network? Is it possible to know if the effect will be better or worse on the networks? Are there any recommendations/guidelines if I want to do this?
  
  **Tags:** neural-networks, reference-request, data-augmentation, training-datasets
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-04-01. Score: 2 | Accepted answer
  
  This is not advisable. If you train your model with random data your model is not learning anything useful, because there is no information to gain from those examples. Even worse it may (and likely is) trying to generalize off of your incorrect examples, which will lessen the effect your real examples have. Essentially, you are just dampening your training set with noise.
  
  You are moving in the right direction though. 75 examples will not be enough if your problem has any complexity at all. And unless you know some correlation between the inputs `a`, `b` and the output `c`, you don't want to generate data (and even if you did know some correlation, it is not always suggested to generate data). If it is impossible to get any more data, you might want to consider a statistical model, rather than a neural network.
  
  > **Comments:**
  > **username_0**: Thank you. Is there any way I can know what a minimum number of samples should be? I used to use a rule of thumb that the examples should be at least 10 times the degree of freedom.  
  > **username_1**: (@Mhmd) I use a slightly different method that I was taught in school. Train your model on 80% of your training set and see how much this hurts your performance. If you have enough data you won't see much of a difference between 80% and 100% of your training set. If you don't have enough data you performance will suffer.  
  > **username_1**: Yes, we know it is bad, but we don't know if it will have a negative effect on the resulting networks. It could happen by chance that we create a set of perfectly correct examples.  
  > **username_1**: OP has a correlation that says the points are increasing as a and b increase. Sure we could generate data that fit the rules provided and this would bring the error rate down, but error rate isn't the end all be all. Say we want to detect fraudulent transactions and assume 99% of transactions are non-fraudulent. A model could get 99% precision on predicting non-fraudulent transactions by always predicting transactions to be non-fraudulent. This is essentially what we would be doing by generating values. We'd be lowering the error rate by bounding our output, but lose a lot of predictive power.  
  
  ---
  
  ### Answer by username_2 on 2018-04-01. Score: 1
  
  If you add fake samples to the training set, your neural network learns the new dataset that you just made. Your fake samples are estimations, so you add noise to your training set.
  
  You can use leave one out cross validation for evaluating your model.
  
  
  ---
  
  