Answers.Body: <p>The reasoning behind synthetic data is the same behind classic data augmentation,so the goal is to increase the amount of training instances to improve generalization.</p>
  <p>The difference with classic data augmentation techniques is that classic data augmentation rely on linear transformations like flipping or rotation that do not provide as much variation as we would like. Synthetic data instead are usually generated with generative models, which sample <em>new data</em> from a posterior distribution learned from real data. This difference is huge. Imagine augmenting data with a flipped pictures of the same dog flipped around vs augmenting the data with a pictures of different dogs not present in your current real data.</p>
  <p>Now the problem of course is that training generative model to produce good samples is as hard. Training a GAN on MNIST or other toy datasets is rather easy, doing so on real datasets with much less instances and all sort of imperfections (picture taken with different cameras to give an example) is many times a rabbit hole in itself. So you always want to do a good list of pros and cons before deciding to invest time in training a model just for synthetic data generation.</p>
  <p>On the other hand, it's not impossible to find synthetic datasets ready to be used out of the shelf. In that case exploring the data to check they fit your use case is usually a no brainer and time worth spending for sure. An example is <a href="https://www.synthcity.xyz/" rel="nofollow noreferrer">SynthCity</a>, a dataset for point cloud segmentation, I found it very handy for a project I was working on where we didn't have lot of point clouds to start with.</p>
  <h2>Update</h2>
  <p><em>Why not feeding the real data used to train the generative model directly?</em></p>
  <p>First answer is that you always feed the real data as well, i.e. you never train only on synthetic data, the point again is to augment you initial dataset, not replacing it.
  So you move from a situation like this:</p>
  <p>real data + naive augmentation -&gt; model</p>
  <p>to this:</p>
  <p>real data + synthetic data + naive augmentation -&gt; model</p>
  <p>Second answer is that not always synthetic data are generated trough machine learning, there are situations in which you can use analytic methods to produce them (for example fluid simulations trough differential equations). So in this case you're not even starting from real data at all.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-17T15:11:54.303Z
Answers.Text: Thanks. Yes, I can see how giving a model new data that does exist in the original sample can improve performance, but digging deeper, my question is really why we need a separate model to do the generation of new data. Why can't we input the real data (from which the first model learns the distribution) directly into the second model? Can the second model not both learn the distribution and generalize? Or is it hard to architect such a model that does both?
Answers.UserDisplayName: Fijoy Vadakkumpadan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-17T15:30:35.287Z
Answers.Text: I've updated the answer, hope it's more clear!
Answers.UserDisplayName: Edoardo Guerriero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-17T10:19:36.737Z
Answers.LastActivityDate: 2022-11-17T15:30:20.190Z
Answers.LastEditDate: 2022-11-17T15:30:20.190Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>I understand from articles like <a href="https://towardsdatascience.com/implementing-generative-adversarial-networks-gans-for-increasing-a-convolutional-neural-networks-f871e17fe271" rel="nofollow noreferrer">this one</a> that synthetic data generated by one model based on real data can improve the performance of a second model. Can anyone help me understand the intuition behind why this works? Specifically, why not directly use the real data to train the second model? Since the data generated by the first model is at most as good as the real data used for that model, won't the performance be actually better if the real data are directly used to train the second model?</p>
  
Tags: <training><datasets><generative-adversarial-networks><data-augmentation>
Title: Why does data augmentation using synthetic data generated by one model improve the performance of another model?
UserDisplayName: Fijoy Vadakkumpadan
ThreadText: # Why does data augmentation using synthetic data generated by one model improve the performance of another model?
  
  (Asked by: username_0 on 2022-11-17)
  
  I understand from articles like [this one](https://towardsdatascience.com/implementing-generative-adversarial-networks-gans-for-increasing-a-convolutional-neural-networks-f871e17fe271) that synthetic data generated by one model based on real data can improve the performance of a second model. Can anyone help me understand the intuition behind why this works? Specifically, why not directly use the real data to train the second model? Since the data generated by the first model is at most as good as the real data used for that model, won't the performance be actually better if the real data are directly used to train the second model?
  
  **Tags:** training, datasets, generative-adversarial-networks, data-augmentation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-11-17. Score: 1
  
  The reasoning behind synthetic data is the same behind classic data augmentation,so the goal is to increase the amount of training instances to improve generalization.
  
  The difference with classic data augmentation techniques is that classic data augmentation rely on linear transformations like flipping or rotation that do not provide as much variation as we would like. Synthetic data instead are usually generated with generative models, which sample *new data* from a posterior distribution learned from real data. This difference is huge. Imagine augmenting data with a flipped pictures of the same dog flipped around vs augmenting the data with a pictures of different dogs not present in your current real data.
  
  Now the problem of course is that training generative model to produce good samples is as hard. Training a GAN on MNIST or other toy datasets is rather easy, doing so on real datasets with much less instances and all sort of imperfections (picture taken with different cameras to give an example) is many times a rabbit hole in itself. So you always want to do a good list of pros and cons before deciding to invest time in training a model just for synthetic data generation.
  
  On the other hand, it's not impossible to find synthetic datasets ready to be used out of the shelf. In that case exploring the data to check they fit your use case is usually a no brainer and time worth spending for sure. An example is [SynthCity](https://www.synthcity.xyz/), a dataset for point cloud segmentation, I found it very handy for a project I was working on where we didn't have lot of point clouds to start with.
  
  ## Update
  
  *Why not feeding the real data used to train the generative model directly?*
  
  First answer is that you always feed the real data as well, i.e. you never train only on synthetic data, the point again is to augment you initial dataset, not replacing it.
  So you move from a situation like this:
  
  real data + naive augmentation -> model
  
  to this:
  
  real data + synthetic data + naive augmentation -> model
  
  Second answer is that not always synthetic data are generated trough machine learning, there are situations in which you can use analytic methods to produce them (for example fluid simulations trough differential equations). So in this case you're not even starting from real data at all.
  
  > **Comments:**
  > **username_0**: Thanks. Yes, I can see how giving a model new data that does exist in the original sample can improve performance, but digging deeper, my question is really why we need a separate model to do the generation of new data. Why can't we input the real data (from which the first model learns the distribution) directly into the second model? Can the second model not both learn the distribution and generalize? Or is it hard to architect such a model that does both?  
  > **username_1**: I've updated the answer, hope it's more clear!  
  
  ---
  
  