Answers.Body: <p>Yes. In Machine Learning we consider that the samples in your training set are sampled from an underlying distribution called the <strong>data generating distribution</strong>.</p>
  
  <p><a href="https://en.wikipedia.org/wiki/Generative_model" rel="nofollow noreferrer">Generative models</a> classify the samples by trying to <em>learn</em> the distribution of the data. In most cases, either the model is incapable of doing so, or the training samples aren't enough to properly describe the data-generating distribution, so the model learns an <em>approximation</em> of this. This is what you call the <strong>model's distribution</strong>.</p>
  
  <p>You can find more info about these concepts in a more <a href="https://datascience.stackexchange.com/a/54349/34269">detailed answer</a> I wrote. If you're familiar with GANs, you can also read <a href="https://stats.stackexchange.com/a/435159/119015">this post</a>, to see where these two concepts come into play when training the two networks.    </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-05T14:09:48.970Z
Answers.Text: This answer seems correct, but why are you talking about generative models? Are model distributions only associated with generative models? Is the terminology "data distribution" and "model distributions" only useful in the case of GANs or generative models? Is this terminology widely known in machine learning, or only Goodfellow's book uses it?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-05T15:27:57.843Z
Answers.Text: Generative models is a broad category of  ML models including the generator from GANs, naive bayes, etc. The other category are discriminative models, which don't try to explicitly model the data's distribution, i.e. there's no notion of a *model's distribution*.  The terms "generative" and "discriminative" models are widely used, as is the notion of the *data-generating* distribution. I'm unaware, though, if the latter was coined by Goodfellow or not (it might have also different names). Nowadays, though, it is the only widely-used terminology I'm aware of.
Answers.UserDisplayName: Djib2011
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-05T10:40:35.467Z
Answers.LastActivityDate: 2020-04-05T14:09:12.930Z
Answers.LastEditDate: 2020-04-05T14:09:12.930Z
Answers.UserDisplayName: Djib2011
Body: <p>Is there any difference between the model distribution and data distribution, or are they the same?</p>
  
Tags: <machine-learning><comparison><models><probability-distribution><statistics>
Title: What is the difference between model and data distributions?
UserDisplayName: Bhuwan Bhatt
ThreadText: # What is the difference between model and data distributions?
  
  (Asked by: username_0 on 2020-04-05)
  
  Is there any difference between the model distribution and data distribution, or are they the same?
  
  **Tags:** machine-learning, comparison, models, probability-distribution, statistics
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-05. Score: 4
  
  Yes. In Machine Learning we consider that the samples in your training set are sampled from an underlying distribution called the **data generating distribution**.
  
  [Generative models](https://en.wikipedia.org/wiki/Generative_model) classify the samples by trying to *learn* the distribution of the data. In most cases, either the model is incapable of doing so, or the training samples aren't enough to properly describe the data-generating distribution, so the model learns an *approximation* of this. This is what you call the **model's distribution**.
  
  You can find more info about these concepts in a more [detailed answer](https://datascience.stackexchange.com/a/54349/34269) I wrote. If you're familiar with GANs, you can also read [this post](https://stats.stackexchange.com/a/435159/119015), to see where these two concepts come into play when training the two networks.
  
  > **Comments:**
  > **username_2**: This answer seems correct, but why are you talking about generative models? Are model distributions only associated with generative models? Is the terminology "data distribution" and "model distributions" only useful in the case of GANs or generative models? Is this terminology widely known in machine learning, or only Goodfellow's book uses it?  
  > **username_1**: Generative models is a broad category of  ML models including the generator from GANs, naive bayes, etc. The other category are discriminative models, which don't try to explicitly model the data's distribution, i.e. there's no notion of a *model's distribution*.  The terms "generative" and "discriminative" models are widely used, as is the notion of the *data-generating* distribution. I'm unaware, though, if the latter was coined by Goodfellow or not (it might have also different names). Nowadays, though, it is the only widely-used terminology I'm aware of.  
  
  ---
  
  