Answers.Body: <p>The main reason of using the term <em>data distribution</em> over the <em>random variable</em> is to note for the intrinsic relationship the different data samples have with each other.</p>
  <blockquote>
  <p>It is a mathematical way of saying this data is almost random
  (consider the average pixel on a <em>human face dataset</em>) but it has some
  underlaying relationship (the concept of <em>human faces</em>).</p>
  </blockquote>
  <p>This is also helpful to describe that different images can be similar looking but represent different underlaying concepts. For example: consider 2 dataset: <em>human faces</em>, <span class="math-container">$x_{human}$</span>, and animal faces, <span class="math-container">$x_{animal}$</span>. Although most of the images might look similar at a pixel level: similar colors, similar textures, similar backgrounds... The underlaying concept depicted in them differs: <em>human faces</em> vs <em>animal faces</em>.</p>
  <p>So, how do you represent that mathematically?: <strong>with two overlapping distributions of different shape</strong>.</p>
  <p><a href="https://i.sstatic.net/ub7nD.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ub7nD.png" alt="enter image description here" /></a></p>
  <p>This basically mean: <em>&quot;even though the images might seem random, might overlap in some domains (color, textures, histogram...) there is an intrinsic difference between the two&quot;</em> and this difference (the shape of the distribution) is the target intrinsic concept that the neural network will have to learn.</p>
  <p>In the specific case of GANs the left distribution would be the &quot;Generated Faces&quot; and this distribution will try to fit the right distribution that would be &quot;Real Faces&quot;.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-11T11:08:24.743Z
Answers.Text: So, can I safely infer that there is no need to imagine any random vectors?
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-11T12:22:20.160Z
Answers.Text: For the data distribution you are right. There is no need. Another thing that may be confusing is that GANs model the data distribution starting from a random vector (random noise vector) but that is just a trick because in the end that random vector can be seen as the Z vector of an auto-encoder (the vector that tightly packs all the info)
Answers.UserDisplayName: JVGD
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-11T09:02:42.213Z
Answers.LastActivityDate: 2022-07-11T09:11:59.573Z
Answers.LastEditDate: 2022-07-11T09:11:59.573Z
Answers.UserDisplayName: JVGD
Body: <p>Consider the following two paragraphs taken from the paper titles <strong><a href="https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf" rel="nofollow noreferrer">Generative Adversarial Nets</a></strong> by <em>Ian J. Goodfellow et.al</em></p>
  <p>#1: <strong>Abstract</strong></p>
  <blockquote>
  <p>We propose a new framework for estimating generative models via an
  adversarial process, in which we simultaneously train two models: a
  generative model G that captures the <strong>data distribution</strong>, and a
  discriminative model D that estimates the probability that a sample
  came from the training data rather than G. The training procedure for
  G is to maximize the probability of D making a mistake. This framework
  corresponds to a minimax two-player game. In the space of arbitrary
  functions G and D, a unique solution exists, with G recovering the
  training <strong>data distribution</strong> and D equal to 1.......</p>
  </blockquote>
  <p>#2: <strong>Excerpt from Introduction</strong></p>
  <blockquote>
  <p>The promise of deep learning is to discover rich, hierarchical models
  that represent <strong>probability distributions</strong> over the kinds of data
  encountered in artificial intelligence applications, such as natural
  images, audio waveforms containing speech, and symbols in natural
  language corpora.</p>
  </blockquote>
  <p>here we saw the word <strong>distribution</strong> thrice. It is very common to encounter the phrase <em>data distribution</em> in machine learning papers. For the types of data we use in artificial intelligence, there can be infinite random samples. We collect some instances and form a dataset, based on which we try to get the data distribution.</p>
  <p>Even most of the literature uses the phrases <em>data distribution</em> or <em>probability distribution</em>. I personally never came across papers that explicitly talk about the random vector for the probability distribution under consideration. What can be the reason for it? Why won't research papers give the list of random variables for the distribution they are discussing? Is it immaterial or is it obvious from the context?</p>
  <hr />
  <p>Note: For example, if our discussion is about a 'human face images' dataset and if we use the phrase 'data distribution' then should I assume random variables to be 'pixels' of the image or high-level features like 'eyes, ears, etc.,' or some other? Or is it immaterial? If immaterial, then what should I need to perceive about probability distribution? Should I imagine it as an arbitrary random vector?</p>
  
Tags: <terminology><datasets><probability-distribution><random-variable>
Title: What should be taken as random variables in the distributions of datasets?
UserDisplayName: hanugm
ThreadText: # What should be taken as random variables in the distributions of datasets?
  
  (Asked by: username_0 on 2022-07-11)
  
  Consider the following two paragraphs taken from the paper titles **[Generative Adversarial Nets](https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf)** by *Ian J. Goodfellow et.al*
  
  #1: **Abstract**
  
  > We propose a new framework for estimating generative models via an
  > adversarial process, in which we simultaneously train two models: a
  > generative model G that captures the **data distribution**, and a
  > discriminative model D that estimates the probability that a sample
  > came from the training data rather than G. The training procedure for
  > G is to maximize the probability of D making a mistake. This framework
  > corresponds to a minimax two-player game. In the space of arbitrary
  > functions G and D, a unique solution exists, with G recovering the
  > training **data distribution** and D equal to 1.......
  
  #2: **Excerpt from Introduction**
  
  > The promise of deep learning is to discover rich, hierarchical models
  > that represent **probability distributions** over the kinds of data
  > encountered in artificial intelligence applications, such as natural
  > images, audio waveforms containing speech, and symbols in natural
  > language corpora.
  
  here we saw the word **distribution** thrice. It is very common to encounter the phrase *data distribution* in machine learning papers. For the types of data we use in artificial intelligence, there can be infinite random samples. We collect some instances and form a dataset, based on which we try to get the data distribution.
  
  Even most of the literature uses the phrases *data distribution* or *probability distribution*. I personally never came across papers that explicitly talk about the random vector for the probability distribution under consideration. What can be the reason for it? Why won't research papers give the list of random variables for the distribution they are discussing? Is it immaterial or is it obvious from the context?
  
  ---
  
  Note: For example, if our discussion is about a 'human face images' dataset and if we use the phrase 'data distribution' then should I assume random variables to be 'pixels' of the image or high-level features like 'eyes, ears, etc.,' or some other? Or is it immaterial? If immaterial, then what should I need to perceive about probability distribution? Should I imagine it as an arbitrary random vector?
  
  **Tags:** terminology, datasets, probability-distribution, random-variable
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-11. Score: 0
  
  The main reason of using the term *data distribution* over the *random variable* is to note for the intrinsic relationship the different data samples have with each other.
  
  > It is a mathematical way of saying this data is almost random
  > (consider the average pixel on a *human face dataset*) but it has some
  > underlaying relationship (the concept of *human faces*).
  
  This is also helpful to describe that different images can be similar looking but represent different underlaying concepts. For example: consider 2 dataset: *human faces*, $x\_{human}$, and animal faces, $x\_{animal}$. Although most of the images might look similar at a pixel level: similar colors, similar textures, similar backgrounds... The underlaying concept depicted in them differs: *human faces* vs *animal faces*.
  
  So, how do you represent that mathematically?: **with two overlapping distributions of different shape**.
  
  [![enter image description here](https://i.sstatic.net/ub7nD.png)](https://i.sstatic.net/ub7nD.png)
  
  This basically mean: *"even though the images might seem random, might overlap in some domains (color, textures, histogram...) there is an intrinsic difference between the two"* and this difference (the shape of the distribution) is the target intrinsic concept that the neural network will have to learn.
  
  In the specific case of GANs the left distribution would be the "Generated Faces" and this distribution will try to fit the right distribution that would be "Real Faces".
  
  > **Comments:**
  > **username_0**: So, can I safely infer that there is no need to imagine any random vectors?  
  > **username_1**: For the data distribution you are right. There is no need. Another thing that may be confusing is that GANs model the data distribution starting from a random vector (random noise vector) but that is just a trick because in the end that random vector can be seen as the Z vector of an auto-encoder (the vector that tightly packs all the info)  
  
  ---
  
  