Answers.Body: <p>You're right! The generative model <span class="math-container">$f$</span> is not the same as the probability density  (p.d.f.) function <span class="math-container">$p_{data}$</span>. The kind of phrases you've referred to are to be interpreted informally. You learn <span class="math-container">$f$</span> with the hope that sampling a latent vector <span class="math-container">$z$</span>
  from some known distribution (from which it is easy to sample), results in <span class="math-container">$f(z)$</span> that has the probability density function <span class="math-container">$p_{data}$</span>. However, merely learning <span class="math-container">$f$</span> does not give you the power to estimate what <span class="math-container">$p_{data}(x)$</span> is for some image <span class="math-container">$x$</span>. Learning <span class="math-container">$f$</span> only gives you the power to sample according to <span class="math-container">$p_{data}(\cdot)$</span> (if you've learned an accurate such <span class="math-container">$f$</span>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-04T08:16:22.167Z
Answers.LastActivityDate: 2021-01-04T08:16:22.167Z
Answers.UserDisplayName: Enigman
Body: <p>Let us assume that I am working on a dataset of black and white dog images.</p>
  <p>Each image is of size <span class="math-container">$28 \times 28$</span>.</p>
  <p>Now, I can say that I have a sample space <span class="math-container">$S$</span> of all possible images. And <span class="math-container">$p_{data}$</span> is the probability distribution for dog images. It is easy to understand that all other images get a probability value of zero. And it is obvious that <span class="math-container">$n(S)= 2^{28 \times 28}$</span>.</p>
  <p>Now, I am going to design a generative model that sample from <span class="math-container">$S$</span> using <span class="math-container">$p_{data}$</span> rather than random sampling.</p>
  <p>My generative model is a neural network that takes random noise (say, of length 100) and generates an image of the size <span class="math-container">$28 \times 28$</span>. My function is learning a function <span class="math-container">$f$</span>, which is totally different from the function <span class="math-container">$p_{data}$</span>. It is because of the reason that  <span class="math-container">$f$</span> is from <span class="math-container">$R^{100}$</span> to <span class="math-container">$S$</span> and <span class="math-container">$p_{data}$</span> is from <span class="math-container">$S$</span> to <span class="math-container">$[0,1]$</span>.</p>
  <p>In the literature, I often read the phrases that <strong>our generative model learned <span class="math-container">$p_{data}$</span></strong> or <strong>our goal is to get <span class="math-container">$p_{data}$</span></strong>, etc.,  but in fact, they are trying to learn <span class="math-container">$f$</span>, which just obeys <span class="math-container">$p_{data}$</span> while giving its output.</p>
  <p>Am I going wrong anywhere or the usage in literature is somewhat random?</p>
  
Tags: <terminology>
Title: Confusion between function learned and the underlying distribution
UserDisplayName: hanugm
ThreadText: # Confusion between function learned and the underlying distribution
  
  (Asked by: username_0 on 2021-01-04)
  
  Let us assume that I am working on a dataset of black and white dog images.
  
  Each image is of size $28 \times 28$.
  
  Now, I can say that I have a sample space $S$ of all possible images. And $p\_{data}$ is the probability distribution for dog images. It is easy to understand that all other images get a probability value of zero. And it is obvious that $n(S)= 2^{28 \times 28}$.
  
  Now, I am going to design a generative model that sample from $S$ using $p\_{data}$ rather than random sampling.
  
  My generative model is a neural network that takes random noise (say, of length 100) and generates an image of the size $28 \times 28$. My function is learning a function $f$, which is totally different from the function $p\_{data}$. It is because of the reason that $f$ is from $R^{100}$ to $S$ and $p\_{data}$ is from $S$ to $[0,1]$.
  
  In the literature, I often read the phrases that **our generative model learned $p\_{data}$** or **our goal is to get $p\_{data}$**, etc., but in fact, they are trying to learn $f$, which just obeys $p\_{data}$ while giving its output.
  
  Am I going wrong anywhere or the usage in literature is somewhat random?
  
  **Tags:** terminology
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-01-04. Score: 3
  
  You're right! The generative model $f$ is not the same as the probability density (p.d.f.) function $p\_{data}$. The kind of phrases you've referred to are to be interpreted informally. You learn $f$ with the hope that sampling a latent vector $z$
  from some known distribution (from which it is easy to sample), results in $f(z)$ that has the probability density function $p\_{data}$. However, merely learning $f$ does not give you the power to estimate what $p\_{data}(x)$ is for some image $x$. Learning $f$ only gives you the power to sample according to $p\_{data}(\cdot)$ (if you've learned an accurate such $f$).
  
  
  ---
  
  