Answers.Body: <p>A good example is the degree of freedom in <a href="https://en.wikipedia.org/wiki/Student%27s_t-distribution" rel="nofollow noreferrer">Student's distribution</a>:</p>
  <blockquote>
  <p>‌ The degrees of freedom refers to the number of independent observations in a set of data.</p>
  </blockquote>
  <p>For <a href="https://stattrek.com/probability-distributions/t-distribution.aspx#:%7E:text=The%20degrees%20of%20freedom%20refers,in%20a%20set%20of%20data.&amp;text=Hence%2C%20the%20distribution%20of%20the,or%207%20degrees%20of%20freedom." rel="nofollow noreferrer">example</a>:</p>
  <blockquote>
  <p>When estimating a mean score or a proportion from a single sample, the number of independent observations is equal to the sample size minus one.</p>
  </blockquote>
  <p>e.g, if we have 100 observation <span class="math-container">$X_1, \ldots, X_{100}$</span> and we want to estimate their mean <span class="math-container">$\bar{X}$</span>, as <span class="math-container">$\bar{X} = \frac{X_1 + \cdots + X_{100}}{100}$</span>, if we know the mean, just we need to find the value of 99 variables from <span class="math-container">$X_1, \ldots, X_{100}$</span>. Hence, here the degree of freedom is 99.</p>
  <p>Your referenced paragraph is a general explanation in the paper as well. However, base on the above example, the degree of freedom in the paragraph depends on the likelihood function and number of observations that we have from spectrograms.</p>
  <p>Now, as the DoF of latent variables is not high, using Gibbs sampling we will approximate some observations, and then using them we will compute the value of the latent variables.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-14T12:10:44.720Z
Answers.Text: This seems to be correct, but maybe you should explain what would be the degrees of freedom of a latent variable in the specific context of that paper.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-15T14:10:31.697Z
Answers.Text: How does Gibbs sampling help in this context?
Answers.UserDisplayName: stoic-santiago
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-15T14:14:54.700Z
Answers.Text: @cogito_ai Gibbs sampling helps us to approximate some observations from the distribution and compute the value of latent variables.
Answers.UserDisplayName: OmG
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-14T10:12:46.807Z
Answers.LastActivityDate: 2020-07-15T14:14:05.560Z
Answers.LastEditDate: 2020-07-15T14:14:05.560Z
Answers.UserDisplayName: OmG
Body: <blockquote>
  <p>...Designing such a likelihood function is typically challenging; however, we observe that features like spectrogram are effective when <strong>latent variables have limited degrees of freedom.</strong> This motivates us to infer latent variables via methods like Gibbs sampling, where we focus on approximating the conditional probability of a single variable given the others.</p>
  </blockquote>
  <p>Above is an excerpt from <a href="https://papers.nips.cc/paper/6727-shape-and-material-from-sound.pdf" rel="nofollow noreferrer">a paper I've been reading</a>, and I don't understand what the author means by degrees of freedom of latent variables. Could someone please explain with an example, or add more details?</p>
  <hr />
  <h3>References</h3>
  <p><a href="https://papers.nips.cc/paper/6727-shape-and-material-from-sound.pdf" rel="nofollow noreferrer">Shape and Material from Sound</a> <em>(31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA)</em></p>
  
Tags: <terminology><papers><generative-model><latent-variable>
Title: What is meant by degrees of freedom of latent variables?
UserDisplayName: stoic-santiago
ThreadText: # What is meant by degrees of freedom of latent variables?
  
  (Asked by: username_0 on 2020-07-14)
  
  > ...Designing such a likelihood function is typically challenging; however, we observe that features like spectrogram are effective when **latent variables have limited degrees of freedom.** This motivates us to infer latent variables via methods like Gibbs sampling, where we focus on approximating the conditional probability of a single variable given the others.
  
  Above is an excerpt from [a paper I've been reading](https://papers.nips.cc/paper/6727-shape-and-material-from-sound.pdf), and I don't understand what the author means by degrees of freedom of latent variables. Could someone please explain with an example, or add more details?
  
  ---
  
  ### References
  
  [Shape and Material from Sound](https://papers.nips.cc/paper/6727-shape-and-material-from-sound.pdf) *(31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA)*
  
  **Tags:** terminology, papers, generative-model, latent-variable
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-14. Score: 1 | Accepted answer
  
  A good example is the degree of freedom in [Student's distribution](https://en.wikipedia.org/wiki/Student%27s_t-distribution):
  
  > ‌ The degrees of freedom refers to the number of independent observations in a set of data.
  
  For [example](https://stattrek.com/probability-distributions/t-distribution.aspx#:%7E:text=The%20degrees%20of%20freedom%20refers,in%20a%20set%20of%20data.&text=Hence%2C%20the%20distribution%20of%20the,or%207%20degrees%20of%20freedom.):
  
  > When estimating a mean score or a proportion from a single sample, the number of independent observations is equal to the sample size minus one.
  
  e.g, if we have 100 observation $X\_1, \ldots, X\_{100}$ and we want to estimate their mean $\bar{X}$, as $\bar{X} = \frac{X\_1 + \cdots + X\_{100}}{100}$, if we know the mean, just we need to find the value of 99 variables from $X\_1, \ldots, X\_{100}$. Hence, here the degree of freedom is 99.
  
  Your referenced paragraph is a general explanation in the paper as well. However, base on the above example, the degree of freedom in the paragraph depends on the likelihood function and number of observations that we have from spectrograms.
  
  Now, as the DoF of latent variables is not high, using Gibbs sampling we will approximate some observations, and then using them we will compute the value of the latent variables.
  
  > **Comments:**
  > **username_2**: This seems to be correct, but maybe you should explain what would be the degrees of freedom of a latent variable in the specific context of that paper.  
  > **username_0**: How does Gibbs sampling help in this context?  
  > **username_1**: @cogito_ai Gibbs sampling helps us to approximate some observations from the distribution and compute the value of latent variables.  
  
  ---
  
  