Answers.Body: <p>Yes, it exists, and it's called the <a href="https://en.wikipedia.org/wiki/Likelihood_function" rel="nofollow noreferrer">likelihood function</a> (but if you want to actually calculate that &quot;scalar&quot;, you better use the <a href="https://en.wikipedia.org/wiki/Likelihood_function#Log-likelihood" rel="nofollow noreferrer">log likelihood</a>)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-02T17:10:33.823Z
Answers.LastActivityDate: 2024-04-02T17:10:33.823Z
Answers.UserDisplayName: Alberto
Answers.Body: <p>In general there're many ways to achieve your request in statistics such as goodness-of-fit chi-squared test, likelihood function as mentioned in another answer, KL-Divergence, etc. And perhaps in your high dimensional case, <a href="https://en.wikipedia.org/wiki/Mahalanobis_distance" rel="nofollow noreferrer">Mahalanobis distance</a> is a good fit.</p>
  <blockquote>
  <p>The Mahalanobis distance is a measure of the distance between a point <span class="math-container">${\displaystyle P}$</span> and a distribution <span class="math-container">${\displaystyle D}$</span>, introduced by P. C. Mahalanobis in 1936... If each of these axes is re-scaled to have unit variance, then the Mahalanobis distance corresponds to standard Euclidean distance in the transformed space. The Mahalanobis distance is thus unitless, scale-invariant, and takes into account the correlations of the data set.</p>
  </blockquote>
  <blockquote>
  <p>For a normal distribution in any number of dimensions, the probability density of an observation <span class="math-container">${\displaystyle {\vec {x}}}$</span> is uniquely determined by the Mahalanobis distance <span class="math-container">${\displaystyle d}$</span>... The Mahalanobis distance is proportional, for a normal distribution, to the square root of the negative log-likelihood (after adding a constant so the minimum is at zero).</p>
  </blockquote>
  <p>Therefore by calculating the Mahalanobis distance for each data point with respect to the mean of your standard multivariate Gaussian distribution, you can obtain a measure of how well your total sampled data points fit the multivariate Gaussian. A smaller summed value indicates a better fit to the distribution and is equivalent to negative log-likelihood essentially as explained in above reference.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-03T06:26:23.797Z
Answers.LastActivityDate: 2024-04-03T06:40:39.373Z
Answers.LastEditDate: 2024-04-03T06:40:39.373Z
Answers.UserDisplayName: cinch
Body: <p>I have a sampled data, say, many vectors <span class="math-container">$m_1, \cdots, m_i \in R^{d}$</span>.</p>
  <p>I want an objective function that rates the probability of this data being drawn from certain distribution (here, specifically standard Gaussian <span class="math-container">$N(0, \mathbf{I}_{d\times d})$</span>). That is to say, I want a function <span class="math-container">$f(m_1, \cdots, m_i)$</span> that takes the vectors <span class="math-container">$m_1, \cdots, m_i \in R^{d}$</span> and outputs a scalar that, <strong>the more likely the vectors are sampled from the distribution <span class="math-container">$N(0, \mathbf{I}_{d\times d})$</span>, the smaller(or bigger) the function's return value is.</strong></p>
  <p>Is there any predefined ones in statistics or math theory?</p>
  <p>I don't know much about math, just some simple ones from probabilities &amp; statistics undergrad course.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-02T12:23:20.113Z
Comments.Text: What does the normal distribution mean? See [probability density function](https://en.wikipedia.org/wiki/Multivariate_normal_distribution).
Comments.UserDisplayName: programjames
Tags: <probability-distribution><metric><statistics><normal-distribution>
Title: How do I determine how likely some data was drawn from a distribution?
UserDisplayName: Da1sy petals
ThreadText: # How do I determine how likely some data was drawn from a distribution?
  
  (Asked by: username_0 on 2024-04-02)
  
  I have a sampled data, say, many vectors $m\_1, \cdots, m\_i \in R^{d}$.
  
  I want an objective function that rates the probability of this data being drawn from certain distribution (here, specifically standard Gaussian $N(0, \mathbf{I}\_{d\times d})$). That is to say, I want a function $f(m\_1, \cdots, m\_i)$ that takes the vectors $m\_1, \cdots, m\_i \in R^{d}$ and outputs a scalar that, **the more likely the vectors are sampled from the distribution $N(0, \mathbf{I}\_{d\times d})$, the smaller(or bigger) the function's return value is.**
  
  Is there any predefined ones in statistics or math theory?
  
  I don't know much about math, just some simple ones from probabilities & statistics undergrad course.
  
  **Tags:** probability-distribution, metric, statistics, normal-distribution
  > **Comments:**
  > **username_1**: What does the normal distribution mean? See [probability density function](https://en.wikipedia.org/wiki/Multivariate_normal_distribution).  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2024-04-02. Score: 0
  
  Yes, it exists, and it's called the [likelihood function](https://en.wikipedia.org/wiki/Likelihood_function) (but if you want to actually calculate that "scalar", you better use the [log likelihood](https://en.wikipedia.org/wiki/Likelihood_function#Log-likelihood))
  
  
  ---
  
  ### Answer by username_3 on 2024-04-03. Score: -1
  
  In general there're many ways to achieve your request in statistics such as goodness-of-fit chi-squared test, likelihood function as mentioned in another answer, KL-Divergence, etc. And perhaps in your high dimensional case, [Mahalanobis distance](https://en.wikipedia.org/wiki/Mahalanobis_distance) is a good fit.
  
  > The Mahalanobis distance is a measure of the distance between a point ${\displaystyle P}$ and a distribution ${\displaystyle D}$, introduced by P. C. Mahalanobis in 1936... If each of these axes is re-scaled to have unit variance, then the Mahalanobis distance corresponds to standard Euclidean distance in the transformed space. The Mahalanobis distance is thus unitless, scale-invariant, and takes into account the correlations of the data set.
  
  > For a normal distribution in any number of dimensions, the probability density of an observation ${\displaystyle {\vec {x}}}$ is uniquely determined by the Mahalanobis distance ${\displaystyle d}$... The Mahalanobis distance is proportional, for a normal distribution, to the square root of the negative log-likelihood (after adding a constant so the minimum is at zero).
  
  Therefore by calculating the Mahalanobis distance for each data point with respect to the mean of your standard multivariate Gaussian distribution, you can obtain a measure of how well your total sampled data points fit the multivariate Gaussian. A smaller summed value indicates a better fit to the distribution and is equivalent to negative log-likelihood essentially as explained in above reference.
  
  
  ---
  
  