What is the definition and upper bound on the variable "m" in the definition of the multivariate normal Fisher Information?
Multivariate normal distribution [edit] The FIM for a $N$-variate multivariate normal distribution, $X \sim N(\mu(\theta), \Sigma(\theta))$ has a special form. Let the $K$-dimensional vector of parameters be $\theta=\left[\begin{array}{lll}\theta_{1} & \ldots & \theta_{K}\end{array}\right]^{\top}$ and the vector of random normal variables be $X=\left[\begin{array}{lll}X_{1} & \ldots & X_{N}\end{array}\right]^{\top} .$ Assume that the mean values of these random variables are $\mu(\theta)=\left[\mu_{1}(\theta) \quad \ldots \quad \mu_{N}(\theta)\right]^{\top}$, and let $\Sigma(\theta)$ be the covariance matrix. Then, for $1 \leq m, n \leq K$, the $(m, n)$ entry of the FIM is: $^{[16]}$ $$ \mathcal{I}_{m, n}=\frac{\partial \mu^{\top}}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \mu}{\partial \theta_{n}}+\frac{1}{2} \operatorname{tr}\left(\Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{n}}\right) $$ where $(\cdot)^{\top}$ denotes the transpose of a vector, $\operatorname{tr}(\cdot)$ denotes the trace of a square matrix, and: $$ \begin{aligned} \frac{\partial \mu}{\partial \theta_{m}} &=\left[\begin{array}{lll} \frac{\partial \mu_{1}}{\partial \theta_{m}} & \frac{\partial \mu_{2}}{\partial \theta_{m}} & \cdots & \frac{\partial \mu_{N}}{\partial \theta_{m}} \end{array}\right]^{\top} \\ \frac{\partial \Sigma}{\partial \theta_{m}} &=\left[\begin{array}{cccc} \frac{\partial \Sigma_{1,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{1,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{1, N}}{\partial \theta_{m}} \\ \frac{\partial \Sigma_{2,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{2,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{2, N}}{\partial \theta_{m}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial \Sigma_{N, 1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{N, 2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{N, N}}{\partial \theta_{m}} \end{array}\right] \end{aligned} $$
Most importantly: What is the variable "m" in the definition of the multivariate normal Fisher Information??
This Wikipedia Definition does not make sense as the fisher information is a metric tensor induced by the hypothesis space and therefore is guaranteed to be symmetric. Writting each entry of the fisher information as $\mathcal{I}_{m,n}$ where $1\leq m, n\leq K$ suggests the Fisher Information Matrix is not symmetric. We do not even have an upper bound on "m"!
Also Is K, the # parameters equal to the N, the number of different mean variables?
What's supposed to be the dimensions of the Fisher info for a multivariate normal?
Source: https://en.wikipedia.org/wiki/Fisher_information
multivariate-normal-distribution fisher-information bounds
GermaniaGermania
2241010 bronze badges
$\begingroup$ I suppose that first part is quoted from Wikipedia, so better to quote it. Also, put the question in the question body. $\endgroup$
$\begingroup$ Is "m" supposed to be the number of mean variables? I think you should have one mean variable per every variance variable . $\endgroup$
– Germania
m is an index into the parameters.
Yes you do have an upper bound on m. The notation you show means that both m and n are bounded by K.
K is the number of parameters, which has nothing to do with the number of mean variables.
The dimension is given in the notation: it's a square matrix.
Neil GNeil G
$\begingroup$ The unknown parameters in a multivariate standard normal are the means and the squared standard deviations. Is it true that K=# unknown means/2=#unknown standard deviations squared/2? If K=#unknown means/2, then K is related with # mean variables. I cannot imagine a case where you know some of the means, and none of the standard deviations squared, and therefore K!= unknown-means/2. $\endgroup$
$\begingroup$ @Germania No, K is the number of parameters in the model. $\endgroup$
$\begingroup$ A parameter is simply what you do not know and have to estimate. By definition of parameter, a mean could be unknown and is estimated, and so is a squared standard deviation. The Multivariate NormalModel has mus and sigmas unknown. $\endgroup$
$\begingroup$ @Germania You have it right there in your question. K is the length of $\theta$. $\endgroup$
$\begingroup$ I don't understand why the means are a function of $\theta$. The means themselves are examples of $\theta$? $\endgroup$
Not the answer you're looking for? Browse other questions tagged multivariate-normal-distribution fisher-information bounds or ask your own question.
Why is the Fisher Information matrix positive semidefinite?
Why exactly is the observed Fisher information used?
Basic question on the definition of the Fisher information
For a Fisher Information matrix $I(\theta)$ of multiple variables, is it true that $I(\theta) = nI_1(\theta)$?
score function of bivariate/multivariate normal distribution
Confused about notation in definition of Fisher Information matrix
What is the fisher information matrix of the multivariate t distribution?
Fisher information of $\rho$ in a symmetric normal $N_p(\mathbf 0,\Sigma)$ distribution