text: # Variance of Sample Variance
  
  Given $$X_1,...,X_n$$ iid to a certain distribution (not necessarily normal), with $$\mathbb{E}(X_i)=\mu$$ and $$\mathbb{V}(X_i)=\sigma^2$$, I'm trying to deduce the standard and mean squared error of the estimator $$\widehat{\sigma}^2=S_n^2$$, where $$S_n^2$$ is the sample variance, given by$$S_n^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\overline{X}_n)^2.$$ To do so I need its variance, $$\mathbb{V}(S_n^2)$$. Since I know the expectation $$\mathbb{E}(S_n^2)=\sigma^2$$, I started by expanding$$\mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4$$ but I'm stuck with the expansion of the term $$\mathbb{E}(S_n^4)$$. Any ideas?
  
  P.S. - I saw user940's answer on this question, but I was looking for a different approach, also not assuming normal distributed random variables.
  
  • Emm how different approach you want? I have typed a 2-3 pages of full derivation for this starting from the Casella Berger Ex hints also. So basically it was just an expansion. If you start from this definition of $S^2$, I believe you will end up facing with the same expansion problem, but slightly longer.
  – BGM
  Commented Oct 17, 2017 at 11:02
  • You're right, I guess there is no way around it. I ended up doing the expansion and left my solution below. Commented Oct 17, 2017 at 12:00
  
  Let $\mu_k$ denote the $k$th central momentum of $X_i$, i.e, $\mu_k=\mathbb{E}((X_i-\mu)^k)$, and $Z_i\equiv X_i-\mu$ for all $i$. Thus $\mathbb{E}(Z_i)=0$. Since$$\mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4,$$we derive an expression of $\mathbb{E}(S_n^4)$ in terms of $n$ and the moments. We can rewrite $S_n^2$ as\begin{align*} S_n^2=\frac{n\sum_{i=1}^n Z_i^2-(\sum_{i=1}^nZ_i)^2}{n(n-1)}. \end{align*} Squaring leads to\begin{align*} S_n^4=\frac{n^2(\sum_{i=1}^nZ_i^2)^2-2n(\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2+(\sum_{i=1}^nZ_i)^4}{n^2(n-1)^2} \end{align*} and so\begin{align*} \mathbb{E}(S_n^4)=\frac{n^2\mathbb{E}((\sum_{i=1}^nZ_i^2)^2)-2n\mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)+\mathbb{E}((\sum_{i=1}^n Z_i)^4)}{n^2(n-1)^2}. \end{align*} Since $Z_1,...,Z_n$ are independent, we have that, for distinct $i,j,k$,\begin{align*} \mathbb{E}(Z_iZ_j)=0,\hspace{5mm}\mathbb{E}(Z_i^3Z_j)=0,\hspace{5mm}\mathbb{E}(Z_i^2Z_jZ_k)=0 \end{align*} and\begin{align*} E(Z_i^2Z_j^2)=\mu_2^2=\sigma^4,\hspace{5mm}\mathbb{E}(Z_i^4)=\mu_4. \end{align*} Then, with simple algebraic simplifications, it can be shown that the following holds\begin{align*} \mathbb{E}((\sum_{i=1}^nZ_i^2)^2)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}((\sum_{i=1}^n Z_i)^4)&=n\mu_4+3n(n-1)\sigma^4. \end{align*} Substituting these into the expansion of $\mathbb{E}(S_n^4)$ and simplifying leads to\begin{align*} \mathbb{E}(S_n^4)=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)} \end{align*} and so\begin{align}\label{var} \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-\sigma^4=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)}-\sigma^4=\frac{\mu_4}{n}-\frac{\sigma^4(n-3)}{n(n-1)}. \end{align}
metadata: {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 1, "mathjax_display_tex": 1, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 10, "wp-katex-eq": 0, "align": 1, "equation": 0, "x-ck12": 0, "texerror": 0}