Fundamental theorem in probability theory and statistics
In probability theory, the central limit theorem (CLT) states that, under appropriate conditions, the distribution of a normalized version of the sample mean converges to a standard normal distribution. This holds even if the original variables themselves are not normally distributed. There are several versions of the CLT, each applying in the context of different conditions.
The theorem is a key concept in probability theory because it implies that probabilistic and statistical methods that work for normal distributions can be applicable to many problems involving other types of distributions.
This theorem has seen many changes during the formal development of probability theory. Previous versions of the theorem date back to 1811, but in its modern form it was only precisely stated as late as 1920.
In statistics, the CLT can be stated as: let formula_1 denote a statistical sample of size formula_2 from a population with expected value (average) formula_3 and finite positive variance formula_4, and let formula_5 denote the sample mean (which is itself a random variable). Then the limit as formula_6 of the distribution of formula_7 is a normal distribution with mean formula_8 and variance formula_4.
In other words, suppose that a large sample of observations is obtained, each observation being randomly produced in a way that does not depend on the values of the other observations, and the average (arithmetic mean) of the observed values is computed. If this procedure is performed many times, resulting in a collection of observed averages, the central limit theorem says that if the sample size is large enough, the probability distribution of these averages will closely approximate a normal distribution.
The central limit theorem has several variants. In its common form, the random variables must be independent and identically distributed (i.i.d.). This requirement can be weakened; convergence of the mean to the normal distribution also occurs for non-identical distributions or for non-independent observations if they comply with certain conditions.
The earliest version of this theorem, that the normal distribution may be used as an approximation to the binomial distribution, is the de Moivre–Laplace theorem.
Independent sequences.
Classical CLT.
Let formula_10 be a sequence of i.i.d. random variables having a distribution with expected value given by formula_3 and finite variance given by formula_12 Suppose we are interested in the sample average
formula_13
By the law of large numbers, the sample average converges almost surely (and therefore also converges in probability) to the expected value formula_3 as formula_15
The classical central limit theorem describes the size and the distributional form of the stochastic fluctuations around the deterministic number formula_3 during this convergence. More precisely, it states that as formula_2 gets larger, the distribution of the normalized mean formula_18, i.e. the difference between the sample average formula_19 and its limit formula_20 scaled by the factor formula_21, approaches the normal distribution with mean formula_22 and variance formula_12 For large enough formula_24 the distribution of formula_19 gets arbitrarily close to the normal distribution with mean formula_3 and variance formula_27
The usefulness of the theorem is that the distribution of formula_18 approaches normality regardless of the shape of the distribution of the individual formula_29 Formally, the theorem can be stated as follows:
 ;templatestyles src="Math_theorem/styles.css" / ;
Lindeberg–Lévy CLT — 
Suppose formula_30 is a sequence of i.i.d. random variables with formula_31 and formula_32 Then, as formula_2 approaches infinity, the random variables formula_18 converge in distribution to a normal formula_35:
In the case formula_37 convergence in distribution means that the cumulative distribution functions of formula_18 converge pointwise to the cdf of the formula_35 distribution: for every real number formula_40
formula_41
where formula_42 is the standard normal cdf evaluated at formula_43 The convergence is uniform in formula_44 in the sense that
formula_45
where formula_46 denotes the least upper bound (or supremum) of the set.
Lyapunov CLT.
In this variant of the central limit theorem the random variables formula_47 have to be independent, but not necessarily identically distributed. The theorem also requires that random variables formula_48 have moments of some order and that the rate of growth of these moments is limited by the Lyapunov condition given below.
 ;templatestyles src="Math_theorem/styles.css" / ;
Lyapunov CLT — Suppose formula_49 is a sequence of independent random variables, each with finite expected value formula_50 and variance Define
formula_51
If for some "Lyapunov’s condition"
formula_52
is satisfied, then a sum of formula_53 converges in distribution to a standard normal random variable, as formula_54 goes to infinity:
In practice it is usually easiest to check Lyapunov's condition for 
If a sequence of random variables satisfies Lyapunov's condition, then it also satisfies Lindeberg's condition. The converse implication, however, does not hold.
Lindeberg (-Feller) CLT.
In the same setting and with the same notation as above, the Lyapunov condition can be replaced with the following weaker one (from Lindeberg in 1920).
Suppose that for every formula_56
formula_57
where formula_58 is the indicator function. Then the distribution of the standardized sums
formula_59
converges towards the standard normal distribution 
CLT for the sum of a random number of random variables.
Rather than summing an integer number formula_2 of random variables and taking formula_61, the sum can be of a random number formula_62 of random variables, with conditions on formula_62.
 ;templatestyles src="Math_theorem/styles.css" / ;
Robbins CLT — Let formula_64 be independent, identically distributed random variables with formula_65 and formula_66, and let formula_67 be a sequence of non-negative integer-valued random variables that are independent of formula_64. Assume for each formula_69 that formula_70 and
formula_71
where formula_72 denotes convergence in distribution and formula_73 is the normal distribution with mean 0, variance 1.
Then
formula_74
Multidimensional CLT.
Proofs that use characteristic functions can be extended to cases where each individual formula_75 is a random vector in with mean vector formula_76 and covariance matrix formula_77 (among the components of the vector), and these random vectors are independent and identically distributed. The multidimensional central limit theorem states that when scaled, sums converge to a multivariate normal distribution. Summation of these vectors is done component-wise. 
For formula_78 let
formula_79
be independent random vectors. The sum of the random vectors formula_80 is
formula_81
and their average is
formula_82
Therefore,
formula_83
The multivariate central limit theorem states that
formula_84
where the covariance matrix formula_85 is equal to
formula_86
The multivariate central limit theorem can be proved using the Cramér–Wold theorem.
The rate of convergence is given by the following Berry–Esseen type result:
 ;templatestyles src="Math_theorem/styles.css" / ;
Theorem — Let formula_87 be independent formula_88-valued random vectors, each having mean zero. Write formula_89 and assume formula_90 is invertible. Let formula_91 be a formula_92-dimensional Gaussian with the same mean and same covariance matrix as formula_93. Then for all convex sets formula_94,
formula_95 is necessary.
The Generalized Central Limit Theorem.
The Generalized Central Limit Theorem (GCLT) was an effort of multiple mathematicians (Bernstein, Lindeberg, Lévy, Feller, Kolmogorov, and others) over the period from 1920 to 1937. The first published complete proof of the GCLT was in 1937 by Paul Lévy in French. An English language version of the complete proof of the GCLT is available in the translation of Gnedenko and Kolmogorov's 1954 book.
The statement of the GCLT is as follows:
"A non-degenerate random variable" "Z" "is "α"-stable for some" 0 ; "α" ≤ 2 "if and only if there is an independent, identically distributed sequence of random variables" "X"1, "X"2, "X"3, "... and constants" "a""n" ; 0, "b""n" ∈ ℝ "with"
"a""n" ("X"1 + ... + "X""n") − "b""n" → "Z".
"Here → means the sequence of random variable sums converges in distribution; i.e., the corresponding distributions satisfy" "F""n"("y") → "F"("y") "at all continuity points of" "F."
In other words, if sums of independent, identically distributed random variables converge in distribution to some "Z", then "Z" must be a stable distribution.
Dependent processes.
CLT under weak dependence.
A useful generalization of a sequence of independent, identically distributed random variables is a mixing random process in discrete time; "mixing" means, roughly, that random variables temporally far apart from one another are nearly independent. Several kinds of mixing are used in ergodic theory and probability theory. See especially strong mixing (also called α-mixing) defined by formula_96 where formula_97 is so-called strong mixing coefficient.
A simplified formulation of the central limit theorem under strong mixing is:
 ;templatestyles src="Math_theorem/styles.css" / ;
Theorem — Suppose that formula_49 is stationary and formula_99-mixing with formula_100 and that formula_101 and Denote then the limit
formula_102
In fact,
formula_106
where the series converges absolutely.
The assumption formula_103 cannot be omitted, since the asymptotic normality fails for formula_108 where formula_109 are another stationary sequence.
There is a stronger version of the theorem: the assumption formula_110 is replaced with X_n\right and the assumption formula_100 is replaced with
formula_112
Existence of such formula_113 ensures the conclusion. For encyclopedic treatment of limit theorems under mixing conditions see .
Martingale difference CLT.
 ;templatestyles src="Math_theorem/styles.css" / ;
Theorem —  Let a martingale formula_114 satisfy
Remarks.
Proof of classical CLT.
The central limit theorem has a proof using characteristic functions. It is similar to the proof of the (weak) law of large numbers.
Assume formula_119 are independent and identically distributed random variables, each with mean formula_120 and finite variance The sum formula_121 has mean formula_122 and variance Consider the random variable
formula_123
where in the last step we defined the new random variables each with zero mean and unit variance The characteristic function of formula_124 is given by
formula_125
where in the last step we used the fact that all of the formula_126 are identically distributed. The characteristic function of formula_127 is, by Taylor's theorem,
formula_128
where formula_129 is "little o notation" for some function of formula_130 that goes to zero more rapidly than By the limit of the exponential function the characteristic function of formula_131 equals
formula_132
All of the higher order terms vanish in the limit The right hand side equals the characteristic function of a standard normal distribution formula_117, which implies through Lévy's continuity theorem that the distribution of formula_124 will approach formula_135 as Therefore, the sample average
formula_136
is such that 
formula_137
converges to the normal distribution from which the central limit theorem follows.
Convergence to the limit.
The central limit theorem gives only an asymptotic distribution. As an approximation for a finite number of observations, it provides a reasonable approximation only when close to the peak of the normal distribution; it requires a very large number of observations to stretch into the tails.
The convergence in the central limit theorem is uniform because the limiting cumulative distribution function is continuous. If the third central moment formula_138 exists and is finite, then the speed of convergence is at least on the order of formula_139 (see Berry–Esseen theorem). Stein's method can be used not only to prove the central limit theorem, but also to provide bounds on the rates of convergence for selected metrics.
The convergence to the normal distribution is monotonic, in the sense that the entropy of formula_124 increases monotonically to that of the normal distribution.
The central limit theorem applies in particular to sums of independent and identically distributed discrete random variables. A sum of discrete random variables is still a discrete random variable, so that we are confronted with a sequence of discrete random variables whose cumulative probability distribution function converges towards a cumulative probability distribution function corresponding to a continuous variable (namely that of the normal distribution). This means that if we build a histogram of the realizations of the sum of n independent identical discrete variables, the piecewise-linear curve that joins the centers of the upper faces of the rectangles forming the histogram converges toward a Gaussian curve as n approaches infinity; this relation is known as de Moivre–Laplace theorem. The binomial distribution article details such an application of the central limit theorem in the simple case of a discrete variable taking only two possible values.
Common misconceptions.
Studies have shown that the central limit theorem is subject to several common but serious misconceptions, some of which appear in widely used textbooks. These include: 
Relation to the law of large numbers.
The law of large numbers as well as the central limit theorem are partial solutions to a general problem: "What is the limiting behavior of as n approaches infinity?" In mathematical analysis, asymptotic series are one of the most popular tools employed to approach such questions.
Suppose we have an asymptotic expansion of formula_141:
formula_142
Dividing both parts by "φ"1("n") and taking the limit will produce "a"1, the coefficient of the highest-order term in the expansion, which represents the rate at which "f"("n") changes in its leading term.
formula_143
Informally, one can say: ""f"("n") grows approximately as "a"1"φ"1("n")". Taking the difference between "f"("n") and its approximation and then dividing by the next term in the expansion, we arrive at a more refined statement about "f"("n"):
formula_144
Here one can say that the difference between the function and its approximation grows approximately as "a"2"φ"2("n"). The idea is that dividing the function by appropriate normalizing functions, and looking at the limiting behavior of the result, can tell us much about