Central limit theorem

Motivation

We know the Sample mean ${\overline{X}}_n$ estimates the true population mean $\mu$. We also know by the Law of large numbers that as $n$ approaches infinity, ${\overline{X}}_n$ approaches $\mu$.

However, our sample size is usually never infinite. Without using LLN, can we still get an accurate measurement of the error of $|{\overline{X}}_n-\mu |$ without $n\to \infty$?

Claim

$n$ does not need to approach infinity. However, we can say that as the sample size increases, ${\overline{X}}_n$ will approach the Normal, even if $X_1,X_2,\dots ,X_n$ are not Normal.

In other words, for a large enough size of $n$, we have approximately

$${\overline{X}}_n\sim \mathcal{N}\left(\mathbb{E}({\overline{X}}_n),\text{Var}({\overline{X}}_n)\right)$$

Formally

Let $Z_n$ be the standardized version (see Standardization) of ${\overline{X}}_n$. That is, given $\frac{\sigma }{\sqrt{n}}=\text{SD}({\overline{X}}_n)$ and $\mu =\mathbb{E}({\overline{X}}_n)$, we have

$$Z_n=\frac{{\overline{X}}_n-\mu }{\sigma /\sqrt{n}}=\sqrt{n}\left(\frac{{\overline{X}}_n-\mu }{\sigma }\right)$$

Then for every $z\in \mathbb{R}$,

$$\underset{n\to \infty }{\lim }P(Z_n\le z)=\Phi (z)$$

We say that $Z_n$ converges in distribution (or in law) to $\mathcal{N}(0,1)$. That is, for large enough values of $n$, $Z_n$ is approximately equal to $\mathcal{N}(0,1)$.