Normal distribution

Also called the Gaussian distribution. The support of the Normal distribution is the set of all real numbers: $k\in \mathbb{R}$. There are two parameters:

• The mean (aka Expectation) is represented by $\mu$.
• The Variance is represented by ${\sigma }^2$. Then, $\sigma$ is the standard deviation.

One of the most important and widely used distributions in probability. One of the many reasons is its primary involvement in the Central limit theorem.

Standard Normal distribution

A Random variable $Z$ has the standard Normal distribution with mean $\mu =0$ and variance ${\sigma }^2=1$ if its PDF is the function $\phi$:

$$\phi (z)=\frac{1}{\sqrt{2\pi }}{e}^{-z^2/2}$$

where the support $z\in \mathbb{R}$, or $-\infty <z<\infty$. The textbook writes $Z\sim \mathcal{N}(0,1)$.

The standard Normal distribution is so special that we use $\phi$ instead of $f$ to represent the PDF. Furthermore, we usually use $Z$ as the r.v. letter.

The standard Normal CDF does not have a closed-form expression. We call it $\Phi$ and it is

$$\Phi (z)=P(Z\le z)={\int }_{-\infty }^z\phi (t)dt$$

Key properties

• The PDF is symmetric. This means that $\phi (z)=\phi (-z)$ for all $z\in \mathbb{R}$.
• Probabilities are also symmetric. We have $P(Z\le -z)=P(Z\ge z)$.
• If $Z\sim \mathcal{N}(0,1)$, then it is true that $-Z\sim \mathcal{N}(0,1)$. $P(-Z\le z)=P(Z\ge -z)=1-\Phi (-z)=\Phi (z)$.
• We have $\mathbb{E}(Z)=0$ and $\text{Var}(Z)=1$.

MGF

Using LOTUS, we can derive the MGF to be:

$$M_Z(t)=\mathbb{E}(e^{tZ})={\int }_{-\infty }^{\infty }{e}^{tz}\frac{1}{\sqrt{2\pi }}{e}^{-z^2/2}dz=e^{t^2/2}$$

For all $t\in \mathbb{R}$. The process is long and not trivial so I didn’t write it down.

General Normal distribution

We could define the general Normal by its PDF, but we can also use a location-scale transformation, just like what we did with the general Uniform distribution.

If $Z\sim \mathcal{N}(0,1)$, then for any $\mu \in \mathbb{R}$ and $\sigma >0$, we say that $X=\mu +\sigma Z$ has the Normal distribution with mean $\mu$ and variance ${\sigma }^2$, and $X\sim \mathcal{N}(\mu ,{\sigma }^2)$.

PDF and CDF

Given $X\sim \mathcal{N}(\mu ,{\sigma }^2)$, the CDF of $X$ is given by $F_X(x)=\Phi \left(\frac{x-\mu }{\sigma }\right)$. The PDF of $X$ is calculated by

$$f_X(x)=\phi \left(\frac{x-\mu }{\sigma }\right)\frac{1}{\sigma }=\frac{1}{\sigma \sqrt{2\pi }}\cdot \exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)$$

where $\exp (u)=e^u$. Writing it like this just makes the exponent bigger and easier to see.

Expectation and variance

We have $\mathbb{E}(X)=\mu$ and $\text{Var}(X)={\sigma }^2$.

MGF

Using location-scale transformation properties of the MGF, we can easily get the general form of the Normal now. We have, for all $t\in \mathbb{R}$

$$M_X(t)=e^{\mu t}\cdot M_Z(\sigma t)=e^{\mu t}\cdot e^{({\sigma }^2{t}^2)/2}=\exp \left(\mu t+\frac{1}{2}{\sigma }^2{t}^2\right)$$

Standardization

This is the process of converting a general Normal r.v. to a standard Normal. If $X\sim \mathcal{N}(\mu ,{\sigma }^2)$, then the standardized version (also called the z-score) of $X$ is given by

$$Z=\frac{X-\mu }{\sigma }\sim \mathcal{N}(0,1)$$

Note that we’re dividing by the standard deviation, not the variance.

This is similar to how we can go back and forth between the standard and general Uniform, which is also a continuous distribution.

The 68-95-99.7 rule

Useful for quick approximations of Normal probabilities. Given $X\sim \mathcal{N}(\mu ,{\sigma }^2)$, we have

• $P(\mu -\sigma <X<\mu +\sigma )\approx 0.68$
• $P(\mu -2\sigma <X<\mu +2\sigma )\approx 0.95$
• $P(\mu -3\sigma <X<\mu +3\sigma )\approx 0.997$