Multivariate Normal distribution

The Multivariate Normal (MVN) is a Joint distribution that generalizes the Normal distribution. It helps us determine if a linear combination of multiple Random variables is also approximately normal, given that they follow the Normal.

Random vector

A random vector is just an ordered list of random variables. The order matters. For example, $(X,Y)$ is different from $(Y,X)$. A vector $\mathbf{X}=(X_1,X_2,\dots X_k)$ is called a $k$-dimensional vector.

Definition

A random vector $\mathbf{X}=(X_1,X_2,\dots X_k)$ has the Multivariate Normal joint distribution if every linear combination of the $X_j$‘s also has a Normal distribution.

In other words, $\mathbf{X}$ is MVN if for every choice of constants $t_1,t_2,\dots ,t_k$, we require $t_1{X}_1+\cdots +t_k{X}_k$ to also have a Normal distribution.

We also allow $t_1{X}_1+t_2{X}_2+\dots t_k{X}_k$ to be constant. However, the probability for this needs to be 100%, otherwise it does not follow MVN.

When the vector is 2-dimensional, the MVN distribution is also called the Bivariate Normal.

Parameters

The parameters of a MVN distribution are:

• The means of $X_1,X_2,\dots ,X_k$
• The Variances of $X_1,X_2,\dots ,X_k$
• The covariance or Correlation between each pair of random variables among $X_1,X_2,\dots ,X_k$

The variance and covariance parameters are often listed in a $k\times k$ matrix called the covariance matrix or variance-covariance matrix, where row and columns are the r.v.s $X_1,X_2,\dots ,X_k$.

• From the definition of covariance this implies the diagonal entries are the variance.
• The bottom left triangle is equivalent to the upper right triangle.

Properties

If $(X_1,X_2,\dots ,X_k)$ is Multivariate Normal, then each individual r.v. $X_j$ is also Normal. We can show this by treating $X_1$ as a linear combination of the other random variables:

$$X_1=1\cdot X_1+0\cdot X_2+0\cdot X_3+\cdots +0\cdot X_k$$

and therefore by the definition of MVN, $X_1$ is Normal.

However, the converse is not true: knowing that $X_1$, $X_2$, etc. are Normal does not imply that $(X_1,X_2,\dots )$ is Multivariate Normal.

If $X_1,X_2,\dots ,X_k$ are Independent Normal random variables, then $(X_1,X_2,\dots ,X_k)$ is a Multivariate Normal. This follows from the fact that the sum of independent Normals is Normal.

Suppose that $(X_1,X_2,\dots )$ is MVN. We can show that a set of variables $Y_1,Y_2,\dots$ is also MVN if for every choice of constants $t_1,t_2,\dots$, we can rewrite it as a linear combination of $X_1,X_2,\dots$.

• If every linear combination of $Y_1,Y_2,\dots$ can be expressed equivalently as a linear combination of $X_1,X_2,\dots$, then there’s a direct mapping from one to the other.
• And since $(X_1,X_2,\dots )$ is MVN, it follows that $(Y_1,Y_2,\dots )$ must also be MVN.

If $(X,Y)$ is Bivariate Normal and the Covariance $\text{Cov}(X,Y)=0$, then $X$ and $Y$ are independent. Remember that in the general case, just because the covariance is zero doesn’t mean the variables are independent. But it’s true for this case.