The Multivariate Normal (MVN) is a Joint distribution that generalizes the Normal distribution. It helps us determine if a linear combination of multiple Random variables is also approximately normal, given that they follow the Normal.
Random vector
A random vector is just an ordered list of random variables. The order matters. For example, is different from . A vector is called a -dimensional vector.
Definition
A random vector has the Multivariate Normal joint distribution if every linear combination of the ‘s also has a Normal distribution.
In other words, is MVN if for every choice of constants , we require to also have a Normal distribution.
We also allow to be constant. However, the probability for this needs to be 100%, otherwise it does not follow MVN.
When the vector is 2-dimensional, the MVN distribution is also called the Bivariate Normal.
Parameters
The parameters of a MVN distribution are:
- The means of
- The Variances of
- The covariance or Correlation between each pair of random variables among
The variance and covariance parameters are often listed in a matrix called the covariance matrix or variance-covariance matrix, where row and columns are the r.v.s .
- From the definition of covariance this implies the diagonal entries are the variance.
- The bottom left triangle is equivalent to the upper right triangle.
Properties
If is Multivariate Normal, then each individual r.v. is also Normal. We can show this by treating as a linear combination of the other random variables:
and therefore by the definition of MVN, is Normal.
However, the converse is not true: knowing that , , etc. are Normal does not imply that is Multivariate Normal.
If are Independent Normal random variables, then is a Multivariate Normal. This follows from the fact that the sum of independent Normals is Normal.
Suppose that is MVN. We can show that a set of variables is also MVN if for every choice of constants , we can rewrite it as a linear combination of .
- If every linear combination of can be expressed equivalently as a linear combination of , then there’s a direct mapping from one to the other.
- And since is MVN, it follows that must also be MVN.
If is Bivariate Normal and the Covariance , then and are independent. Remember that in the general case, just because the covariance is zero doesn’t mean the variables are independent. But it’s true for this case.