Covariance

The covariance between $X$ and $Y$ is a summary of their relationship. It’s asking if one variable goes up in value, whether the other variable tends to go up as well (positive covariance) or down (negative).

While in practice we don’t use it as often directly (often, the Correlation is easier to interpret), it is involved in a lot of important topics.

Definition

Mathematically, it is defined as (using square brackets simply for differentiation):

$$\text{Cov}(X,Y)=\mathbb{E}\left([X-\mathbb{E}(X)]\cdot [Y-\mathbb{E}(Y)]\right)$$

Just like Variance, we have another formula that is much easier to work with in practice:

$$\text{Cov}(X,Y)=\mathbb{E}(XY)-\mathbb{E}(X)\cdot \mathbb{E}(Y)$$

The similarity is not a coincidence. The variance is just a special case of the covariance. Given $X$, we have $\text{Var}(X)=\text{Cov}(X,X)=\mathbb{E}(XX)-\mathbb{E}(X)\cdot \mathbb{E}(X)=\mathbb{E}(X^2)-\mathbb{E}(X{)}^2$.

Properties

• If $X$ and $Y$ are Independent, then $\text{Cov}(X,Y)=0$. However, the equation being true does not imply independence.
• $\text{Cov}(X,Y)=\text{Cov}(Y,X)$
• $\text{Cov}(X,c)=0$ for any constant $c$.
• $\text{Cov}(aX,Y)=a\text{Cov}(X,Y)$ for any constant $a$.
• $\text{Cov}(X+Y,Z)=\text{Cov}(X,Z)+\text{Cov}(Y,Z)$
• $\text{Cov}(X+Y,Z+W)=\text{Cov}(X,Z)+\text{Cov}(X,W)+\text{Cov}(Y,Z)+\text{Cov}(Y,W)$