Independence

Events

Events $A$ and $B$ are independent if

$$P(A\cap B)=P(A)\cdot P(B)$$

Intuitively, it means that if we know the probability of event $A$, this does not tell us anything about event $B$, and whether it has occurred.

In addition, if $P(A)>0$ and $P(B)>0$ and $A$ and $B$ are independent, then the Conditional probability of $P(A\mid B)=P(A)$ as well as $P(B\mid A)=P(B)$.

Random variables

Two Random variables $X,Y$ are independent if

$$P(X\le x,Y\le y)=P(X\le x)\cdot P(Y\le y)$$

for all $x,y\in \mathbb{R}$. In terms of the CDF and joint CDF, we can also write

$$F_{X,Y}(x,y)=F_X(x)\cdot F_Y(y)$$

Continuous

If $X$ and $Y$ are continuous with joint PDF $f_{X,Y}(x,y)$, then the following statements are all equivalent:

• $X$ and $Y$ are independent.
• $f_{X,Y}(x,y)=f_X(x)\cdot f_Y(y)$ for all $x,y\in \mathbb{R}$. That is, we can factor the joint PDF to be the product of two functions $g(x)$ and $h(y)$.
• $f_{Y\mid X}(y\mid x)=f_Y(y)$ for all $x$ such that $f_X(x)>0$.
• $f_{X\mid Y}(x\mid y)=f_X(x)$ for all $y$ such that $f_Y(y)>0$.

Expectation of their product

If $X$ and $Y$ are independent r.v.s, then $\mathbb{E}(XY)=\mathbb{E}(X)\cdot \mathbb{E}(Y)$. Note that it is not always true the other way around (the equation being true doesn’t imply independence).

Proof

We first know that $f_{X,Y}(x,y)=f_X(x)\cdot f_Y(y)$ is true. Then by 2D LOTUS, we know that $\mathbb{E}(XY)={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }xy\cdot f_X(x)\cdot f_Y(y)dxdy$. The inner integral can be refactored to become $y\cdot f_Y(y){\int }_{-\infty }^{\infty }x\cdot f_X(x)dx$, and the integral is now simply the definition of the PDF, which integrates to 1.

The complete integral is now ${\int }_{-\infty }^{\infty }y\cdot f_Y(y)\cdot \mathbb{E}(X)dy$. In this context, $\mathbb{E}(X)$ is constant because it does not depend on $y$, so we can take it outside the integral, leaving us the definition of the PDF again. Therefore, we get $\mathbb{E}(X)\cdot \mathbb{E}(Y)$.