Law of the unconscious statistician

Discrete version

Suppose $X$ is a discrete r.v. and $Y=g(X)$. By definition, the Expectation $\mathbb{E}(g(X))=\mathbb{E}(Y)={\sum }_{y}y\cdot P(Y=y)$. However, this requires us to find $P(Y=y)$, the PMF of $Y$. Instead, if we already have the PMF of $X$, we can use that instead:

$$\mathbb{E}(Y)=\sum _{y}y\cdot P(Y=y)=\sum _{x}g(x)\cdot P(X=x)$$

Continuous version

This formula is even more useful in the continuous version, because finding the PDF of $Y$ given $X$‘s PDF is often very nontrivial.

Often, when we’re told to not use the PDF when finding expectation, this just means to use LOTUS instead.

Suppose $X$ is a continuous r.v. and $Y=g(X)$, where $g:\mathbb{R}\to \mathbb{R}$. Again, $\mathbb{E}(g(X))=\mathbb{E}(Y)={\int }_{-\infty }^{\infty }{f}_Y(y)dy$. We want to avoid finding $f_y(Y)$, so instead we can calculate $\mathbb{E}(Y)$ as follows, given we know $f_X$.

$$\mathbb{E}(Y)={\int }_{-\infty }^{\infty }y\cdot f_Y(y)dy={\int }_{-\infty }^{\infty }g(x)\cdot f_X(x)dx$$

Calculating variance

This formula is useful when we want to calculate the Variance of $X$ and need $\mathbb{E}(X^2)$. We simply let $Y=X^2$ and so

$$\mathbb{E}(X^2)={\int }_{-\infty }^{\infty }{x}^2\cdot f_X(x)dx$$

2D version

Consider two r.v.s $X$ and $Y$ with different distributions, and let their Joint distribution be $W=XY$. Another way of interpreting this is as a transformation $g(X,Y)=XY$. If we wanted to find $\mathbb{E}(W)$, calculating its PDF may be difficult. Instead, we claim that

$$\mathbb{E}(W)=\mathbb{E}(g(X,Y))=\sum _x\sum _{y}g(x,y)P(X=x,Y=y)$$

for two discrete distributions. If $X$ and $Y$ are both continuous and their joint PDF is $f_{X,Y}$, then we have

$$\mathbb{E}(W)={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }g(x,y)\cdot f_{X,Y}(x,y)dxdy$$