Expectation

The expectation (also called expected value or just the mean) of a discrete r.v. $X$ is a weighted average of all the possible values of $X$. the weight is the probability that a value occurs.

For a r.v. $X$ with $n$ possible values $x_1,x_2,\dots ,x_n$, we have

$$\mathbb{E}(X)=\sum _{i=1}^n{x}_i\cdot P(X=x_i)$$

This also applies when $n=\infty$, but only when the infinite series converges absolutely (otherwise, $\mathbb{E}(X)$ is undefined). We usually assume $\mathbb{E}(X)$ is well-defined.

Tip

Remember that $\mathbb{E}(X)$ is not a function. It’s a real number and a constant, and does not depend on anything. Given a r.v. $X$, we have a $\mathbb{E}(X)$ that (usually) exists and is a single number.

One important and useful property is the Linearity of expectation.

Finding the expectation

“Adam’s law”

Also known as the Law of Iterated Expectations (LIE), Law of Total Expectation, or the Tower property. “Adam’s law” is a joke from Harvard relating it to “Eve’s law.”

For any random variables $X$ and $Y$,

$$\mathbb{E}(Y)=\mathbb{E}(\mathbb{E}(Y\mid X))$$

where $\mathbb{E}(Y\mid X)$ is the conditional expectation of $X$ and $Y$.

Sum of indicator variables

To calculate the expectation of $X$, we can use the following strategy:

1. Represent $X$ as a sum of Indicator variables. To figure out what the indicators are, think about what $X$ is exactly counting.
2. Calculate the expected value of each indicator.
3. By LOE, we sum up all the expectations to get $\mathbb{E}(X)$.

Darth Vader rule

“This is not a reference to a discoverer. But the designation may capture the somewhat counterintuitive—if not slightly unsettling and surreal—impression which the result can evoke on first encounter.” from P. Muldowney et al. (2012)

We can also calculate the expectation via a survival function. Let $X$ be a r.v. with support $[\mathrm{0..}b]\in \mathbb{Z}$. then we can write

$$\mathbb{E}(X)=\sum _{k=1}^{b}P(X\ge k)$$