Sample mean

Given a set of measurements $X_1,X_2,\dots ,X_n$ that are i.i.d., we can estimate their mean using a sample mean ${\overline{X}}_n$, which is given by

$${\overline{X}}_n=\frac{X_1+X_2+\cdots +X_n}{n}=\frac{1}{n}\sum _{i=1}^n{X}_i$$

This is just the definition of the average. There’s nothing special about it, except we’re adding up random variables instead of numbers.

It is important to note we’re not assuming the distribution that $X_i$ follows. It plays a large role in properties like the Law of large numbers and the Central limit theorem.

The sample mean is also a random variable

From the definition above, we can see that the sample mean is simply the sum of a bunch of r.v.s, divided by a constant. This means that ${\overline{X}}_n$ is also a sample mean, with a Expectation and Variance.

Finding the expectation

By LOE, we have

$$\mathbb{E}({\overline{X}}_n)=\mathbb{E}\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)=\frac{1}{n}\sum _{i=1}^n\mathbb{E}(X_i)$$

For some $X_i$ where $i\in [\mathrm{1..}n]$, let $\mathbb{E}(X_i)=\mu$. Then, it must be true that $\mathbb{E}(X_i)=\mu$ for all $X_i$. This is because they all have the same distribution.

Therefore $\mathbb{E}({\overline{X}}_n)=\frac{1}{n}\cdot n\cdot \mu =\mu$. In practice, this means we can just find $\mathbb{E}(X_i)$ for any of the individual r.v.s to find $\mathbb{E}({\overline{X}}_n)$.

Finding the variance

Using Properties of variance, we have

$$\text{Var}({\overline{X}}_n)=\text{Var}\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)=\frac{1}{n^2}\cdot \text{Var}\left(\sum _{i=1}^n{X}_i\right)$$

Since we know all the variables are Independent, it follows directly that

$$\text{Var}({\overline{X}}_n)=\frac{1}{n^2}\cdot \text{Var}\left(\sum _{i=1}^n{X}_i\right)=\frac{1}{n^2}\cdot \sum _{i=1}^n\text{Var}(X_i)=\frac{1}{n^2}\cdot n\cdot {\sigma }^2=\frac{{\sigma }^2}{n}$$

In practice, we can find ${\sigma }^2=\text{Var}(X_i)$ for any $X_i$, and then simply divide by $n$.