Sometimes we want to find the probability of events that involve two or more Random variables. For example, finding , where and represent tomorrow’s high and low temp, respectively.
Also, while we’ve discussed ways of summarizing a distribution of a single r.v. (with Expectation and Variance), we would also like a way to explain the relationship between two r.v.s.
Discrete variables
There are three kinds of PMFs that we can observe between two discrete random variables. Given two discrete and , we have:
- The joint PMF is given by the function and equals . Three or more variables follow the same pattern.
- WLOG, the marginal PMF of is the function given by
Note that this is just the original PMF that we’ve learned before. Here, we are summing up across the whole support of . The only new part is that can be defined in terms of the joint PMF.
- WLOG, for each value , the conditional PMF of , given , is simply the equation for Conditional probability.
Here, we’re holding the value of constant. Depending on , we will get a different conditional probability for .
Continuous variables
Motivation
Remember that in PDFs, the probability that a variable is exactly one value is zero. But, over a range, the probability is positive.
Consider two continuous r.v.s, and . If we were to draw a plane, with on the horizontal axis and on the vertical, something similar occurs.
- Any point, line, or curve in the plane has zero probability.
- However, regions of the plane (e.g. ) will have positive probability.
Then, the probability of an event involving both and will be the volume, which we will calculate with a double integral over the relevant region.
Given two continuous r.v.s and , we can define the following.
Joint CDF
This is a function given by
It is very similar to a regular CDF for one variable. Three or more variables follow the same pattern. Like regular CDFs, this definition also holds for discrete r.v.s.
Joint PDF
The joint PDF is the derivative of their joint CDF with respect to and .
Valid joint PDFs are nonnegative for all and integrates to over , which makes sense, given that it is the total volume under the PDF.
For any set , we have
which just means that given and , we integrate over .
Bound definitions
Be careful with how the bounds are defined for and . is different from and also changes the bounds for the integration. The latter would have the double integral
This is because always has to be greater than , so its lower bound has to start where ends.
Marginal PDF
Just like with the marginal PMF, WLOG we can get the marginal PDF of by integrating over the entire support of , i.e. . This is just the regular PDF for .
Conditional PDF
WLOG, for each value of , the conditional PDF of , given , is
Note that , so we’re actually calculating the region defined by and taking the limit as .
Factoring
If the joint PDF of and can be factored as for all , where and are nonnegative functions, then and are Independent.
- Useful when we don’t know their marginal PDFs, or if and are the marginal PDFs.
- Then, to get their marginal PDFs, we simply “rescale” and so they both integrate to 1.