Sometimes we want to find the probability of events that involve two or more Random variables. For example, finding , where and represent tomorrow’s high and low temp, respectively.

Also, while we’ve discussed ways of summarizing a distribution of a single r.v. (with Expectation and Variance), we would also like a way to explain the relationship between two r.v.s.

Discrete variables

There are three kinds of PMFs that we can observe between two discrete random variables. Given two discrete and , we have:

  • The joint PMF is given by the function and equals . Three or more variables follow the same pattern.
  • WLOG, the marginal PMF of is the function given by

Note that this is just the original PMF that we’ve learned before. Here, we are summing up across the whole support of . The only new part is that can be defined in terms of the joint PMF.

  • WLOG, for each value , the conditional PMF of , given , is simply the equation for Conditional probability.

Here, we’re holding the value of constant. Depending on , we will get a different conditional probability for .

Continuous variables

Motivation

Remember that in PDFs, the probability that a variable is exactly one value is zero. But, over a range, the probability is positive.

Consider two continuous r.v.s, and . If we were to draw a plane, with on the horizontal axis and on the vertical, something similar occurs.

  • Any point, line, or curve in the plane has zero probability.
  • However, regions of the plane (e.g. ) will have positive probability.

Then, the probability of an event involving both and will be the volume, which we will calculate with a double integral over the relevant region.

Given two continuous r.v.s and , we can define the following.

Joint CDF

This is a function given by

It is very similar to a regular CDF for one variable. Three or more variables follow the same pattern. Like regular CDFs, this definition also holds for discrete r.v.s.

Joint PDF

The joint PDF is the derivative of their joint CDF with respect to and .

Valid joint PDFs are nonnegative for all and integrates to over , which makes sense, given that it is the total volume under the PDF.

For any set , we have

which just means that given and , we integrate over .

Bound definitions

Be careful with how the bounds are defined for and . is different from and also changes the bounds for the integration. The latter would have the double integral

This is because always has to be greater than , so its lower bound has to start where ends.

Marginal PDF

Just like with the marginal PMF, WLOG we can get the marginal PDF of by integrating over the entire support of , i.e. . This is just the regular PDF for .

Conditional PDF

WLOG, for each value of , the conditional PDF of , given , is

Note that , so we’re actually calculating the region defined by and taking the limit as .

Factoring

If the joint PDF of and can be factored as for all , where and are nonnegative functions, then and are Independent.

  • Useful when we don’t know their marginal PDFs, or if and are the marginal PDFs.
  • Then, to get their marginal PDFs, we simply “rescale” and so they both integrate to 1.