7_R

This is an example that was taken from [1] in order to explain and demonstrate the definitions of marginal, joint, conditional distributions. The table represents a results of the survey that was conducted to determine people's favorite sports in London's West End.

Table 1

Table 2 (contigency table) represents frequency/probability distribution of the first table.

Table 2

Joint frequency/probability. The joint probability is probability of two events occuring together at the same time [1,2]. For events A and B, the joint probability is usually denoted as:

P (A and B) or P (A, B) or P (A & B) or P (A ∩ B)

For example, using Table 2 the joint probability of someone being a male and liking rugby is 0.2 or someone being a female and liking football is 0.15.

The yellow cells in Table 3, the joint probability distribution, must sum to 1 since everyone in the distribution must be in one of the cells.

The joint probability is symmetrical meaning that,

P (A and B) = P (B and A)

For example, P(Male and Football) = P(Football and Male).

Table 3

Marginal distribution. The probability of one event in the presence of all (or a subset of) outcomes of the other random variable is called the marginal probability or the marginal distribution [1,2]. In other words, it is the probability of an event irrespective of the outcome of another variable — P(A) or P(B). For example, the probability of X=A for all outcomes of Y.

If all probabilities for the two variables were laid out together in a table (X as columns, Y as rows), then the marginal probability of one variable (X) would be the sum of probabilities for the other variable (Y rows) on the margin of the table (see red cells on Table 4).

P(X=A) = sum P(X=A, Y=yi) for all y

For example,

P(Male) = 0.24 + 0.2 + 0.1 = 0.54, ignores the sport that male prefer

and

P(Football) = 0.24 + 0.15 = 0.39, ignores the gender.

Note whether ignoring one variable or another, the marginal distributions must sum to 1.

P(Male) + P(Female) = 0.54 + 0.46 = 1

or

P(Football) + P(Rugby) + P(Other) = 0.39 + 0.25 + 0.36 = 1

Table 4

Conditional probability. The conditional probability defines the probability of one event occurring given that another event has occurred (by assumption, presumption, assertion or evidence) [1,2]. It is denoted as

P(A given B) or P(A | B)

and calculated as follows

P (A | B) = P (A and B) / P(B), the conditional probability of event A given event B

and

P(B | A) = P(B and A) / P(A), the conditional probability of event B given event A

For example, the probability that a person would like football given that they are a male,

P(Football | Male) = P(Football and Male) / P(Male),

thus

P(Football | Male) = 0.24 / 0.54 = 0.44

or the probability that a person is a female given that they enjoy rugby

P(Female | Rugby) = P(Female and Rugby) / P(Rugby),

thus

P(Football | Male) = 0.05 / 0.25 = 0.2

Bayes Theorem. Bayes Theorem is another way of calculating a conditional probability without the joint probability [3, 4]. Based on the conditional probabilities, where

P (A | B) = P (A and B) / P(B)

or

P(B | A) = P(B and A) / P(A)

since P(B and A) = P (A and B), results in

P (A | B) * P(B) = P(B | A) * P(A)

thus

P (A | B) = P(B | A) * P(A) / P(B)

or

P (B | A) = P(A | B) * P(B) / P(A)

For example, given the probability that a person is a male given that they enjoy football (P(Male| Football) = 0.615), the probability that a person would like football given that they are a male equals

P(Football | Male) = P(Male| Football) * P(Football) /P(Male)

P(Football | Male) = 0.615 * 0.39 / 0.54 = 0.44, which is similar to previous answer from conditional probabilities.

---

References

1. https://towardsdatascience.com/marginal-joint-and-conditional-probabilities-explained-by-data-scientist-4225b28907a4

2. https://machinelearningmastery.com/joint-marginal-and-conditional-probability-for-machine-learning/

3. https://www.probabilitycourse.com/chapter1/1_4_3_bayes_rule.php

4. https://betterexplained.com/articles/understanding-bayes-theorem-with-ratios/