I am seeking the most transparent exposition of Bayes' Theorem (for undergraduates). I would prefer to avoid mentioning "prior" and "posterior," and instead focus on frequencies. The Wikipedia entry is quite good in some ways, but I disagree with this:

"The role of Bayes’ theorem is best visualized with tree diagrams."

The tree diagrams do little for me. But the associated table is quite useful:

          Wikipedia image.

So $P( A \& B ) = \frac{2}{20}$.

$P( A | B ) = \frac{2}{8}$ and $P(B) = \frac{8}{20}$. So $$P( A \& B ) = P( A | B )\; P(B) = \tfrac{2}{8} \tfrac{8}{20} = \tfrac{2}{20}.$$

$P( B | A ) = \frac{2}{5}$ and $P(A) = \frac{5}{20}$. So $$P( A \& B ) = P( B | A )\; P(A) = \tfrac{2}{5} \tfrac{5}{20} = \tfrac{2}{20}.$$

And this leads to $$P( A \& B ) = P( A | B )\; P(B) = P( B | A )\; P(A),$$ which is one form of Bayes' Theorem.

So here are my questions:

Q1. What is a compelling story one can associate with this specific table, identifying $A$ and $B$ with real-world events that make sense with that table's data?

Q2. What is the clearest justification one could make for $P( A \& B ) = P( A | B )\; P(B)$, independent of any particular interpretation of $A$ and $B$?

  • 1
    $\begingroup$ Can you just have two groups of students who are sitting at Table A and Table B? Say, e.g., 2 wearing red and 3 wearing blue at Table A, and 6 wearing red and 9 wearing blue at Table B. I think students can derive the identities for Bayes' Theorem from such a setup. $\endgroup$ – Benjamin Dickman Aug 25 at 3:07
  • 4
    $\begingroup$ +: If tree diagrams don't do it for you, then maybe Venn Diagrams might? $\endgroup$ – Benjamin Dickman Aug 25 at 3:09
  • 1
    $\begingroup$ Isn’t the Wikipedia image above already basically a Venn diagram, albeit with rectangles instead of the usual circles? $\endgroup$ – Joe Aug 26 at 12:34
  • 2
    $\begingroup$ I think the table makes things more complicated. It has more structure and information than is necessary. It would be better to have a circle that is the whole event space, with a portion shaded representing A, and then a circle within the outer circle representing B. You can infer Pr[A|B] is the portion of B that is shaded, which is Pr[A∧B]/Pr[B]. $\endgroup$ – Solomonoff's Secret Aug 26 at 17:58
  • $\begingroup$ What is your criteria, upon which you will judge, as your title insists, the most transparent exposition of Bayes' Theorem? What counts as a compelling story (and for whom should we aim to compel?), or the clearest justification one could make (according to whom?). The terms you emphasize in your questions are rather subjective. Care to make those terms objective? $\endgroup$ – Namaste Sep 2 at 22:10

Q1. A real-world story that I've heard many times over when Bayes' Theorem is explained concerns medical lab tests where one of the events in your table would be "actually has the disease the test is designed to detect" while another is "experiences a positive test result when the lab test is administered"; however, these scenarios tend to be confusing for people who don't understand positive and negative in the context of test results.

Q2. One way I think about this is to view the conditional probability as being defined by $$P(A|B) = \frac{P(A \cap B)}{P(B)}$$

and $P(A|B) := 0$ when $P(B) = 0$ rather than defining the probability of the intersection as the product of the other two probabilities.

The exposition that goes along with it is "Assume within our universe, event B has occurred (i.e., given $B$). Then $P(A|B)$ is the probability of $A$ in the new, restricted universe".


Your Answer

