Bayes Theorem and the Monty Hall Problem
This is the first post in a sequence titled “Basic concepts in the formalisation of thought.”
Having written a few posts on Causality I’m now going to write a short sequence exploring basic concepts in the formalisation of thought. This first post will deal with Bayes Theorem. However, because there are already good explanations of Bayes Theorem (for example, this one by Eliezer Yudkowsky and this one on community blog Less Wrong) I’m not going to do a standard basic introduction but will instead apply Bayes Theorem to a well known puzzle, the Monty Hall problem, to show how it can aid with probabilistic reasoning.
What is Bayes Theorem
Bayes Theorem is simply an equation that allows you to calculate conditional probabilities. That is, the probability of A, given B. The equation is:
Which just means that, for example, the probability that I’m hungry given that my belly is rumbling is equal to the number of times that I’m hungry and my belly rumbles out of the number of times that my belly rumbles which is intuitively what we would expect it to say.
There are a number of variants of this formula, but the one that matters for us is Bayes Theorem under the condition that you are given not just one piece of information but two. That formula is as follows:
What is the Monty Hall Problem?
You’re on a quiz program and are faced with three doors. A door has been selected at random before the show and a car placed behind it. The other doors have nothing behind them. You are asked to choose a door.
Once you have done so, the host chooses, at random, one of the other doors. He never chooses the one with the car behind it. He opens the door and then asks whether you would like to change your answer?
Should you do so?
An intuitive response to the Monty Hall Problem?
The initial response is normally “No” or, more to the point, “Who cares. Whatever door I choose there’s a 1/3 chance of winning so why both changing.”
The answer, however, is actually “Yes.” Think about it like this: You had a 1/3 chance of choosing the winning door in the first place. Now, if you change, you will lose. You had a 2/3 chance of choosing a losing door in the first place. Now, if you change, you will win (because the other door has been opened to reveal no prize so you will inevitably change to the winning door). So 2/3 of the time you win by changing, 1/3 of the time you don’t.
What information do you gain?
The reasoning is easy enough to understand. What’s harder is to figure out what new piece of information the presenter gave you by opening the door. It becomes a bit more clear when we consider the following situation: If the host had simply pointed at a door at random, regardless of whether it was the winning door, and not opened it, then changing doors would not be beneficial. So the information is given by the fact that he cannot open the winning door.
So imagine a different situation: The host says you can either guess the door straight up or, before you do, he will point at the door. He will tell you the correct door 2/3 of the time and lie and tell you the incorrect one 1/3 of the time. Do you simply guess or do you wait for him to tell you and then choose the door he points out? If you guess you have a 1/3 chance of winning, if you wait, he is telling the truth 2/3 of the time. So you wait.
The new piece of information in the first case is similar to that: By not choosing a door, the host is giving you information that that is the correct door. It just so happens that this information is only true 2/3 of the time but, it’s still additional information on those occasions.
To put it another way: Even if you were told you’d picked a wrong door, you would still not know which of the other two doors the prize was behind. But after the other wrong door was selected, you would. So his actions in choosing a door give you new information if you knew you’d picked the wrong door. And given that you know you’re 2/3 likely to have picked the wrong door and only 1/3 likely to have picked the right, this information is of use,
Bayes Theorem and the Monty Hall Problem
All of this is well and good in relation to the specific problem but, unless you got it right the first time you heard it, what it has revealed is that there is a flaw in the way that you process probabilistic information. And this flaw isn’t necessarily dissolved simply by understanding one circumstance under which it was revealed. This is where Bayes Theorem comes in. Bayes Theorem is an approach that increases your general skills of probabilistic reasoning.
So Bayes theorem, remember, is about conditional probability: The probability that A is true given the evidence B.
Bayes Theorem makes conditioning on the situation (right door/wrong door) explicit. It would still be possible to know Bayes Theorem but not think to use it in this case. But if you approach probabilistic questions with Bayes Theorem in mind then you’re explicitly primed to ask whether there is a conditional approach to the problem. Here’s a Bayes Theorem based approach to the Monty Hall Problem.
So what we want to discover is the probability that the unselected and unopened door contains the prize given the information of the choice and the opening of door.
We now calculate the various components:
Because the host never selects the door you choose or the one with the prize.
Because the position of the prize had an initial probability of 1/3 and this doesn’t change based on the choice.
Now plug these in:
That’s the right answer (the probability that the prize is behind the door that has not been chosen or opened is 2/3 so you should change your choice)
But the important thing about Bayes Theorem isn’t that it gets the right answer, it’s that its use improves your ability to reason with probability so you’re less likely to get other probability questions wrong in the future. Simply by plugging in what you want to know and what you do know, this equation can improve your abilities and not just give you the answer to a specific question.
The limits of Bayes Theorem
Bayes Theorem is more than a solution to a specific problem – it’s a general technique for reasoning with probability. That’s not to say it’s a solve all solution. Some problems have the complexity in calculating those components above and, in those cases, Bayes Theorem won’t help. However, it’s a big step toward being able to solve problems where the components are simple but it’s the coming together of these which is complicated (see Greg Egan’s website for an example of a problem that Bayes Theorem may not help you with).
The next post is Bayes Theorem in practice: Kolmogorov Complexity