An introduction to decision theory

This is part 1 of a sequence titled “Less Wrong and decision theory”

Less Wrong collects together fascinating insights into a wide range of fields. If you understood everything in all of the blog posts, then I suspect you’d be in quite a small minority. However, a lot of readers probably do understand a lot of it. Then, there are the rest of us: The people who would love to be able to understand it but fall short. From my personal experience, I suspect that there are an especially large number of people who fall into that category when it comes to the topic of decision theory.

Decision theory underlies much of the discussion on Less Wrong and, despite buckets of helpful posts, I still spend a lot of my time scratching my head when I read, for example, Gary Drescher’s comments on Timeless Decision Theory. At it’s core this is probably because, despite reading a lot of decision theory posts, I’m not even 100% sure what causal decision theory or evidential decision theory is. Which is to say, I don’t understand the basics. I think that Less Wrong could do with a sequence that introduces the relevant decision theory from the ground up and ends with an explanation of Timeless Decision Theory (and Updateless Decision Theory). I’m going to try to write that sequence.

What is a decision theory?

In the interests of starting right from the start, I want to talk about what a decision theory is. A decision theory is a formalised system for analysing possible decisions and picking from amongst them. Normative decision theory, which this sequence will focus on, is about how we should make decisions. Descriptive decision theory is about how we do make decisions.

Decision theories involves looking at the possible outcomes of a decision. Each outcome is given a utility value, expressing how desirable that outcome is. Each outcome is also assigned a probability. The expected utility of taking an action is equal to the sum of the utilities of each possible outcome multiplied by the probability of that outcome occuring. To put it another way, you add together the utilities of each of the possible outcomes but these are weighted by the probability so that if an outcome is less likely, the value of that outcome is taken into account to a lesser extent.

Before this gets too complicated, let’s look at an example:

Let’s say you are deciding whether to cheat on a test. If you cheat, the possible outcomes are, getting full marks on the test (50% chance, 100 points of utility – one for each percentage point correct) or getting caught cheating and getting no marks (50% chance, 0 utility).

We can now calculate the expected utility of cheating on the test:

(1/2 * 100) + (1/2 * 0) = 50 + 0 = 50

That is, we look at each outcome, determine how much it should contribute to the total utility by multiplying the utility by its probability and then add together the value we get for each possible outcome.

So, decision theory would say (questions of morality aside) that you should cheat on the test if you would get less than 50% on the test if you didn’t cheat.

Those who are familiar with game theory may feel that all of this is very familiar. That’s a reasonable conclusion: A good approximation of what decision theory is that it’s one player game theory.

What are causal and evidential decision theories?

Two of the principle decision theories popular in academia at the moment are causal and evidential decision theories.

In the description above, when we looked at each action we considered two factors: The probability of it occurring and the utility gained or lost if it did occur. Causal and evidential decision theories differ by defining the probability of the outcome occurring in two different ways.

Causal Decision Theory defines this probability causally. That is to say, they ask, what is the probability that, if action A is taken, outcome B will occur. Evidential decision theory asks what evidence the action provides for the outcome. That is to say, it asks, what is the probability of B occurring given the evidence of A. These may not sound very different so let’s look at an example.

Imagine that politicians are either likeable or unlikeable (and they are simply born this way – they cannot change it) and the outcome of the election they’re involved in depends purely on whether they are likeable. Now let’s say that likeable people have a higher probability of kissing babies and unlikeable people have a lower probability of doing so. But this politician has just changed into new clothing and the baby they’re being expected to kiss looks like it might be sick. They really don’t want to kiss the baby. Kissing the baby doesn’t itself influence the election, that’s decided purely based on whether the politician is likeable or not. The politician does not know if they are likeable.

Should they kiss the baby?

Causal Decision Theory would say that they should not kiss the baby because the action has no causal effect. It would calculate the probabilities as follows:

If I am likeable, I will win the election. If I am not, I will not. I am 50% likely to be likeable.

If I don’t kiss the baby, I will be 50% likely to win the election.

If I kiss the baby, I will be 50% likely to win the election.

I don’t want to kiss the baby so I won’t.

Evidential Decision Theory on the other hand, would say that you should kiss the baby because doing so is evidence that you are likeable. It would reason as follows:

If I am likeable, I will win the election. If I am not, I will not. I am 50% likely to be likeable.

If I kissed the baby, there would be an 80% probability that I was likeable (to choose an arbitrary percentage).

If I did not kiss the baby, there would be a 20% probability that I was likeable.

Therefore:

Given the action of me kissing the baby, it is 80% probable that I am likeable and thus the probability of me winning the election is 80%.

Given the action of me not kissing the baby, it is 20% probable that I am likeable and thus the probability of me winning the election is 20%.

So I should kiss the baby (presuming the desire to avoid kissing the baby is only a minor desire).

This is making it explicit but the basic point is this: Evidential Decision Theory asks whether an action provides evidence for the probability of an outcome occuring, Causal Decision Theory asks whether the action will causally effect the probability of an outcome occuring.

The question of whether either of these decision theories works under all circumstances that we’d want them to is the topic that will be explored in the next few posts of this sequences.

Appendix 1: Some maths

I think that when discussing a mathematical topic, there’s always something to be gained from having a basic knowledge of the actual mathematical equations underpinning it. If you’re not comfortable with maths though, feel free to skip the following section. Each post I do will, if relevant, end with a section on the maths behind it but these will always be separate to the main body of the post – you will not need to know the equations to understand the rest of the post. If you’re interested in the equations though, read on:

Decision theory assigns each action a utility based on the sum of the probability of each outcome multiplied by the utility from each possible outcome.  It then applies this equation to each possible action to determine which one leads to the highest utility. As an equation, this can be represented as:

Where U(A) is the utility gained from action A. Capital sigma, the Greek letter, represents the sum for all i, P_i represents the probability of outcome i occurring and D_i, standing for desirability, represents the utility gained if that outcome occurred. Look back at the cheating on the test example to get an idea of how this works in practice if you’re confused.

Now causal and evidential decision theory differ based on how they calculate P_i. Causal Decision Theory uses the following equation:

In this equation, everything is the same as in the first equation except, in the section referring to probability is, the probability is calculated as the probability of O_i occurring if action A is taken.

Similarly, Evidential Decision Theory uses the following equation:

Where the probability is calculated based on the probability of O_i given that A is true.

If you can’t see the distinction between these two equations, then think back to the politician example.

Appendix 2: Important Notes

The question of how causality should be formalised is still an open one, see cousin_it’s comments on the original post at Less Wrong. As an introductory level post, we will not delve into these questions here but it is worth noting their is some debate on how exactly to interpret causal decision theory.

This post was originally posted to Less Wrong. Visit that post to see comments from readers there.

The next post is “Newcombe’s Problem: A problem for Causal Decision Theories”.