Home > An introduction to decision theory > What is decision theory?

## What is decision theory?

This is part of the sequence, “An introduction to decision theory”. While I have previously written about decision theory in relation to the views of a specific online community, this will be a broader and deeper introduction to decision theory starting from the basics and moving to more recent issues under discussion.

At some point in the history of the universe, a decision had never been made. The entire history of the universe to that point had just unfurled quietly without a single choice being made. The first warning sign that this era was over was the beginning of life. However, early living things would have floated or stayed still or did whatever the world told them to. They would not have intervened in the world or pondered about the rational response to the environment.  Not only could it not decide but, if it could have, it would still have been unable to impose its will on the world.

Then a form of life developed that was able to interact with the world. Maybe it gained the power of locomotion. Maybe it gained the power to cling – to decide when not to be moved by the elements. Maybe it gained any number of abilities but, for whatever reason, suddenly it was able to intervene in the world. And a new question came about: how should it act so as to achieve this. This first intervener would have had few cognitive tools to process this question.

The next development is the most important one to our story. Not first life. Not first intervention. But first decision. A form of life that could not only intervene in the world but could decide how to do so. Life that could choose where to move and when to move. And the question became more important: how should one best take advantage of this ability to decide?

This sequence will explore decision theory, one attempt to answer this question, at least in the abstract.

So imagine then a creature – maybe not the first decison maker but one of its descendents. This is a simple creature that gains its energy by eating algae that floats in the water and that survives in virtue of both eating enough and by avoiding being eaten by its preditors. This creature is faced with a decision: to its left there is a substantial patch of algae that would feed it comfortably for some time. To its right, there is a less substantial patch that it could nevertheless survive on, albeit not for as long. The creature is faced with the decision of which patch to approach.

However, here’s the complication: the more substantial path of algae is in an area that would be more exposed to predators if they were around. The less substantial patch is in a more sheltered area. In other words, if preditors are likely to be around then the creature would be better going to the less substantial patch. If they are unlikely to be around, it is better going to the more substantial patch. This allows us to introduce our first tool from decision theory: the utility table.

 Predators present Predators absent Substantial patch -10 5 Less substantial patch 2 2

This utility table contains almost everything that decision theory uses to determine the rational decision. Across the top are the possible states of the world (the world can either be such that the predators are present or absent). Along the side are the possible decisions (the creature can head for either the substantial or the less substantial patch). The table cells then contain a utility value for each possible combination of a decision and world state (so if the creature headed to the substantial patch and predators were present then this would be worth a utility of -10). The utility value is simply a measure of how much the decision maker values the outcome. From this table, the rational decision can be determined given the state of the world. So if the predators are present, then the less substantial path is clearly the best option.

Generally, though, decision theory deals with decision making under conditions of uncertainty. This means it deals with circumstances where the state of the world isn’t known. So, our creature might not know whether there are predators around today. In this case, it could make its judgement based on how likely the predators are to be around. This is the final piece of information that decision theory requires: the probablity that the world is in a certain state.

It combines all of this information together into a single formula for each decision which calculates the expected utility of each action. The action with the highest expected utility is the rational decision. (ETA: Those who have studied decision theory before might be expecting the probability here to take into account the decision in some way. This issue will be discussed in the next post).

$Expected \ Utility (Decision) =\sum_{i}Probability(WorldState_{i})\times Utility(WorldState_{i}\ \wedge \ Decision )$

What does this equation mean? Well basically, we take all of the utilities for each possible decision and add them together. So in the case of the creature deciding to go for the substantial path this is (-10 + 5 = -5) and for the unsubstantial patch is (2 +2 = 4). However, if the state of the world leading to that utility is unlikely to happen, we want that utility to count for less because the creature is unlikely to get it. So all of the utility is weighted by the probability of receiving it (in other words, the probability of the world state occuring as the world state determines what utility the creature will receive).

This is decision theory in its basic form.

The next post will look at a problem with this approach.

1. May 20, 2011 at 6:50 pm

Um, shouldn’t that sum be over P(WorldState | Decision) * U(WorldState) or perhaps P(Decision -> WorldState) * U(WorldState)?

• May 22, 2011 at 2:56 am

Hey Eudaemon,

You’re totally right about the equation presented in this post being limited (this is Savage’s equation of unconditional expected utility). However, this wasn’t an error – this is a starting point from which later versions of decision theory can be developed. The next post will deal with the shortcomings in this current approach.

In theory I could have moved straight to one of the later equations but there’s a number of problems with that:

1.) I think its more clear to start from this point and to show that this approach isn’t sophisticated enough so that the motivation behind later debates in decision theory is clear.

2.) The question of which equation to use is a big one. To jump straight into those issues and explain the important details would have at least doubled the length of the post. I prefer to take it bit by bit.

Thanks for the comment and feel free to point out anything that seems strange in the future (I have clarified slightly in the post to make it more clear).

Thanks,