Thursday, March 13, 2008

The illusion of suboptimality and irrationality

Many times we feel that the world is highly irational and suboptimal (i.e. things can be done is a much better way). This is sometimes the case, but maybe less than as it seems. Our eyes are very misleading about it.

The reason is because the hypothetical reality (that is before any decision has been made) contains a hugte space of possibilities. The decision and actions people take are usually relatively good among the overal space of possibilities.

But the decisions people take may not be the absolute optimum. That is they are not the very best set of decisions. But they are still very good.

After the fact thinking will start off with what decisions already made, and try to look for alternatives based on this optimized decision. Insofar that the decision was not the absolute optimal it will look like there are better decisions and the decision maker was a fool.

Suppose there are a billion possible compositions one can lead to by various combinations of decisions. Suppose further that we sort them throught a single measure. If the actual decision ranked 100th, it should be a great decision, but starting from there one will see that it is the worst out of hundred possibilities.

This mistake has various kinds.:
Ignoring the fact that there were a lot of worse decisions.
Perceiving the "better" decisions as many and forgetting that among the original set of possibilities these super optimal decisions were a tiny unseen minority.
Forgetting possible drawbacks of the super-optimal decisions that just seem irelevant from a current point of view.

Intuitively it feels that there is a deeper side to it. Focusing on theis ex-post space seriousely distorts the whole situation.


i.c. said...

...just some quick first thoughts, crossing my mind:
- one fallacy of the whole argument is that there is hardly an evident possibility to rank alternatives if you take it seriously (of course, you can create a random metric, or accept a social one, or a historical one)
- on the other hand: the idea to be happy with a "good" suboptimal one (and not look further) is certainly quite important; and the idea about the bias is great (though one should not think about past decisions anyway, at least not in a judging way - maybe in a learning way)

Yechezkel Zilber said...


"One fallacy...." It is certainly impossible to rank alternatives reliably. My argument was a theoretical one, and as such it does not depent on its applicability. The idea is valid.

"on the other hand....." it is irelevant. Because the "optimality" I talk about is personal utility wise. As such, if someone is as happy as possible, its actually optimal. Suboptimality of other measures is irelevant.

PS said...

Your page is always interesting.I lifted a lot from this post at my blog.

I'm curious: have you been happy recently, and if so, why?

Yechezkel Zilber said...

Hey Paul,

Thanks for jumping in and for the link.

I am posting somewhat irregularly, but you are welcome here