BAYESIAN ANALYSIS

Bayesian Analysis

Bayesian Analysis

1. Introduction

Named after the 18th-century English cleric Thomas Bayes, the Bayesian approach refers to a distinctive framework for decision making. Accepting the dictum that “probability is the guide to life,” the Bayesian approach provides a model for rational choice in which the expected utility of an action is determined in relation to a person's notions of the probabilities and utilities associated with the potential outcomes of the action under consideration. In considering alternative courses of action, the Bayesian principle is to choose an action with the greatest expected utility. The Bayesian approach has been widely influential in the development of rational choice theory and has been used in the study of rational choice in diverse disciplines, including management science and economics. Three elements central to the Bayesian approach include the Bayesian account of belief, rationality, and learning.

1.1. Bandit problems

Imagine you are staying in an unfamiliar city for a few weeks, and are consigned to eat alone each evening. There are a number of Chinese restaurants, all with essentially the same menu, and all within a short walk of your hotel. Each menu is cheap enough (or your expense account is large enough) that whether the meal is 'good' or 'bad' acts as the sole criterion for choosing one restaurant over the other. Over the course of your stay, a natural dining goal would be to maximize the number of good meals. In the first few days, pursuing this goal might involve trying a number of the restaurants. If the meal on the first night was bad, it seems unlikely you would re-visit that restaurant on the second night.

Towards the end of your stay, however, it becomes increasingly likely you will visit the same reasonably good restaurant repeatedly, even if it does not produce good meals every night. There is less incentive to explore options about which you are less certain and more incentive to exploit options which are reasonably good, and about which you are more certain. Because of the limited nature of your stay, how you search the environment of restaurants is likely to move from an initial exploration phase to a mature exploitation phase.

Another source of information that will affect your decision-making is any potential knowledge you have about the quality of Chinese restaurants in the city. If the city has a reputation for having many excellent Chinese restaurants, then a smaller number of bad meals will encourage the trying of alternatives. On the other hand, if you believe the city does not have many good Chinese restaurants, a relatively modest success rate at one place might encourage repeat dining.

It seems clear that your decision-making in trying to maximize the number of good meals you have is a non-trivial optimization problem. Choices must be sensitive to previous dining outcomes, how many days of your stay remain, what you know about the relevant base-rates for good and bad meals, and the interaction of all these factors.

This real world decision-making scenario has the same essential characteristics as a ...