Admissible decision rules

In classical (frequentist) decision theory, an admissible decision rule is a rule for making a decision that is better in some sense than other rules that may compete with it. Generally speaking, in most decision problems the set of admissible rules is large, even infinite, but as will be seen there are good reasons to stick to admissible rules.

Define sets <math>\Theta,\mathcal{X}<math> and <math>\mathcal{A}<math>, where <math>\Theta\,\!<math> are the states of nature, <math>\mathcal{X}<math> the possible observations and <math>\mathcal{A}<math> the actions that may be taken. A decision rule is a function <math>\delta:{\mathcal{X}}\rightarrow {\mathcal{A}}<math>, i.e., upon observing <math>x\in \mathcal{X}<math>, we choose to take action <math>\delta(x)\,\!<math>.

In addition, we define a loss function <math>L: \Theta \times \mathcal{A} \rightarrow \Re<math> which measures the loss we incur by taking action <math>a \in \mathcal{A}<math> when the true state of nature is <math>\theta \in \Theta<math>. Usually we will take this action upon making an observation of data <math>x \in \mathcal{X}<math>, so that the loss will be <math>L(\theta,\delta(x))\,\!<math>.

It is possible to recast the theory in terms of a utility function, the negative of the loss. However, admissibility is usually defined in terms of a loss function, and we shall follow this convention.

Let <math>x\,\!<math> have cumulative distribution function <math>F(x|\theta)\,\!<math>. Define the risk function as the expectation

<math>R(\theta,\delta)=E^{\mathcal{X}}[{L(\theta,\delta(x))]}\,\!<math>

A decision rule <math>\delta^*\,\!<math> dominates a decision rule <math>\delta\,\!<math> if and only if <math>R(\theta,\delta^*)\le R(\theta,\delta)<math> for all <math>\theta\,\!<math>, and the inequality is strict for some <math>\theta\,\!<math>.

A decision rule is admissible if and only if no other rule dominates it; otherwise it is inadmissible. An admissible rule should be preferred over an inadmissible rule since for any inadmissible rule there is an admissible rule that performs at least as well for all states of nature and betters it for some.

Bayes Rules

Let <math>\pi(\theta)\,\!<math> be a cumulative probability distribution on the states of nature. From a Bayesian point of view, we would regard it as a prior distribution, that is, it is our believed probability distribution on the states of nature, prior to observing data. For a frequentist, it is merely a function on <math>\Theta\,\!<math> with no such special interpretation. The Bayes risk of the decision rule <math>\delta\,\!<math>with respect to <math>\pi(\theta)\,\!<math> is the expectation

<math>r(\pi,\delta)=E^\pi[R(\theta,\delta)]\,\!<math>.

If the Bayes risk is finite, we can minimize <math>r(\pi,\delta)\,\!<math> with respect to <math>\delta\,\!<math> to obtain <math>\delta^\pi(x)\,\!<math>, the Bayes rule with respect to <math>\pi(\theta)\,\!<math>. If the Bayes risk is infinite, then the Bayes rule is not defined.

Admissible Rules and Bayes Rules

In the Bayesian approach to decision theory, <math>x\,\!<math> is considered fixed. Instead of averaging over <math>\mathcal{X}\,\!<math> as in the frequentist approach, the Bayesian would average over <math>\Theta\,\!<math>. Thus, we would be interested in computing for our observed <math>x\,\!<math> the expected loss

<math>\rho(\pi,\delta)=E^{\pi} [ L(\theta,\delta(x)) ] \,\!<math>

Since <math>x\,\!<math> is considered fixed and known, we can minimize the expected loss for any <math>x\,\!<math>; by varying <math>x\,\!<math> over its range, we define a function <math>\delta^\pi(x)\,\!<math>, which is known as a generalized Bayes rule. The generalized Bayes rule will be the same as the Bayes rule (relative to <math>\pi\,\!<math>), provided that the Bayes risk is finite.

The interesting thing is that according to the complete class theorems, under rather general conditions every generalized Bayes rule is admissible, and every admissible rule is a (generalized) Bayes rule (with respect to some, possibly improper, prior). Thus, generally speaking, it is sufficient to consider only Bayes rules.

There are some restrictions. For example, as shown by the Stein effect, an improper prior can give rise to an inadmissible decision rule.

References

• James O. Berger Statistical Decision Theory and Bayesian Analysis. Second Edition. Springer-Verlag, 1980, 1985. ISBN 0–387–96098–8.
• Morris De Groot Optimal Statistical Decisions. Wiley Classics Library. 2004. (Originally published 1970.) ISBN 0–471–68029-X.
• Christian P. Robert The Bayesian Choice. Springer-Verlag 1994. ISBN 3–540–94296–3.