Criticism and Model Choice

In subjective probability, there are no right or wrong systems of beliefs, provided they are coherently specified; I have my own individual measures of uncertainty concerning any quantities that I am unsure of, and it is fully admissible that these could be arbitrarily different from the probability beliefs held by others. However, it was noted in the introductions to Chaps. 1 and 2 that mathematically specifying probability distributions which accurately represent systems of beliefs is a non-trivial exercise, and arguably always carries some degree of imprecision. The use of probability models, for example, incorporating assumptions of exchangeability and making the choice of the prior measure Q in de Finetti’s representation theoremDe Finettirepresentation theorem (Sect. 2.2 ), provides tractable approximations of underlying beliefs which at least possess the necessary coherence properties for rational decision-making. Furthermore, there is no philosophical requirement for subjective probability distributions to endure. They need only apply to the specific decision problem being addressed. Indeed, Bayes’ theorem provides the coherent procedure for updating beliefs with new information with respect to a previously stated belief system. But for the next decision, there are other alternatives. In particular, I might want to review my previous decisions and the consequent outcomes, and call into question whether I should adopt a different perspective. Such considerations can be referred to as model criticism and model selectionModel selection. In this chapter, it is supposed that the decision-maker may be considering a range of modelling strategies for representing probabilistic beliefs about a random variable X for an uncertain outcome $$\omega $$ ω , where for sufficient generality $$X:\varOmega \rightarrow \mathbb {R}^n$$ X : Ω → R n could represent a sequence of $$n\ge 1$$ n ≥ 1 real-valued observations. After observing the realised value $$x=X(\omega )$$ x = X ( ω ) , the decision-maker may want to re-evaluate which modelling strategy might have been most appropriate for capturing the true underlying dynamics which gave rise to x.