Nonparametric Models

Parametric probability models provide convenient mathematical structures for approximating an individual’s uncertain beliefs. For example, simple probability distributions with a small number of parameters for modelling exchangeable random quantities (Chap. 4 ) or a linear model for regression-exchangeable observations of a response variable (Chap. 8 ). The appealing simplicity of parametric models also carries a severe limitation: having assumed a parametric model, no amount of observed data can undermine the assumed certainty that the probability distribution or regression function takes that parametric form with probability one. For small sample size problems, this limitation can often seem acceptable, but for larger sample sizes the opportunity for learning potentially more complex underlying relationships grows and parametric models can become prohibitively restrictive. More flexible modelling paradigms with the capacity to increase in complexity with increasing sample size are often referred to as nonparametricNonparametric methods. This name can appear somewhat misleading, as these methods typically allow access to a potentially infinite number of parameters to provide this growth in complexity. However, the term is used to imply modelling freedom away from assuming a fixed, finite-dimensional parametric form. The contrast between the two modelling paradigms is stark. Parametric models place probability one on a particular parametric functional form being true. Nonparametric models assume no such fixed relationship, but instead seek to spread probability mass across a much larger region of appropriate function space, such that positive mass will be assigned to arbitrarily small neighbourhoods surrounding any unknown true underlying function belonging to a much broader function class. The higher complexity of nonparametric models can lead to a loss of analytic tractability or an increase in computational burden when performing Bayesian inference. However, there are some notable exceptions, and the next two chapters provides an overview of some popular nonparametric formulations which can be readily deployed in practical applications, either for modelling probability distributions in the present chapter or regression functions in Chap. 10 .