On the dangers of modelling through continuous distributions: A Bayesian perspective

We point out that Bayesian inference on the basis of a given sample is not always possible with continuous sampling models, even under a proper prior. The reason for this paradoxical situation is explained, and linked to the fact that any dataset consisting of point observations has zero probability under a continuous sampling distribution. A number of examples, both with proper and improper priors, highlight the issues involved. A solution is proposed through the use of set observations, which take into account the precision with which the data were recorded. Use of a Gibbs sampler makes the solution practically feasible. The case of independent sampling from (possibly skewed) scale mixtures of Normals is analysed in detail for a location-scale model with a commonly used noninformative prior. For Student-t sampling with unrestricted degrees of freedom the usual inference, based on point observations, is shown to be precluded whenever the sample contains repeated observations. We show that Bayesian inference based on set observations, however, is possible and illustrate this by an application to a skewed dataset of stock returns.