Optimal adaptive Bayesian design in choice experiments: performance for the population versus individuals

The literature on conjoint analysis has examined the performance of adaptive designs of choice experiments, comparing them to simpler static designs. In assessing the “estimation accuracy” of designs, however, this literature has focussed on estimators of individual parameters rather than population parameters. In contrast, our paper assesses not only estimation accuracy for individual parameters but also for population parameters. In particular, we compare the performance of an optimal adaptive Bayesian design to a simple, crude design in which questions are selected randomly. We find that, for the population parameters, the estimation accuracy of the adaptive Bayesian design is uniformly worse than the random design. More generally, we conclude that this adaptive Bayesian design is more suitable when the researcher’s focus is on the specific individuals in the sample. When, however, the focus is on the population, this adaptive Bayesian design does not have a clear advantage over a random design.