A comparison of partial profile designs for discrete choice experiments with an application in software development

In a discrete choice experiment, each respondent chooses the best product or service sequentially from many groups or choice sets of alternative goods. The alternatives, called profiles, are described by level combinations from a set of predefined attributes. Respondents sometimes make their choices on the basis of only one dominant attribute rather than making trade-offs among all the attributes. For example, in studies involving price as an attribute, respondents may always choose the profile with the lowest price. Also, a choice task including many attributes may encourage respondent decisions that are not fully compensatory. To thwart these behaviors, the investigator can hold the levels of some of the attributes constant in every choice set. The resulting designs are called partial profile designs. In this paper, we construct D-optimal partial profile designs for estimating main-effects models. We use a Bayesian design algorithm that integrates the D-optimality criterion over a prior distribution of likely parameter values. To determine the constant attributes in each choice set, we provide three alternative generalizations of an approach that makes use of balanced incomplete block designs. Each of our three generalizations constructs partial profile designs accommodating attributes with any number of levels and allowing flexibility in the numbers of choice sets and constant attributes. We show results from an actual experiment in software development performed using one of these algorithms. Finally, we compare the algorithms with respect to their statistical efficiency and ability to avoid failures due to the presence of a dominant attribute.