Toward a better understanding of elicitation effects in stated preference studies

Empirical evidence accumulated over several decades suggests that survey-based welfare measures for public goods can be very sensitive to the format of the value elicitation, e.g., an up-or-down vote or an open-ended willingness-to-pay question. The underlying drivers of these effects remain poorly understood. As myriad formats are employed in practice, this raises concerns for both academics and policymakers. We design and implement a controlled experiment to cleanly test for elicitation effects among a set of four oft-used formats: single binary choice, double-bounded binary choice, payment card, and open-ended. The experiment retains important field context properties (e.g., the funding of a public, environmental good) and varies only the elicitation format, while holding fixed ancillary characteristics of the elicitations (such as framing, decision rule, payment method, and incentive compatibility). We find all formats lead to statistically identical welfare estimates. On one hand, this evidence suggests that variance in design characteristics other than the elicitation format may explain some prior results. On the other, to the extent that characteristics of our elicitations can be mirrored in the field, this offers a pathway for mitigating elicitation effects.