Testing for Manipulation: Experimental Evidence on Dark Patterns

Several countries and supranational authorities are debating whether to regulate or ban dark patterns, deceptive users’ interfaces. A key empirical component to this debate is how to assess manipulation. In this study, we develop a transaction test which measure to what extent the dark patterns lead to decisions inconsistent with elicited preferences. We conducted a large preregistered online study (N=7430) with a representative population of six countries to identify both the effect of dark patterns on consumers’ choice consistency and the potential counteracting effects of protective measures. Our treatments include three dark patterns - hiding information on the product, toying-with-emotions, and the use of psychological profiling to personalize the display for the consumer – and two versions of a protective measure that discloses information and requires subject to confirm the selection. Participants are assigned to either a motivated delay or incentive compatible time pressure environment, allowing to identify the impact of treatments on consumers paying enough attention and on situationally vulnerable consumer. Dark patterns do manipulate consumers, showing remarkable effects on both average and vulnerable consumers. The cool down intervention has a null effect. We stress test the transaction test in a controlled experiment, where the preference elicitation is incentive compatible, we collect repeated measurement of choices among lotteries and we manipulate the extent of the mistake. In this additional experiment, the TWE treatment resulted in greater inconsistency compared to the control group, particularly in lotteries where the point of indifference was less likely to be located at the boundaries of the MPL grid. While subjects learned to be consistent through multiple rounds of choice and with decision problems further from their area of indifference, the learning effect is less pronounced under the TWE treatment.