Machine Learning, Behavioral Targeting and Regression Discontinuity Designs

The availability of behavioral data on customers and advances in machine learning methods have enabled scoring and targeting of customers in a variety of domains, including pricing, advertising, recommendation and personal selling. Typically, such targeting involves first training a machine learning algorithm on a training dataset, using that algorithm to score current or potential customers, and when the score crosses a threshold, a treatment such as an offer, an advertisement or a recommendation is assigned. In this paper, we highlight regression discontinuity designs (RDD) as a low-cost alternative to obtaining causal estimates in settings where machine learning is used for behavioral targeting. Our investigation leads to several new insights. Under appropriate conditions, RDD recovers the local average treatment effect (LATE). Further, we show that RDD recovers the average treatment effect (ATE) when: (1) The score is orthogonal to the slope of the treatment and (2) When the selection threshold is equal to the mean value of the score. We also show that RDD can estimate the bounds on the ATE even if we are unable to get point estimates of the ATE. That RDD can estimate ATE or bounds on ATE is a novel perspective that has been understudied in the literature. We also distinguish between two types of scoring: Intercept versus slope based and highlight the practical value of RDD in each context. Finally, we apply RDD in an empirical context where a machine learning based score was used to select consumers for retargeted display advertising. We obtain LATE estimates of the impact of the retargeted advertising program on both online and offline purchases, and also estimate bounds on the ATE. Our LATE estimates and ATE bounds add to the understanding of the effectiveness of retargeting programs in particular on offline purchases which has received less attention.