Behavioral Targeting, Machine Learning and Regression Discontinuity Designs

The availability of behavioral and other data on customers and advances in machine learning methods have enabled targeting of customers in a variety of domains, including pricing, advertising, recommendation systems and personal selling contexts. Typically, such targeting involves first training a machine learning algorithm on a training dataset, and then using that algorithm to score current or potential customers. When the score crosses a threshold, a treatment (such as an offer, an advertisement or a recommendation) is assigned. In this paper, we demonstrate that this has given rise to opportunities for causal measurement of the effects of such targeted treatments using regression discontinuity designs (RDD). Investigating machine learning in a regression discontinuity framework leads to several insights. First, we characterize conditions under which regression discontinuity designs can be used to measure not just local average treatment effects (LATE), but also average treatment effects (ATE). In some situations, we show that RD can be used to find bounds on the ATE even if we are unable to find point estimates. We then apply this to the machine learning based targeting contexts by studying two different ways in which the score required for targeting is generated, and explore the utility of RDD to these contexts. Finally, we apply our approach in the empirical context of the targeting of retargeted display advertising. Using a dataset from a context where a machine learning based targeting policy was employed in parallel with a randomized controlled trial, we examine the performance of the RDD estimate in estimating the treatment effect, validate it using a placebo test and demonstrate its practical utility.