How Does Treatment Self-Selection Affect Inferences About Political Communication?

Ecological validity is vital to experimental research because designs that are too artificial may not speak to any real-world political phenomenon. One such concern is treatment of self-selection: if individuals in the real-world self-select treatments, such as political communications, how well does the sample average treatment effect estimate the effects of message exposure for those individuals who would—if given the choice—opt-in to and out of receiving treatment? This study shows that randomization masks effect heterogeneity between individuals who would select different messages if given the choice. Yet, such selections are themselves complex, revealing additional challenges for realistically studying treatments prone to self-selection. The evidence of effect heterogeneity raises questions about the appropriateness of random assignment experiments for studying political communication and the results more broadly advance our understanding of citizens’ selection into and responses to communications when, as they often do, have choice over what messages to receive.