Instrumenting While Experimenting: An Empirical Method for Competitive Pricing at Scale

Accurate operational decisions require precise knowledge of the causal effects of such decisions on outcomes, a task that becomes increasingly complex in dynamic business environments. We propose an idea of “instrumenting while experimenting,” whereby researchers can create their own instruments by “injecting” small, random variations directly into the decision-making process and then use such variations to obtain causal estimates of the impact of varying business decisions at scale without disrupting everyday operations. To illustrate the effectiveness of this idea, we partner with a leading U.S. e-commerce retailer and develop a competitive pricing method in the context of increasing competition in online retailing. Our method allows retailers to respond more accurately to competitors’ price changes at scale. Operationally, we first construct a parsimonious demand model to capture the key trade-offs in competitive pricing. This model accounts for potential shifts in customer behaviors based on whether the focal retailer holds a price advantage relative to its competitors. Next, we design and implement a large-scale randomized price experiment on over 10,000 products. Leveraging the experiment as well as the control function approach, we are able to obtain unbiased estimates of key pricing components in the demand model, in particular, price elasticities of customers in both price advantage and disadvantage regions as well as the sales lift when undercutting competitors in price. Lastly, we recommend price responses by solving a constrained optimization problem that uses the estimated demand model as an input. We test this pricing method through another large-scale controlled field experiment on over 10,000 products and demonstrate significant improvements—increasing revenue by over 15% and increasing profit by over 10%. Simulation analyses reveal that these improvements are attributable to the joint implementation of demand modeling (contributing 17% of the total improvement), price optimization (36%), and our proposed estimation method (48%).