Dirichlet process mixtures of order statistics with applications to retail analytics

The rise of ‘big data’ has led to the frequent need to process and store data sets containing large numbers of high dimensional observations. Because of storage restrictions, these observations might be recorded in a lossy‐but‐sparse manner, with information collapsed onto a few entries which are considered important. This results in informative missingness in the observed data. Our motivating application comes from retail analytics, where the behaviour of product sales is summarized by the price elasticity of each product with respect to a small number of its top competitors. The resulting data are vectors of order statistics, because only the top few entries are observed. Interest lies in characterizing the behaviour of a product's competitors, and clustering products based on how their competition is spread across the market. We develop non‐parametric Bayesian methodology for modelling vectors of order statistics that utilizes a Dirichlet process mixture model with an exponentiated Weibull kernel. Our approach allows us added flexibility for the distribution of each vector, while providing parameters that characterize the decay of the leading entries. We implement our methods on a retail analytics data set of the cross‐elasticity coefficients, and our analysis reveals distinct types of behaviour across the different products of interest.