Statistical inference for online decision making with Lasso loss function: In a contextual multi-armed bandit setting

The paper focuses on the statistical inference of contextual bandit problem with least absolute shrinkage and selection operator (Lasso) loss function. We propose a completely online decision making algorithm which balances the exploration and exploitation trade-off through the ɛ-greedy policy and updates the decision policy online in combination with the stochastic dual averaging algorithm. We establish the asymptotic normality of the estimator for the linear regression parameter in Lasso both in ergodic sense and non-ergodic sense. Then the plug-in method and batch-mean method are extended to estimate the covariance matrix of the normal distribution. We also analyze the asymptotic normality of the online inverse probability weighted value estimator. Simulations and real data applications on music artist recommendation and web recommendation are conducted to illustrate the performance of the proposed algorithm and theoretical results.