IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2505.13897.html
   My bibliography  Save this paper

Valid Post-Contextual Bandit Inference

Author

Listed:
  • Ramon van den Akker
  • Bas J. M. Werker
  • Bo Zhou

Abstract

We establish an asymptotic framework for the statistical analysis of the stochastic contextual multi-armed bandit problem (CMAB), which is widely employed in adaptively randomized experiments across various fields. While algorithms for maximizing rewards or, equivalently, minimizing regret have received considerable attention, our focus centers on statistical inference with adaptively collected data under the CMAB model. To this end we derive the limit experiment (in the Hajek-Le Cam sense). This limit experiment is highly nonstandard and, applying Girsanov's theorem, we obtain a structural representation in terms of stochastic differential equations. This structural representation, and a general weak convergence result we develop, allow us to obtain the asymptotic distribution of statistics for the CMAB problem. In particular, we obtain the asymptotic distributions for the classical t-test (non-Gaussian), Adaptively Weighted tests, and Inverse Propensity Weighted tests (non-Gaussian). We show that, when comparing both arms, validity of these tests requires the sampling scheme to be translation invariant in a way we make precise. We propose translation-invariant versions of Thompson, tempered greedy, and tempered Upper Confidence Bound sampling. Simulation results corroborate our asymptotic analysis.

Suggested Citation

  • Ramon van den Akker & Bas J. M. Werker & Bo Zhou, 2025. "Valid Post-Contextual Bandit Inference," Papers 2505.13897, arXiv.org.
  • Handle: RePEc:arx:papers:2505.13897
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2505.13897
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Maximilian Kasy & Anja Sautmann, 2021. "Adaptive Treatment Assignment in Experiments for Policy Choice," Econometrica, Econometric Society, vol. 89(1), pages 113-132, January.
    2. Nelson, Daniel B & Foster, Dean P, 1994. "Asymptotic Filtering Theory for Univariate ARCH Models," Econometrica, Econometric Society, vol. 62(1), pages 1-41, January.
    3. Jeganathan, P., 1995. "Some Aspects of Asymptotic Theory with Applications to Time Series Models," Econometric Theory, Cambridge University Press, vol. 11(5), pages 818-887, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Keisuke Hirano & Jack R. Porter, 2023. "Asymptotic Representations for Sequential Decisions, Adaptive Experiments, and Batched Bandits," Papers 2302.03117, arXiv.org, revised Feb 2025.
    2. Hirano, Keisuke & Porter, Jack R., 2020. "Asymptotic analysis of statistical decision rules in econometrics," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 283-354, Elsevier.
    3. Da Fonseca José & Grasselli Martino & Ielpo Florian, 2014. "Estimating the Wishart Affine Stochastic Correlation Model using the empirical characteristic function," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 18(3), pages 253-289, May.
    4. Asai, Manabu & McAleer, Michael, 2015. "Leverage and feedback effects on multifactor Wishart stochastic volatility for option pricing," Journal of Econometrics, Elsevier, vol. 187(2), pages 436-446.
    5. Gushchin, Alexander A. & Kuchler, Uwe, 1997. "Asymptotic inference for a linear stochastic differential equation with time delay," SFB 373 Discussion Papers 1997,43, Humboldt University of Berlin, Interdisciplinary Research Project 373: Quantification and Simulation of Economic Processes.
    6. Stentoft, Lars, 2005. "Pricing American options when the underlying asset follows GARCH processes," Journal of Empirical Finance, Elsevier, vol. 12(4), pages 576-611, September.
    7. Christoffersen, Peter & Heston, Steven & Jacobs, Kris, 2010. "Option Anomalies and the Pricing Kernel," Working Papers 11-17, University of Pennsylvania, Wharton School, Weiss Center.
    8. Doornik, Jurgen A. & Ooms, Marius, 2008. "Multimodality in GARCH regression models," International Journal of Forecasting, Elsevier, vol. 24(3), pages 432-448.
    9. Buccheri, Giuseppe & Corsi, Fulvio & Flandoli, Franco & Livieri, Giulia, 2021. "The continuous-time limit of score-driven volatility models," Journal of Econometrics, Elsevier, vol. 221(2), pages 655-675.
    10. Pentti Saikkonen & Rickard Sandberg, 2016. "Testing for a Unit Root in Noncausal Autoregressive Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 37(1), pages 99-125, January.
    11. Werker, Bas J.M. & Zhou, B., 2022. "Semiparametric testing with highly persistent predictors," Other publications TiSEM 2974ce9c-97c1-44cd-9331-0, Tilburg University, School of Economics and Management.
    12. Chao Qin & Daniel Russo, 2024. "Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification," Papers 2402.10592, arXiv.org, revised Jul 2024.
    13. McMillan, David G. & Speight, Alan E. H., 2001. "Non-ferrous metals price volatility: a component analysis," Resources Policy, Elsevier, vol. 27(3), pages 199-207, September.
    14. Marc Hallin & Bas Werker, 2003. "Semiparametric efficiency, distribution-freeness, and invariance," ULB Institutional Repository 2013/2119, ULB -- Universite Libre de Bruxelles.
    15. Masahiro Kato & Masaaki Imaizumi & Takuya Ishihara & Toru Kitagawa, 2023. "Asymptotically Optimal Fixed-Budget Best Arm Identification with Variance-Dependent Bounds," Papers 2302.02988, arXiv.org, revised Jul 2023.
    16. Neely, Christopher J., 1999. "Target zones and conditional volatility: The role of realignments," Journal of Empirical Finance, Elsevier, vol. 6(2), pages 177-192, April.
    17. Qihui Chen & Zheng Fang, 2019. "Inference on Functionals under First Order Degeneracy," Papers 1901.04861, arXiv.org.
    18. Aït-Sahalia, Yacine & Park, Joon Y., 2016. "Bandwidth selection and asymptotic properties of local nonparametric estimators in possibly nonstationary continuous-time models," Journal of Econometrics, Elsevier, vol. 192(1), pages 119-138.
    19. Drost, F.C. & Klaasens, C.A.J. & Werker, B.J.M., 1994. "Adaptive Estimation in Time Series Models," Papers 9488, Tilburg - Center for Economic Research.
    20. Werner Ploberger & Peter C.B. Phillips, 1998. "Rissanen's Theorem and Econometric Time Series," Cowles Foundation Discussion Papers 1197, Cowles Foundation for Research in Economics, Yale University.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2505.13897. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.