IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2602.23257.html

Randomization Tests in Switchback Experiments

Author

Listed:
  • Jizhou Liu
  • Liang Zhong

Abstract

Switchback experiments assign an experimental unit, such as a market or a platform, to treatment or control over successive blocks of time. Inference can be challenging because these experiments often contain only a small number of randomized blocks, while outcomes may exhibit serial dependence, seasonality, and treatment effects that persist across periods. We develop a conditional randomization test for the null of no total treatment effect that is finite-sample valid under standard assumptions on temporal interference without requiring a parametric model for outcomes. We also develop randomization tests for these assumptions: a carryover test that assesses whether past assignments continue to affect outcomes beyond a prespecified horizon and a non-anticipation test that assesses whether future assignments affect current outcomes. For hypotheses about average treatment effects, we establish asymptotic validity of studentized randomization tests under additional regularity conditions. Finally, we derive power approximations and characterize the tradeoffs between experimental design choices, the number of informative randomized comparisons, and statistical power. Numerical experiments with stylized potential outcomes and a dynamic rideshare model illustrate finite-sample performance and implications for experimental design.

Suggested Citation

  • Jizhou Liu & Liang Zhong, 2026. "Randomization Tests in Switchback Experiments," Papers 2602.23257, arXiv.org, revised Sep 2026.
  • Handle: RePEc:arx:papers:2602.23257
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2602.23257
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    2. Iavor Bojinov & David Simchi-Levi & Jinglong Zhao, 2023. "Design and Analysis of Switchback Experiments," Management Science, INFORMS, vol. 69(7), pages 3759-3777, July.
    3. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    4. Ramesh Johari & Hannah Li & Inessa Liskovich & Gabriel Y. Weintraub, 2022. "Experimental Design in Two-Sided Platforms: An Analysis of Bias," Management Science, INFORMS, vol. 68(10), pages 7069-7089, October.
    5. Guillaume Basse & Peng Ding & Avi Feller & Panos Toulis, 2024. "Randomization Tests for Peer Effects in Group Formation Experiments," Econometrica, Econometric Society, vol. 92(2), pages 567-590, March.
    6. Christensen, Peter & Osman, Adam, 2021. "The Demand for Mobility: Evidence from an Experiment with Uber Riders," IZA Discussion Papers 14179, IZA Network @ LISER.
    7. G W Basse & A Feller & P Toulis, 2019. "Randomization tests of causal effects under interference," Biometrika, Biometrika Trust, vol. 106(2), pages 487-494.
    8. Susan Athey & Dean Eckles & Guido W. Imbens, 2018. "Exact p-Values for Network Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 230-240, January.
    9. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    10. Stefan Wager & Kuang Xu, 2021. "Experimenting in Equilibrium," Management Science, INFORMS, vol. 67(11), pages 6694-6715, November.
    11. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, June.
    12. Cyrus J. DiCiccio & Joseph P. Romano, 2017. "Robust Permutation Tests For Correlation And Regression Coefficients," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1211-1220, July.
    13. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    14. Jason Wu & Peng Ding, 2021. "Randomization Tests for Weak Null Hypotheses in Randomized Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1898-1913, October.
    15. Jizhou Liu & Azeem M. Shaikh & Panos Toulis, 2025. "Randomization Inference in Two-Sided Market Experiments," Papers 2504.06215, arXiv.org, revised Mar 2026.
    16. Zhexiao Lin & Peng Ding, 2025. "Unifying regression-based and design-based causal inference in time-series experiments," Papers 2510.22864, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jizhou Liu & Azeem M. Shaikh & Liang Zhong, 2026. "Randomization Tests in Randomized Saturation Designs," Papers 2607.04257, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jizhou Liu & Azeem M. Shaikh & Panos Toulis, 2025. "Randomization Inference in Two-Sided Market Experiments," Papers 2504.06215, arXiv.org, revised Mar 2026.
    2. Jizhou Liu & Azeem M. Shaikh & Liang Zhong, 2026. "Randomization Tests in Randomized Saturation Designs," Papers 2607.04257, arXiv.org.
    3. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org, revised Apr 2025.
    4. Hongyu Chen & David Simchi-Levi, 2026. "Efficient Switchback Experiments with Surrogate Variables: Estimation and Experimental Design," Management Science, INFORMS, vol. 72(6), pages 4854-4870, June.
    5. David M. Ritzwoller & Joseph P. Romano & Azeem M. Shaikh, 2024. "Randomization Inference: Theory and Applications," Papers 2406.09521, arXiv.org, revised Feb 2025.
    6. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    7. Sadegh Shirani & Mohsen Bayati, 2025. "On Evolution-Based Models for Experimentation Under Interference," Papers 2511.21675, arXiv.org.
    8. Chonghuan Wang, 2026. "Experimental Design for Matching," Papers 2601.21036, arXiv.org.
    9. Han, Kevin & Basse, Guillaume & Bojinov, Iavor, 2024. "Population interference in panel experiments," Journal of Econometrics, Elsevier, vol. 238(1).
    10. Zhexiao Lin & Peng Ding, 2025. "Unifying regression-based and design-based causal inference in time-series experiments," Papers 2510.22864, arXiv.org.
    11. Shuze Chen & David Simchi-Levi & Chonghuan Wang, 2024. "Improving the Estimation of Lifetime Effects in A/B Testing via Treatment Locality," Papers 2407.19618, arXiv.org, revised Sep 2025.
    12. Michael Thomas & Marcel Goic & Kirthi Kalyanam, 2025. "Long Lags and Large Returns: Experimental Evidence from Advertising to Businesses," Management Science, INFORMS, vol. 71(10), pages 8750-8766, October.
    13. Iavor Bojinov & David Simchi-Levi & Jinglong Zhao, 2023. "Design and Analysis of Switchback Experiments," Management Science, INFORMS, vol. 69(7), pages 3759-3777, July.
    14. Lemmens, Aurélie & Roos, Jason M.T. & Gabel, Sebastian & Ascarza, Eva & Bruno, Hernán A. & Gordon, Brett R. & Israeli, Ayelet & McDonnell Feit, Elea & Mela, Carl F. & Netzer, Oded, 2026. "Personalization and targeting: how to experiment, learn & optimize," International Journal of Research in Marketing, Elsevier, vol. 43(2), pages 537-559.
    15. Ruoxuan Xiong & Susan Athey & Mohsen Bayati & Guido Imbens, 2024. "Optimal Experimental Design for Staggered Rollouts," Management Science, INFORMS, vol. 70(8), pages 5317-5336, August.
    16. David Simchi-Levi & Chonghuan Wang, 2026. "Pricing Experimental Design: Causal Effect, Expected Revenue and Tail Risk," Management Science, INFORMS, vol. 72(2), pages 1157-1174, February.
    17. Luofeng Liao & Christian Kroer, 2024. "Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions," Papers 2406.15522, arXiv.org, revised Mar 2025.
    18. Zikun Ye & Zhiqi Zhang & Dennis J. Zhang & Heng Zhang & Renyu Zhang, 2026. "Deep Learning-Based Causal Inference for Large-Scale Combinatorial Experiments: Theory and Empirical Evidence," Management Science, INFORMS, vol. 72(7), pages 5611-5634, July.
    19. Julius Owusu, 2023. "Randomization Inference of Heterogeneous Treatment Effects under Network Interference," Papers 2308.00202, arXiv.org, revised Jun 2025.
    20. Xiaokang Luo & Tirthankar Dasgupta & Minge Xie & Regina Y. Liu, 2021. "Leveraging the Fisher randomization test using confidence distributions: Inference, combination and fusion learning," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(4), pages 777-797, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2602.23257. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.